JP2012133707A - Image processing device and control method of image processing device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To generate image in which grain noises or scratches are added to an input image as random noise.SOLUTION: When adding grain noises taken from two-dimensional noise data, the positional difference from the previous position is evaluated every time when taking the noises. When the difference is too small, the taking position is changed. When adding scratches taken from noise data of plural patterns and when the condition is to continuously add the scratches for a certain period of time, the positional difference from the previous position is evaluated with respect to the adding position of the scratches. When the difference is too large, the addition is determined as invalid.

Description

  The present invention relates to an image processing apparatus that gives a film-like effect to digital image data.

  In recent years, as one of the image expressions of a digital camera, there has been proposed a synthesis method for adding granular noise (hereinafter referred to as grain noise) to a digital image in order to produce a grainy effect by film shooting on the captured image.

  In Patent Document 1, granular pattern data that is the source of grain noise is calculated from a film, and a plurality of these are arranged at random positions according to the image size, thereby adding spatially random noise as grain noise. Have

Japanese Patent Laid-Open No. 11-085955

  When adding grain noise to a moving image in a digital camera, it is necessary to change the graininess of the grain noise randomly in addition to being spatially randomly arranged.

  The present invention has been made in view of the above-described problems, and adds noise data cut out from pattern data to an image, and gives an effect that looks like random noise spatially and temporally when giving a film-like effect. It is to provide a processing apparatus.

  The image processing apparatus of the present invention is an image processing apparatus capable of giving a film-like grain effect to an input image of a plurality of frames, and storage means for storing noise data composed of two-dimensional data; Cutout means for cutting out noise data of a predetermined size from the noise data, designation means for designating a cutout position on the noise data cut out by the cutout means, and the previous predetermined on the noise data designated by the designation means Changing means for changing the current cut position specified by the specifying means when the distance between the cut position of the noise data of the size and the cut position of the noise data of the predetermined size is within a predetermined range; and the change The means is such that the distance between the changed current cutting position and the previous cutting position is not within the predetermined range. Change, characterized in that it comprises a synthesizing means for synthesizing the image to be the input noise based on the noise data of a predetermined size was cut out by the cutout section.

  Further, the image processing apparatus of the present invention is an image processing apparatus capable of giving an image effect of a scratched wind of film to an input image of a plurality of frames, and a storage means for storing noise data of a plurality of patterns, A selection unit that selects noise data to be pasted to the image of the current frame from the noise data of the plurality of patterns, and a determination to determine a position to paste the noise data selected by the selection unit on the image of the current frame Means for combining the noise data selected by the selecting means at the position determined by the determining means, and the position at which the noise data is pasted on the image of the previous frame by the combining means. The distance from the position where the noise data is pasted to the image of the current frame determined by the determining means is within a predetermined range. Is larger, characterized in that it comprises a disabling means for disabling the synthesis of the noise data to the image of said current frame by said synthesis means.

  As described above, according to the present invention, there is provided an image processing apparatus that adds noise data cut out from pattern data to an image and gives an effect that looks like random noise spatially and temporally when giving a film-like effect. It is to provide.

1 is a block diagram illustrating an example of an image processing apparatus according to a first embodiment. A block diagram showing an example of an image processing apparatus according to a second embodiment The figure which shows the noise data of the image processing apparatus which is 1st Embodiment. The figure which shows the structure of the synthesized image of the image processing apparatus which is 1st Embodiment. The figure which shows the noise data of the image processing apparatus which is 2nd Embodiment. The figure which shows the structure of the synthesized image of the image processing apparatus which is 2nd Embodiment. Flowchart of noise data cutout position determination in the first embodiment The figure which shows the cutout position of the noise data in 1st Embodiment The table | surface which shows the position after correction | amendment of the cut-out position of the noise data in 1st Embodiment The figure which shows the duplication degree by the difference in the correction position of the noise data in 1st Embodiment Flowchart for determining the paste position of noise data in the second embodiment The figure which shows a time change whether the necessity of addition of the scratching scratch in 2nd Embodiment is required.

  Hereinafter, preferred embodiments of the present invention will be described.

<First Embodiment>
The present embodiment is an image processing apparatus capable of adding grain noise, which is granular noise, to an image in order to give a granular feeling as a film-like noise effect.

  FIG. 1 is a block diagram of a digital video camera as an image processing apparatus according to the first embodiment. The image sensor 100 photoelectrically converts the incident light that has been imaged. The photoelectrically converted signal is input to the camera signal processing unit 101. The camera signal processing unit 101 performs various kinds of image quality processing on the photoelectrically converted signal, converts it into a video signal, and outputs it. The output video signal is encoded by the encoder 111 in a predetermined recording format and recorded on the recording medium 112.

  Next, signal processing in the camera signal processing unit 101 will be described. The signal input to the signal processing unit 101 is first subjected to various signal processing by the image processing unit 104 for each of the luminance component, the color component, and the black component. The processed signal is stored in the first frame memory 108 as a captured image.

  The two-dimensional 109 is a memory that stores two-dimensional noise data 301 as grain noise. The noise data 301 is read from the memory 109. The cutout processing unit 106 cuts out noise data 302 having a predetermined position and size from the memory data 301. The cut out noise data 302 is stored in the second frame memory 110. The noise data 302 read from the second frame memory 110 is resized by the enlargement processing unit 107 into noise data 402 having a size necessary for synthesis with the captured image.

  The synthesis processing unit 105 reads the captured image 401 stored in the first frame memory 108 and the noise data 402 processed by the enlargement processing unit 107 at a predetermined timing, performs synthesis, and outputs a synthesized image. The composition processing unit 105 can change the composition ratio of the captured image 401 and the noise data 402. The intensity of the grain noise added can be changed by changing the composition ratio. A method for generating and adding grain noise will be described in detail later.

  The signal generator 103 is a signal generator (Signal Generator) that generates a signal for controlling the drive timing of the image sensor 100. The generated signal is supplied to the image sensor 100.

  A system controller 102 controls a camera block including the image sensor 100 and the camera signal processing unit 101. A signal accumulation period and readout timing are instructed to the image sensor 100. For the image processing unit 104, parameters necessary for image quality setting are set by various signal processing. For the synthesis processing unit 105, a frame memory to be synthesized is designated, a synthesis ratio is designated, and a synthesis timing is instructed. In order to generate grain noise, the cutout processing unit 106 is instructed to read out from the memory 109, the cutout position and size, and the timing to write out the cutout data to the second frame memory 110. I do. The enlargement processing unit 107 is instructed to read data from the second frame memory 110 and to instruct the size at the time of resizing. The signal generator 103 is instructed to drive timing for driving the image sensor 100.

  Here, with reference to FIGS. 3 and 4, the generation method and the addition method of the grain noise used when adding the grain noise that gives the film-like grain effect to the image data, which is a characteristic process of the present embodiment, are described in detail. Describe. In the method of arranging a plurality of granular pattern data as in Patent Document 1, in order to arrange the pattern data in synchronization with the update period of the moving image, that is, the frame rate, a high-speed that performs different arrangements each time within the frame rate period. A simple processing system is required. Further, in the method of selecting and adding a plurality of grain noise patterns, a plurality of memory areas for storing the grain noise are required.

  Therefore, in this embodiment, grain noise is previously stored as two-dimensional data, noise data corresponding to the entire frame of an image input at an arbitrary position is extracted, and the extracted noise data is combined with a moving image. . At this time, by synchronizing the cut-out timing with the frame rate of the moving image and changing the cut-out position every frame, it becomes possible to synthesize spatially and temporally random grain noise into the moving image. .

  FIG. 3 is a diagram showing the relationship between two-dimensional noise data (grain noise data 301) and cut-out noise data 302 cut out therefrom. FIG. 4 is a diagram showing the concept of combining the cut-out noise data 302 and the captured image. In the present embodiment, the grain noise data 301 is a pixel group in which a minimum granularity unit is one pixel and a random number according to a Gaussian distribution is recorded for each pixel as a pixel value. Here, the random numbers are not limited to those according to the Gaussian distribution, and many types of random numbers such as uniform random numbers and exponential random numbers are conceivable, and the types of random numbers are not limited. Various forms such as image data or a data string can be considered as the form of actual data as noise data, and the form is not limited to any of these forms. Further, the minimum granular unit is not limited to one pixel, but of course, the smaller the unit, the higher the randomness.

  For the grain noise data 301, the cutout start positions X and Y are randomly determined by using dynamic values that can be acquired within the camera, such as processing time, camera posture information, and position information. Data of a predetermined size is cut out from the grain noise data 301 as cut-out noise data 302 from the determined start position, and is combined with the picked-up image as shown in FIG. 4 to add grain noise to the picked-up image.

  Here, when the cutout start position is randomly changed each time, if the previous cutout position and the current cutout position are close to each other or only shifted in the horizontal or vertical direction, a two-dimensional pattern is obtained as a synthesis result. Appears to move.

  Therefore, in the present embodiment, the current cutout position is set so that the amount of movement from the previous cutout position to the current cutout position does not fall within a predetermined range.

  With reference to FIG. 7, the clipping control of the noise data 301 instructed by the system controller 102 in the present embodiment to the clipping processing unit 106 will be described. This control is performed in a shooting mode in which a film-like noise effect is added to a shot image. Alternatively, the moving images stored in the memory such as the recording medium 212 are continuously read out and performed as post processing.

  In step S701, candidate values to be extracted from this time are instructed from the noise data stored in the memory 109 using dynamic values as described above. In order to synthesize the cut-out data as grain noise, it is determined at random each time and is set to a position different from the previous cut-out position. When the current cutout position to be instructed is determined, the process proceeds to step S702.

  In step S702, the current frame rate information of the image sensor 100 is acquired based on the drive timing signal instructed to the signal generator 103. If information is acquired, it will progress to step S703.

  In step S <b> 703, a determination reference value for determining how far the current cutout position is from the previous cutout position stored in the temporary memory (not shown) is calculated. Since the grain noise to be synthesized in synchronism with the frame rate is updated, the grain noise update period becomes longer as the frame rate becomes smaller, that is, the captured image update period becomes longer, and it continues to be seen. Therefore, if the previous cutout position and the current cutout position are close to each other, the synthesized grain noise appears to move. In view of this, the reference (predetermined value) for determining how close the current cutout position is to the previous cutout position is changed according to the frame rate information acquired in step S702. For example, at a frame rate of 60 fps (first frame rate), when the current position is within 10 pixels (first predetermined value) around the previous position, it is determined that the position is close. On the other hand, at 30 fps (second frame rate), the determination criterion is determined so as to determine that the current position is within 20 pixels (second predetermined value) in the vicinity and that the position is close to the predetermined value or less. When the determination criterion is determined, the process proceeds to step S704.

  In step S704, a distance between cutout positions is calculated that indicates how much the current cutout position calculated in step S701 has changed from the previous cutout position of the noise data on the grain noise data. If it calculates, it will progress to step S705.

  In step S705, the distance from the previous cutout position calculated in step S704 to the cutout position instructed this time is compared with the determination criterion calculated in step S703. If it is determined that the distance is greater than the criterion, that is, the cutout position has changed significantly from the previous time to the current time, the process proceeds to step S707. On the other hand, if it is determined that the distance is smaller than the criterion, that is, the cutout position has changed slightly from the previous time to this time, the process proceeds to step S706.

  In step S706, correction is performed on the position information of the cutout position instructed this time calculated in step S701. A method for correcting the position information of the cutout position will be described later. When the position information of the cutout position is corrected, the process proceeds to step S707.

  In step S707, the position information of the cutout position corrected in step S706 is stored in a temporary memory so as to be used for the next cutout position control, and the process proceeds to step S708.

  In step S708, the cutout position corrected in step S706 is instructed to the cutout processing unit 106, and the process ends.

  Next, the cutout position correction processing in step S706 will be described. 8, 9, and 10 are diagrams illustrating correction conditions and correction positions when correcting the cutout position from the noise data 301 stored in the memory 103. It is assumed that the width and height of the noise data 301 are W pixels and H pixels, respectively (W ≧ 4, H ≧ 4). The noise data is cut out in units of pixels, and the upper left is the origin (0, 0) of the cutting position. The size of the cut-out noise data 302 to be cut out is a width (W / 2) pixel and a height (H / 2) pixel.

  For example, as shown in FIG. 3, when the current cutout position is (X, Y), the cutout start position (X, Y) for cutting out the cutout noise data of the above size is (0 ≦ X <W / An area in the range of 2, 0 ≦ Y <Y / 2) can be specified (FIG. 8). Further, the region area where the cutout start position can be designated is divided into four regions, regions 1, 2, 3, and 4 shown in FIGS. 8 (a), (b), (c), and (d). Region 1 is a region including the cutout position (0 ≦ X <W / 4, 0 ≦ Y <H / 4), and region 2 is the cutout position (W / 4 ≦ X <W / 2, 0 ≦ Y <). H / 4). Further, the region 3 is (0 ≦ X <W / 4, H / 4 ≦ Y <H / 2), and the region 4 is (W / 4 ≦ X <W / 2, H / 4 ≦ Y <H / 2). It is an area including the cutout position in each range.

  When correcting the cutout position, the noise data cut out at the previous cutout position and the cutout position instructed this time have a smaller area of overlapping data, and if the position is farther away, the randomness at the time of synthesis is higher. It is preferable because it improves. Therefore, in the correction control, it is determined where the current cutout position is in the areas 1 to 4, and the cutout position is corrected on the diagonal area according to the location. At this time, as shown in the table of FIG. 9, the cutout position (X, Y) is corrected to the corrected cutout position (X ′, Y ′).

  For example, if the current cut-out position is in area 1 and the corrected position is set to area 4 (FIG. 10 1003), compared to the case of moving to area 2 and area 3 (FIG. 10 1001, 1002). The area where the cut out noise data overlaps is reduced. As a result, the movement of the noise data is less noticeable, and the temporal randomness of the grain noise is improved.

  As described above, in the first embodiment, an arbitrary range is randomly extracted from the two-dimensional noise data every time the captured image is updated, and this is combined with the captured image. Thereby, it is possible to generate an image to which granular noise is added even if it is a moving image. Further, even if the randomly determined cut-out position is close to the previous range within the predetermined range, the randomness of the noise can be ensured by correcting the position outside the predetermined range. In this embodiment, compared to patent literature, noise data can be generated simply by cutting out data of a predetermined size from already read data, and the time required for generation can be reduced.

  Furthermore, if the current cut position of the noise data is nearer than the previous cut position, the cut noise data moves by changing the current cut position. Noise randomness is improved.

  In the present embodiment, the size of the cut-out noise data with respect to the grain noise data 301 (width W, height H) is set to width (W / 2) pixels and height (H / 2) pixels. However, the size of the cut-out noise data with respect to the grain noise data is not limited to this, and may be larger or smaller than this embodiment as long as it does not exceed the size of the grain noise data. However, if the size of the cut-out noise data is increased, the randomness is easily lost. If the size is small, the grain noise data 301 is too large for the required size of the cut-out noise data. End up. Therefore, in the present embodiment, the ratio of the size of the noise data 301 and the cut-out noise data is set to about 4: 1, and further, the processing of the present embodiment for controlling the start position not to be close is included, thereby ensuring the randomness of noise. And saving memory capacity are more favorably realized.

  In this embodiment, the area where the cutout start position can be divided is divided into four. However, the allocation of the number of divisions and the divided areas is not limited to this.

  Further, in the present embodiment, it is determined whether the distance between the current and previous cutout positions is equal to or smaller than a predetermined value, and when the distance is equal to or smaller than the predetermined value, correction for moving the region of the cutout start position as described above is performed. Went. However, the correction method is not limited to this, and the coordinate may be simply moved beyond the distance at which the distance between the current and previous cutout positions is greater than a predetermined value. That is, when the predetermined value is 10 pixels around at a frame rate of 60 fps, the position (coordinates) may be changed so that it becomes 11 pixels or more around. In the first place, when designating the start position of this time from the beginning, coordinates within 10 pixels in the vicinity may be removed from the candidates and designated at random.

  Further, in this embodiment, it is determined whether the distance between the current and previous cutout positions is equal to or less than a predetermined value, and when the distance is equal to or less than the predetermined value, the correction for moving the cutout start position area as described above is performed. Went. However, the method for determining the cutout start position is not limited to this, and the area to which the current and previous cutout positions belong is determined (whether it belongs to any of the areas 1, 2, 3, and 4 in FIG. 8), and the same area. May be corrected so as to be moved to a different area. At this time, if the movement of the area is a diagonal area, the area where the previous and current cutout areas overlap is smaller as in the present embodiment, which is preferable. Furthermore, when this judgment method is combined with the method for judging whether or not the distance between the current and previous cutout positions is equal to or less than a predetermined value, it is effective even when the start position is near the boundary of the region. Judgment is possible.

<Second Embodiment>
The present embodiment is an image processing apparatus capable of adding a scratch scratch, which is vertical line noise, to an image in order to give a film-scratched effect as a film-like effect. 5 and 6 explain the extraction and synthesis of noise data for scratches in the present embodiment. FIG. 5 is a diagram showing a relationship between noise data composed of a plurality of patterns of scratches and noise data cut out therefrom. Scratch flaw noise data 501 has one pixel in the horizontal direction as a minimum unit, scratch scratches are stored in the vertical direction, and the intensity is determined by a random number. In the vertical direction, the intensity changes with a plurality of pixels as a minimum unit. As a result, the density and thickness of the scratches change in the vertical direction, thereby expressing the “blur” of the scratches. Various random numbers can be considered, including a Gaussian distribution, but it does not depend on the type.

  FIG. 6 is a diagram showing the concept of combining the cut scratch and the captured image. In the present embodiment, the cut-out noise data 502 is cut out from the scratch flaw noise data 501 as shown in FIG. 5 and resized to a predetermined image size to generate the pasted noise data 602. Then, the pasting position of pasting noise data 602 is determined and pasted based on the pasting position of pasting noise data 602 and the pasting time at that position.

  FIG. 2 is a block diagram of a digital video camera as an image processing apparatus according to the second embodiment. The image sensor 200 photoelectrically converts the incident light that has been imaged. The photoelectrically converted signal is input to the camera signal processing unit 201. The camera signal processing unit 201 performs various kinds of image quality processing on the photoelectrically converted signal, converts it into a video signal, and outputs it. The output video signal is encoded in a predetermined recording format by the encoder unit 211 and recorded on the recording medium 212.

  Next, signal processing in the camera signal processing unit 201 will be described. The signal input to the signal processing unit 201 is first subjected to various types of signal processing on the signal components of the luminance component, color component, and black component by the image quality processing unit 204. The processed signal is stored in the frame memory 209 as a captured image.

  A memory 210 stores scratch scratch noise data 501 including a plurality of patterns. The noise data 501 is read from the memory 210. The cut-out processing unit 206 cuts out the noise data 502 of scratches having a designated position and size on the noise data 501. The extracted noise data 502 is resized by the enlargement processing unit 207 into noise data 602 having a size necessary for synthesis with the captured image 601. The pasting processing unit 208 designates a pasting position for synthesizing the noise data 602 at an arbitrary position independent of the cutout position.

The synthesis processing unit 205 reads the captured image 601 stored in the frame memory 209 and the noise data 602 processed by the pasting processing unit 208 at a predetermined timing, performs synthesis, and outputs a synthesized image.
The signal generator 203 is a signal generator that generates a signal for controlling the drive timing of the image sensor 200. The generated signal is supplied to the image sensor 200 and the system controller 202.

  A system controller 202 controls a camera block including the image sensor 200 and the camera signal processing unit 201. A signal accumulation period and readout timing are instructed to the image sensor 200. For the image quality processing unit 204, parameters necessary for image quality setting are set by various signal processing. For the synthesis processing unit 205, the frame memory to be synthesized is designated, the synthesis ratio is designated, and the synthesis timing is instructed. The cutout processing unit 206 is instructed to read data from the memory 210 and instruct the cutout position and size in order to generate a scratch. The enlargement processing unit 207 is instructed to specify the size at the time of resizing. The paste processing unit 208 is instructed of the paste position at the time of synthesis. The signal generator 203 is instructed to drive timing for driving the image sensor 200.

  Next, the pasting control of the pasting noise data 602 instructed by the system controller 202 in the present embodiment to the pasting processing unit 208 will be described with reference to FIGS. This control is performed in a shooting mode in which image processing is performed as if a scratch is added to the shot image. Alternatively, the moving images stored in the memory such as the recording medium 212 are continuously read out and performed as post processing. This processing is started in conjunction with the vertical synchronization signal, and a series of processing is repeated by the synchronization signal.

  In step S1101, the value of the superimposition counter (first detection means), which is a counter that measures the time during which the scratches stored in the temporary memory (not shown) are superimposed on the captured image, is set to the superimposition target time (first detection means). It is determined whether or not the first predetermined time has been reached. If the target time has not been reached, the process proceeds to step S1108. On the other hand, if it has reached, the process proceeds to step S1102. Here, the target time for superimposition is a time for determining how long the same scratches can be continuously displayed on the captured image, and may be determined according to various conditions or may be determined by the user. .

  In step S1102, the superposition counter that has reached the target time is cleared to the initial state, and the process proceeds to step S1103.

  In step S1103, it is acquired whether or not this time scratch scratches extracted from the noise data stored in the memory 210 are to be superimposed. For example, the presence / absence information may be a value randomly obtained from random number generation means in order to randomly superimpose scratches. When the presence / absence information is acquired, the process proceeds to step S1104.

  In step S1104, it is determined whether the presence / absence information obtained in step S1103 is superimposed. The determination result is stored in a temporary memory. Presence / absence information may be defined as a binary value. For example, if it is a value obtained from random number generation means, superimposition is present if the value is greater than a predetermined threshold, and conversely if there is less than the threshold. And decide. If it looks as a time change, the process of step S1104 is executed, and as shown in FIG. 12, periods with and without superimposition occur randomly for each predetermined frame. In the present embodiment, the presence / absence of superposition is determined for each frame. As a result of the determination, if it is determined that there is superposition, the process proceeds to step S1105. On the other hand, if it is determined that there is no overlap, the process proceeds to step S1121.

  In step S <b> 1105, the cut start position, the cut width, and the height of the cut noise data 502 are determined from the scratch flaw noise data 501 stored in the memory 210. By changing these, the pattern (length, strength, number, etc.) of the scratches to be superimposed changes. Therefore, it may be determined using random number generation means so as to cut out from a different start position each time. When the cutout position is calculated, the process proceeds to step S1106.

In step S1106, the cutout position determined in step S1105 is instructed to the cutout processing unit 206, and the process proceeds to step S1107.
In step S1107, the superimposition target time is determined and stored in the temporary memory, and then the process proceeds to step S1108.
In step S1108, addition (increment) of the superposition counter is performed, and the process proceeds to step S1109.

  In step S1109, a superposition continuation counter (second detection means) is added to measure the elapsed time since it was determined that there was no scratch scratch superposition, and the flow proceeds to step S1110.

  In step S1110, the position where the noise data cut out by the cut-out processing unit 206 is to be combined and pasted on the currently captured image is determined. The pasting position is a random pasting position depending on the timing of synthesis. For example, when a pasting position is acquired using random number generation means, scratches are superimposed at different positions on the screen every time the composition is performed, and it appears as random noise. When the pasting position is acquired, the process proceeds to step S1111.

  In step S1111, the current frame rate information of the image sensor 200 is acquired based on the drive timing signal instructed to the signal generator 203. If information is acquired, it will progress to step S1112.

  In step S1112, a determination reference value (predetermined value) for determining how far the current pasting position and the previous pasting position are located is calculated. Since the scratch scratch to be synthesized is updated in synchronization with the frame rate, the scratch scratch update cycle becomes longer as the frame rate decreases, that is, the captured image update cycle becomes longer, and the scratch scratch continues to be seen. If the distance between the previous pasting position and the current pasting position is large, the combined scratches appear to move. Therefore, the criterion for determining how far the current and previous pasting positions are is changed according to the frame rate information acquired in step S1111. For example, at a frame rate of 60 fps (first frame rate), when the current position is at a position farther than 10 pixels (first predetermined value) before and after the horizontal position with the previous position as the center, a far position that is larger than the predetermined value. It is determined that On the other hand, at 30 fps (second frame rate), the criterion is such that if the current position is farther than 20 pixels before and after the horizontal (second predetermined value), it is determined that the position is farther than the predetermined value. To change. When the determination criterion is determined, the process proceeds to step S1113.

  In step S1113, the distance of how much the current pasting position calculated in step S1110 has moved from the previous pasting position of the noise data is calculated. If it calculates, it will progress to step S1114.

  In step S1114, the distance between the pasting positions calculated in step S1113 is compared with the determination criterion calculated in step S1113. If it is determined that the distance is greater than the criterion (predetermined value), that is, the pasting position has changed significantly from the previous time to this time, the process proceeds to step S1116. On the other hand, if it is determined that the distance is smaller than the criterion, that is, the pasting position has changed slightly from the previous time to this time, the process proceeds to step S1115.

In step S1115, the superposition continuation counter upper limit value (second predetermined time) defined in advance is changed to be larger. If it changes, it will progress to step S1117.
In step S1116, the superposition continuation counter upper limit value specified in advance is changed to be smaller. If it changes, it will progress to step S1117.
In step S1117, the superposition continuation counter upper limit value is compared with the value of the superposition continuation counter stored in the temporary memory. If the superposition continuation counter reaches the upper limit, the process proceeds to step S1122. On the other hand, if the upper limit has not been reached, the process proceeds to step S1118.
In step S1118, the determined pasting position is stored in a temporary memory and used for the next control. If it preserve | saves, it will progress to step S1119.
In step S 1119, the current paste position is instructed to the paste processing unit 208. When the instruction is given, the process proceeds to step S1120.
In step S1120, the composition processing unit 205 is instructed to validate the superimposition of the scratch scratch instructed to be pasted and the captured image. When superimposition is enabled, control is terminated.
In step S1121, the value of the superposition continuation counter stored in the temporary memory is cleared to the initial state and stored. When the storage ends, the process proceeds to step S1122.

  In step S <b> 1122, the composition processing unit 205 is instructed to invalidate the superimposition of the scratches instructed for pasting and the captured image. The invalidation method may be a method in which only the captured image is output without performing the synthesis process itself, or a method in which only the captured image is output by reducing the synthesis ratio of the scratches to the captured image, although the synthesis process is performed. Well, any realization means may be used. When superimposing is disabled, the control is terminated.

  In the second embodiment, an arbitrary position is randomly extracted every time in accordance with the update of the captured image from the noise data composed of a plurality of patterns of scratches stored in advance in the storage unit, and this is captured image And at any position. As a result, it is possible to add to the moving image scratch scratches that occur randomly and temporally and spatially.

  Furthermore, when the addition of scratches continues for a certain period or longer, the current pasting position of noise data is compared with the previous pasting position. At this time, it is possible to realize an operation close to the actual occurrence of scratches by restricting the addition of scratches to a position where the movement of the position is far away.

  In the first and second embodiments described above, the grain noise cut-out position change control and the scratch scratch addition invalidity control depend on the frame rate of the moving image, and the grain noise change condition and the scratch damage control. The invalid condition was changed. However, when the synthesis ratio of noise data is small, it is difficult for the noise component to be visually recognized in the synthesized image. Therefore, in the grain noise cut-out control, even if the current cut-out position is near the previous position, it is difficult to recognize the pattern as moving.

  Therefore, it goes without saying that if the setting of the composite ratio of the captured image and noise data in the synthesis can be changed by a user operation or automatically, the condition can be changed according to the synthesis ratio of the noise data. That is, the smaller the synthesis ratio of noise data to the captured image is, the larger the predetermined value can be.

  In the first and second embodiments described above, the case where grain noise and scratches are added in the moving image shooting operation of the digital video camera has been described. However, the present invention is not limited to moving image shooting, and can be used for continuous shooting in still image shooting and adding different patterns of grain noise during each shooting. Needless to say, it can be used to add the effect of film shooting in the reproduction of moving images or still images.

(Other embodiments)
The object of the present invention can also be achieved as follows. That is, a storage medium in which a program code of software in which a procedure for realizing the functions of the above-described embodiments is described is recorded is supplied to the system or apparatus. The computer (or CPU, MPU, etc.) of the system or apparatus reads out and executes the program code stored in the storage medium.

  In this case, the program code itself read from the storage medium realizes the novel function of the present invention, and the storage medium and program storing the program code constitute the present invention.

  Examples of the storage medium for supplying the program code include a flexible disk, a hard disk, an optical disk, and a magneto-optical disk. Further, a CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD-R, magnetic tape, nonvolatile memory card, ROM, or the like can also be used.

  Further, by making the program code read by the computer executable, the functions of the above-described embodiments are realized. Furthermore, when the OS (operating system) running on the computer performs part or all of the actual processing based on the instruction of the program code, the functions of the above-described embodiments are realized by the processing. Is also included.

  Furthermore, the following cases are also included. First, the program code read from the storage medium is written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer. Thereafter, based on the instruction of the program code, the CPU or the like provided in the function expansion board or function expansion unit performs part or all of the actual processing.

100, 200 Image sensor 101, 201 Camera signal processing unit 102, 202 System controller 103, 203 Signal generator 104, 204 Image processing unit 105, 201 Compositing processing unit 106, 206 Cutout processing unit 107, 207 Enlargement processing unit 108 First Frame memory 109, 210 memory 110 second frame memory 111, 211 encoder 112, 212 recording medium 208 pasting processing unit 209 frame memory

Claims (18)

An image processing apparatus capable of giving a noise effect to an input image of a plurality of frames,
Storage means for storing noise data composed of two dimensions;
Clipping means for cutting out noise data corresponding to the entire frame of an image input from the noise data;
Designating means for designating a cutout position on the noise data cut out by the cutout means;
Combining the noise data cut out by the cut-out means with the input image,
The image processing apparatus characterized in that the designation means designates the current cutout position so that the distance between the previous cutout position and the current cutout position on the noise data is greater than a predetermined value. The designation means is:
Indicating means for indicating the cut-out position based on a random value;
When the distance between the current cut position instructed by the instruction means and the previous cut position is smaller than a predetermined value, the current cut position instructed by the instruction means is set to a position where the distance is larger than the predetermined value. Change means to change;
The image processing apparatus according to claim 1, further comprising: The specifying unit includes a detection unit that divides an area where a cutout position can be specified on the noise data into a plurality of areas, and detects which area the cutout position instructed by the instruction unit belongs to;
The image processing apparatus according to claim 2, wherein the changing unit changes the current cutout position instructed by the instruction unit to a position on an area different from the previous cutout position. The designation means is:
Indicating means for indicating the cut-out position based on a random value;
A detection unit that divides an area where a cutout position can be specified on the noise data into a plurality of areas, and detects which area the cutout position instructed by the instruction unit belongs to;
When it is detected by the detection means that the position of the current cutout position and the previous cutout position indicated by the instruction means belong to the same area, the current cutout position indicated by the instruction means is changed to the previous cutout position. Change means for changing to a position on an area different from the cutout position of
The image processing apparatus according to claim 1, further comprising:   The changing unit uses a first predetermined value when an image to which the plurality of frames are input has a first frame rate, and has a second frame rate smaller than the first frame rate. The image processing apparatus according to claim 1, wherein a second predetermined value that is larger than the first predetermined value is sometimes used.   6. The composition unit according to claim 1, wherein the synthesizing unit resizes the noise data of the predetermined size to a size of the input image data and combines the noise data with the input image data. The image processing apparatus described.   The image according to any one of claims 1 to 6, wherein the noise data is data composed of a group of pixels whose pixel values are determined by random numbers, and the noise effect is a film-like granular effect. Processing equipment. An image processing apparatus capable of giving a noise effect to an input image of a plurality of frames,
Storage means for storing noise data of a plurality of patterns;
Selecting means for selecting noise data to be pasted to the image of the current frame from the plurality of patterns of noise data;
Determining means for determining a position to paste the noise data selected by the selecting means on the image of the current frame;
Combining means for pasting the noise data selected by the selection means by combining the noise data selected by the selection means at the position determined by the determination means;
When the distance between the pasting position where the noise data is pasted on the previous frame image and the pasting position where the noise data is pasted on the current frame image is larger than a predetermined value, the synthesizing unit An image processing apparatus that restricts pasting of data. The plural patterns of noise data are composed of two-dimensional data,
9. The image according to claim 8, wherein the selection unit selects noise data to be pasted on the image of the current frame by cutting out noise data based on a randomly designated position from the two-dimensional data. Processing equipment.   The image processing apparatus according to claim 8 or 9, wherein the noise data of the plurality of patterns is determined by a random number with one pixel in the horizontal direction as a minimum unit and a plurality of pixels in the vertical direction as a minimum unit. . First detection means for detecting a time during which the same noise data selected by the selection means is continuously pasted;
The selecting means selects new noise data when it is detected by the first detecting means that the time during which the same noise data is continuously pasted exceeds a first predetermined time. The image processing apparatus according to claim 8, wherein the image processing apparatus is an image processing apparatus. Second detection means for detecting a duration of time during which noise data is continuously pasted to the input image of the plurality of frames;
11. The noise data pasting is invalidated when the second detecting means detects that the duration exceeds a second predetermined time. The image processing apparatus described in one.   The determination unit uses a first predetermined value when an image to which the plurality of frames are input has a first frame rate, and has a second frame rate smaller than the first frame rate. 13. The image processing apparatus according to claim 8, wherein a second predetermined value having a wider range than the first predetermined value is sometimes used.   The image processing apparatus according to claim 8, further comprising a determination unit that determines, for each predetermined frame, whether or not the noise data selected by the selection unit is to be pasted. Storage means for storing noise data configured in two dimensions, and synthesis means for synthesizing noise data of a predetermined size cut out from the noise data into the input image. An image processing apparatus control method capable of giving a noise effect to an image,
A cutout step of cutting out noise data of a predetermined size from the noise data;
A designation step for designating a cutout position on the noise data cut out in the cutout step;
A synthesis step of synthesizing noise data based on the noise data of a predetermined size cut out in the cutout step with the input image by a synthesis unit;
In the specifying step, the current cutout position is specified such that the distance between the previous cutout position on the noise data and the current cutout position is greater than a predetermined value. Storage means for storing a plurality of patterns of noise data; and a combining means for pasting the noise data selected from the plurality of patterns of noise data by combining the input data with the input image. A control method of an image processing apparatus capable of giving a noise effect to an image,
A selection step of selecting noise data to be pasted to the image of the current frame from the plurality of patterns of noise data;
A determination step for determining a position to paste the noise data selected in the selection step on the image of the current frame;
A synthesis step of pasting the noise data selected in the selection step by synthesis at the position determined in the determination step by a synthesis unit;
In the synthesis step, if the distance between the pasting position where the noise data is pasted on the image of the previous frame and the pasting position where the noise data is pasted on the image of the current frame is greater than a predetermined value, the current noise A method for controlling an image processing apparatus, characterized in that pasting of data is restricted.   A computer-executable program in which a procedure of a control method for an image processing apparatus according to claim 16 is described.   A computer-readable storage medium storing a program in which a procedure of a control method for an image processing apparatus according to claim 16 is described.

Images