JP6044234B2 - Image processing apparatus and image processing method - Google Patents

Description

  The present invention relates to an image processing apparatus and an image processing method, and can be applied to, for example, an image processing apparatus and an image processing method for obtaining the saliency of an image region that changes over time.

  In recent years, in the image processing field, a method for calculating bottom-up type attention as an objective index value has been studied. The index value obtained by the bottom-up type attention is used, for example, for a technique in which a robot instantaneously detects an approach to an object (see Patent Document 2) or for example, a visual content evaluation technique (see Patent Document 3). Has been. Here, bottom-up attention refers to a reaction in which a person or animal instantaneously specifies a region of interest.

  Itti's saliency model described in Non-Patent Document 1 is a method for calculating a saliency value obtained by quantifying the degree to which a person instantaneously pays attention to points or regions in an image, that is, the level of saliency. Exists.

  In the retinal ganglion cells in the retina of the eye, there is a region called a receptive field, and when this receptive field is stimulated by light, the information is transmitted to the brain. The receptive field is composed of two parts, a circular center in the center and the surrounding area (surround). By applying a mechanism that can be placed in the receptive field, Itti et al. A saliency map was constructed by constructing a model that digitizes the places where the signal is strong (the place where attention is drawn).

  The technique described in Patent Document 1 uses the saliency value described in Non-Patent Document 1 to calculate a saliency map based on the feature amount of the input image, and uses this saliency map to focus on the input image. And the main subject area and the background area are separated.

  The Itti saliency model takes into account the brightness, hue (hue, lightness, saturation, etc.) and direction of the image, but this is calculated from only one image at a certain time and changes in the time domain. Does not consider changes between images.

  Therefore, the Itti saliency model has a problem that the saliency due to a time change in which the color of a wide region changes rapidly, which is generally considered to be highly saliency, is not reflected.

  There exists a technique of nonpatent literature 2 as a technique corresponding to the above-mentioned subject. In Non-Patent Document 2, Guo et al. Calculate a saliency value reflecting a time change of an image.

  FIG. 2 is an explanatory diagram for explaining a saliency value calculation method for calculating a saliency value reflecting a time change.

  As shown in FIG. 2, in the conventional saliency value calculation method, the luminance and hue saliency calculation unit 102 is saliency related to the luminance and hue (hue, lightness, and saturation) of the input image from the image input unit 101. Is calculated and given to the adding unit 105.

  Further, the input image from the image input unit 101 is accumulated in the time direction saliency calculating unit 103 and the accumulating unit 104. In addition, an image accumulated by the accumulation unit 104 for a predetermined time is given to the time direction saliency calculation unit 103.

  Then, the time direction saliency calculation unit 103 calculates a time direction saliency value based on the input image from the image input unit 101 and the past input image, and provides the time direction saliency value to the addition unit 105. At this time, if there is no past input image corresponding to the delay, the time direction saliency is all treated as “0”.

  The adding unit 105 multiplies the luminance color saliency value from the luminance color saliency calculation unit 102 and the time direction saliency value from the time direction method saliency calculation unit 103 by a predetermined weight. Then add. As a result, it is possible to calculate an integrated saliency value that reflects not only the luminance color saliency value of the input image but also the time direction saliency value. This integrated saliency value is given to the data processing unit 106 at the subsequent stage.

JP 2011-35636 A JP 2007-12047 A JP 2011-39778 A

Laurent Itti, Christof Koch, Ernst Niebur, “A Model of Saliency-based Visual Attention for Rapid Scene Analysis”, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.20.No.11, pp.1254-1259, 199811 Moon Chenlei Guo, Liming Zhang, “A Novel Multiresolution Spatiotemporal Saliency Detection Model and Its Applications in Image and Video Compression”, IEEE Transactions on Image Processing, Vol.19.No.1, pp185-198, January 2010

  However, the technique described in Non-Patent Document 2 can reflect a temporal change of the same position of a point in an image. However, when an object including a similar color moves, the object is originally moved. Nevertheless, there is a problem in that the saliency is calculated so that the point that should have moved does not accompany movement because the similar color remains in the same position in the image after the elapse of time.

  Therefore, when calculating the saliency of the input image, a highly accurate saliency is calculated by adding the degree that the motion vector of the target point or region is different from the surrounding motion vector to the term reflecting the passage of time. There is a need for an image processing apparatus and an image processing method.

In order to solve this problem, the first aspect of the present invention includes (1) saliency calculation means for obtaining a saliency value based on a feature amount of each pixel in an input image constituting a moving image, and (2) an input image. A time change saliency calculating means for obtaining a time change saliency value that changes with time based on the saliency value of the image and a saliency value of the past image, and (3) an input image with reference to the past image Motion saliency calculating means for calculating a motion saliency value indicating the saliency of the movement of each region obtained by dividing the image, Integrated saliency calculating means for obtaining an integrated saliency value that reflects the movement of an object in a moving image , and the motion saliency calculating means divides the input image into a plurality of regions and stores past images. To search for the motion of each region in the input image. The motion saliency value of the target area is acquired based on the difference amount of the motion vector between the target area and one or a plurality of peripheral areas in the input image. A motion saliency calculating unit that determines the motion saliency based on a sum of differences between a motion vector of the target region in the input image and each of the motion vectors of the plurality of surrounding regions. An image processing apparatus characterized by obtaining a sex value .

According to a second aspect of the present invention, in the image processing method for determining the saliency of the moving image, (1) the saliency calculating unit determines the saliency value based on the feature amount of each pixel in the input image constituting the moving image. And (2) time change saliency calculating means for obtaining a time change saliency value that changes with time based on the saliency value of the input image and the saliency value of the past image. And (3) a motion saliency calculating step in which the motion saliency calculating means calculates a motion saliency value indicating the saliency of the motion of each region obtained by dividing the input image with reference to the past image. (4) The integrated saliency calculating unit reflects the movement of the object in the moving image based on the saliency value, the time change saliency value, and the motion saliency value obtained by the saliency calculating unit. possess the integration saliency calculating step of obtaining a value, motion The articulation calculation means divides the input image into a plurality of regions, refers to past images, performs a motion search step of performing a motion search of each region in the input image, and a motion search result of each region by the motion search unit A motion saliency calculating step of obtaining a motion saliency value of the target region based on a difference amount of motion vectors between the target region and one or a plurality of peripheral regions in the input image. The image processing method is characterized in that the motion saliency value is obtained based on the sum of the difference amounts between the motion vector of the target region in the input image and the motion vectors of the plurality of peripheral regions .

  According to the present invention, when calculating the saliency of the input image, a high-accuracy saliency is obtained by adding a degree that the motion vector of the target point or region is different from the surrounding motion vector to the term reflecting the passage of time. Sex can be calculated.

It is a functional block diagram which shows the structure of the image processing apparatus of embodiment. It is a functional block diagram which shows the structure of the image processing apparatus which calculates the saliency according to the conventional time change.

(A) Main Embodiments Hereinafter, embodiments of an image processing apparatus and an image processing method of the present invention will be described in detail with reference to the drawings.

(A-1) Configuration of Embodiment FIG. 1 is a functional block diagram illustrating functions of an image processing apparatus 200 according to the embodiment.

  The hardware configuration of the image processing apparatus 200 includes, for example, a CPU, a ROM, a RAM, an EEPROM, an input / output interface unit, and the like, and the CPU executes an image processing program stored in the ROM. Realize functional blocks. Further, the image processing program may be installed, and even in that case, when the image processing apparatus 200 is realized, the processing of FIG. 1 is realized.

  In FIG. 1, an image processing apparatus 200 includes an image input unit 201, a luminance tint saliency calculating unit 202, a time direction saliency calculating unit 203, a storage unit 204, an adding unit 205, a data processing unit 206, an area dividing unit 207, A motion search unit 208 and a motion saliency calculation unit 209 are included. A configuration including the region dividing unit 207, the motion search unit 208, and the motion saliency calculation unit 209 is also referred to as a motion saliency component generation unit 210.

  The image input unit 201 supplies the input image to each of the luminance and tint saliency calculating unit 202, the time direction saliency calculating unit 203, the accumulating unit 204, and the region dividing unit 207. The image captured by the image input unit 201 is assumed to be an image constituting a moving image.

  The luminance and tint saliency calculating unit 202 obtains the luminance and tint saliency value of the input image from the image input unit 201 and gives the obtained luminance and tint saliency value to the adding unit 205. That is to say, the luminance tint saliency calculating unit 202 calculates a saliency map of the input image input this time, for example, the input image input at time t.

  Various methods can be widely applied to the method of calculating the luminance / tint saliency value by the luminance / tint saliency calculating unit 202. For example, the saliency map calculating method of Itti et al. Can be applied.

  For example, the luminance and tint saliency calculating unit 202 prepares a plurality of images (scale images) reduced in stages from the original image for the input image from the image input unit 201. The luminance tint saliency calculating unit 202 obtains luminance (Intensity), hue component (Color Channel), and direction component (Orientation) for each pixel (pixel) for each scale image, and the luminance between the scale images. By calculating the difference between the hue component and the direction component, the saliency values of the luminance, hue component, and direction component are obtained. These saliency values are referred to as luminance color saliency values.

  The time direction saliency calculating unit 203 obtains the saliency value in the temporally changed image based on the input image from the image input unit 201 and the past image stored in the storage unit 204 for a predetermined time. Is. The temporal direction saliency calculating unit 203 obtains the temporally changing saliency by taking the difference between the saliency value of each pixel and the saliency value of each pixel of the past image in the input image. This saliency that changes with time is called a time change saliency value.

  Various methods can be widely applied to the time-direction saliency value by the time-direction saliency calculation unit 203. For example, the saliency calculation method of Guo et al. Described in Non-Patent Document 2 can be applied.

  The time direction saliency calculating unit 203 receives an image accumulated for a predetermined time from the accumulation unit 204, but until a delay time elapses, that is, until a past image is given from the accumulation unit 204. The time direction saliency values are all “0”.

  The accumulation unit 204 accumulates the input image from the image input unit 201 for a predetermined time. The accumulation unit 204 gives the image accumulated in accordance with the accumulation order to the time direction saliency calculation unit 203 after accumulation for a predetermined time. That is, past images accumulated for a predetermined time are given to the time direction saliency calculating unit 203.

  The movement saliency component generation unit 210 generates a movement saliency component of the object (target) in the image based on the input image from the image input unit 201 and the past input image stored in the storage unit 204. To do.

  Further, as described above, the motion saliency component generation unit 210 includes the region dividing unit 207, the motion search unit 208, and the motion saliency calculation unit 209.

  The area dividing unit 207 divides the input image from the image input unit 201 into predetermined areas. The region dividing unit 207 divides the input image into a plurality of regions in order to perform a motion search by a motion search unit 208 described later. Here, the image area dividing method by the area dividing unit 207 divides the input image, for example, with a predetermined number of pixels (for example, 16 × 16 pixels) as one area. Of course, one area to be divided is not limited to 16 × 16 pixels. For example, 4 × 4 pixels, 8 × 8 pixels, or the like may be used as one area. Further, for example, a shape other than a square (for example, a rectangular shape) may be divided as one region.

  The motion search unit 208 acquires the area division information divided by the area division unit 207 and the input image from the area division unit 207. The motion search unit 208 performs a motion search based on the current input image and the past input image stored in the storage unit 204, and obtains motion vector information of each area of the input image. Furthermore, the motion search unit 208 provides the segmentation information of the input image and the motion vector information of each region to the motion saliency calculation unit 209.

  The motion saliency calculating unit 209 receives the region division information of the input image and the motion vector information of each region from the motion search unit 208, obtains the motion saliency value of each divided region, and gives it to the adding unit 205. . That is, the motion saliency calculation unit 209 obtains a difference amount between the motion vectors of the target region and the surrounding region. As a result, the sum of the difference amounts between the motion vector of the target region and the motion vector of the peripheral region can be output as the motion saliency value.

  The adding unit 205 calculates an integrated saliency value reflecting the motion of the object in the moving image based on the saliency value by the saliency calculating unit, the time change saliency value, and the motion saliency value. It is what you want.

  In other words, the adding unit 205 performs the luminance tint saliency value from the luminance tint saliency calculating unit 202, the time direction saliency value from the time direction saliency calculating unit 203, and the movement from the motion saliency calculating unit 209. A predetermined weighting coefficient is assigned to the saliency value, and components of each saliency value are added to obtain an integrated saliency value.

  The data processing unit 206 performs predetermined data processing using the integrated saliency value obtained by the adding unit 205. The content of the data processing of the data processing unit 206 is not particularly limited as long as the processing uses the saliency value of the moving image.

(A-2) Operation of Embodiment Next, an operation of processing for calculating a saliency value of a moving image by the image processing apparatus 200 of this embodiment will be described.

  In FIG. 1, when one frame image as a moving image is input to the image input unit 201, the input image is divided into a luminance tint saliency calculation unit 202, a time direction saliency calculation unit 203, a storage unit 204, and a region division. Input to the unit 207.

  The luminance and hue saliency calculating unit 202 calculates saliency values related to luminance and hue based on the input image, and the calculated saliency values are given to the adding unit 205.

  For example, the luminance and tint saliency calculating unit 202 calculates a saliency value related to luminance and tint by the technique described in Non-Patent Document 1.

  For example, the luminance and tint saliency calculating unit 202 generates a plurality of images (scale images) that are reduced in stages from the original image for the input image from the image input unit 201. The luminance tint saliency calculating unit 202 obtains luminance (Intensity), a hue component (Color Channel), and a direction component (Orientation) as feature amounts for each scale image, and the luminance, By calculating the difference between the hue component and the direction component, the saliency values of the luminance, the hue component, and the direction component are obtained.

  For example, assuming that the RGB component of the σth scale image is a red component r (σ), a green component g (σ), and a blue component b (σ), the luminance tone saliency calculating unit 202 performs the σth scale image. The luminance I (σ) of each pixel with respect to is calculated as in the following equation (1).

I (σ) = {r (σ) + g (σ) + b (σ)} / 3 (1)
Then, the luminance tint saliency calculating unit 202 obtains a luminance difference between the scale images. At this time, since the size of each scale image is different, the luminance tint saliency calculating unit 202 obtains a difference for each pixel after extending the smaller size side to the larger side. The larger side represents the center pixel, and the smaller side represents the average including the center pixel and the surrounding pixels. The luminance tint saliency calculating unit 202 can obtain the luminance difference I (c, s) = | I (c) −I (s) |.

Next, the luminance tint saliency calculating unit 202 normalizes each RGB component with the luminance in order to exclude the influence of the luminance on the color. Using normalized RGB components, values of red R (σ), green G (σ), blue B (σ), and yellow Y (σ) are obtained as hue components.

  The luminance and tint saliency calculating unit 202 sets the value to zero when each value of the hue component becomes a negative number according to the above equation (2).

  Since the hue at a certain central portion is suppressed by the hue at the peripheral portion, the luminance hue saliency calculating unit 202 includes red and green and blue and yellow as combinations of hues. Therefore, the brightness and tint saliency calculating unit 202 obtains the saliency of the hue component by obtaining the difference between the central part and the peripheral part for the difference between red and green (or blue and yellow).

RB (c, s) = | {R (c) −G (c)} − {G (s) −R (s)} | (3)
BY (c, s) = | {B (c) −Y (c)} − {Y (s) −B (s)} | (4)
Next, the luminance tint saliency calculating unit 202 uses, for example, a Gaussian filter (for example, a Kabor filter) to convolve and integrate the four directions of rotation angles θ = 0 °, 45 °, 90 °, and 135 °. I do. The luminance tint saliency calculating unit 202 obtains the directional contrast O (c, s, θ) based on the difference between the directional Center-Surround scales.

  Note that the technology described in Non-Patent Document 1 can be referred to for the method of calculating the luminance / color saliency value by the luminance / color saliency calculating unit 202.

  The time direction saliency calculating unit 203 obtains the saliency of the temporally changed image based on the input image from the image input unit 201 and the past image accumulated for a predetermined time.

  The time direction saliency calculating unit 203 calculates, for example, a time direction saliency value that changes with time according to the technique described in Non-Patent Document 2. That is, the time direction saliency calculation unit 203 compares the saliency value of a certain image frame constituting the moving image with the saliency value of the image frame accumulated for a predetermined time, and changes in time direction. Determine the saliency value.

  For example, if the RGB component of the input image at time t is a red component r (t), a green component g (t), and a blue component b (t), the time direction saliency calculating unit 203 The luminance I (t) of each pixel is calculated as follows.

I (t) = {r (t) + g (t) + b (t)} / 3 (5)
M (t) = | I (t) −I (t−τ) | (6)
Here, τ represents a delay time. Expression (6) is a luminance difference amount between the input image captured at time t and the past input image stored in the storage unit 204 for a predetermined time.

  The time direction saliency calculating unit 203 treats all the time direction saliency as “0” when there is no past input image corresponding to the delay.

  For the method of calculating the time change saliency value according to the time change by the time direction saliency calculation unit 203, the technique described in Non-Patent Document 2 can be referred to.

  The area dividing unit 207 divides the image frame input from the image input unit 201 into a plurality of areas by a predetermined method. The area dividing unit 207 divides an image frame into a plurality of areas, for example, with 16 × 16 pixels as one area.

  The motion search unit 208 acquires the area division information divided by the area division unit 207. In addition, the motion search unit 208 performs a motion search on each area of the input image frame based on the input image frame and the past image frames stored in the storage unit 204 for a predetermined time. Find the motion vector of.

  The motion search unit 208 is to obtain a motion vector of each area of the frame of the input image. That is, the motion search unit 208 performs a motion search of each area of the current input image frame using the past input image frame accumulated in the accumulation unit 204 for a predetermined time as a reference frame, and obtains a motion vector of each area. Ask.

  Here, various methods can be widely applied as the motion search method by the motion search unit 208. For example, the “all search method” that searches all image search ranges in order, the “spiral search method” that searches while expanding the range in a spiral from a place where the target existence probability is high, and the cost centered on the initial search position on the screen “Diamond Search Method” that searches until the cost is minimized, “MVFAST Method (Motion Vector Field Adaptive Serch Technique)” that corrects candidates for the initial search position in the diamond search method, and MVFAST Method The expanded “PMVFAST”, the “MV bit amount method” for searching based on the bit amount of the motion vector, the rate distortion “RD (Rate Distortion) theory method”, and the like can be widely applied. In addition, the motion search method is not limited to the above-described method, and an extended method or a combined method of the above methods can be applied.

  The motion saliency calculation unit 209 calculates the motion saliency value of each area of the input image frame.

For example, the motion saliency Sv (j, i) for the region of i rows and j columns is calculated by the following equation (7).

  Here, || · || is a vector Euclidean norm. Further, V (j, i) is a motion vector for the region of i rows and j columns.

  Expression (7) is obtained by calculating the difference amount between the motion vector of the target region and the motion vector of each peripheral region, with the target region (region of i rows and j columns) and a region adjacent to the target region in the vertical and horizontal directions as the peripheral region. The sum is used as a motion saliency value.

  That is, according to the motion saliency calculation unit 209, a saliency component can be obtained in consideration of the amount of motion of the object in the image. Thereby, even if a certain object is a similar color, since the motion of the object can be reflected, the precision of the saliency value of the moving image can be improved.

  In this example, the peripheral region is exemplified as four regions adjacent to the target region in the top, bottom, left, and right directions, but includes all or part of the other four regions existing in the diagonal direction of the target region. You may make it.

  The peripheral region may include a region adjacent to the target region and a region not adjacent to the target region, or may be a region not adjacent to the target region. When a region including a region that is not adjacent to the target region is used as a peripheral region, the motion saliency value of the target region is obtained as a difference amount from a motion vector of a region far from the target region. This is effective when the image area occupied by the object is wide (that is, when the object image is large).

  Furthermore, although the case where the motion saliency value is obtained by the square sum of the Euclidean norm is exemplified in Expression (7), the sum of the Euclidean norm may be used.

  Further, when the peripheral region is not adjacent to the target region, a weighting factor that is inversely proportional to the distance between the target region and the peripheral region may be used. Thereby, for example, it is possible to reduce the contribution of the difference amount between the motion vector of the target region and the motion vector of a region relatively far from the target region.

  Furthermore, when the distance between the target region and the peripheral region is larger than a predetermined threshold, the weighting factor by which the difference amount is multiplied may be set to “0”. As a result, the difference amount between the motion vector of the target region and the motion vector of the region separated from the target region by a threshold or more can be deleted.

  The motion saliency calculation unit 209 obtains a motion saliency value Sv (j, i) according to the equation (7) for each region of the input image frame, and gives it to the addition unit 205.

  The adding unit 205 includes a luminance tint saliency value from the luminance tint saliency calculating unit 202, a time direction saliency value from the time direction saliency value calculating unit 203, and a motion saliency from the motion saliency calculating unit 209. The saliency value is received, the predetermined saliency value is multiplied by each saliency value, and the components of each saliency value after being multiplied by the weight coefficient are added to obtain an integrated saliency value.

  Thereby, the saliency reflecting the movement of the object in the image can be obtained.

  The data processing unit 206 performs predetermined data processing using the integrated saliency value from the addition unit 205.

(A-3) Effects of the Embodiment As described above, according to this embodiment, the saliency value that changes with time is further increased with respect to an area that moves differently from the surrounding area. By adding the saliency value corresponding to the amount of movement of the region, the accuracy of the saliency value can be improved.

(B) Other Embodiments In the above-described embodiments, various modified embodiments of the present invention have been described. However, the present invention can also be applied to other modified embodiments described below.

  In the above-described embodiment, the method of calculating the saliency value related to luminance and color is assumed to apply the method described in Non-Patent Document 1, but is not limited to the description technique of Non-Patent Document 1, An expansion method etc. can be widely applied. In the embodiment described above, the method described in Non-Patent Document 2 is applied as the method for calculating the temporal change saliency value. However, the present invention is not limited to this.

  Further, in the above-described embodiment, the case where the hue component is used as an example of the saliency component relating to the color is illustrated, but the saliency relating to the brightness and the saturation may be used.

  In the embodiment described above, the adding unit may perform normalization processing on each saliency value. For example, normalization may be performed so that values on the saliency map belong to a predetermined range, or when a maximum value on the saliency map is detected and a value close to the maximum value exists in the vicinity These values may be averaged. Alternatively, normalization may be performed by multiplying all values on the saliency map by a predetermined value.

  The data processing unit in the embodiment described above can be applied to various data processing using the integrated saliency value. For example, the present invention can be widely applied to processing for separating an object region and other regions in an image, image processing in a navigation system, image processing in a game machine, moving image processing using a video camera, and the like.

  DESCRIPTION OF SYMBOLS 200 ... Image processing apparatus, 201 ... Image input part, 202 ... Luminance tint saliency calculation part, 203 ... Time direction saliency calculation part, 204 ... Accumulation part, 205 ... Addition part, 206 ... Data processing part, 207 ... Area | region Dividing unit, 208... Motion search unit, 209... Motion saliency calculating unit, 210.

Claims (4)

Saliency calculating means for obtaining a saliency value based on the feature amount of each pixel in the input image constituting the moving image;
Based on the saliency value of the input image and the saliency value of the past image, time change saliency calculation means for obtaining a time change saliency value that changes with time,
A motion saliency calculating means for calculating a motion saliency value indicating the saliency of motion of each region obtained by dividing the input image with reference to past images;
Integrated saliency calculation for obtaining an integrated saliency value reflecting the movement of an object in a moving image based on the saliency value, the time change saliency value, and the motion saliency value by the saliency calculating means. and means,
The motion saliency calculating means is
A motion search unit that divides the input image into a plurality of regions, refers to past images, and performs a motion search of each region in the input image;
A motion for obtaining a motion saliency value of a target region based on a motion vector difference amount between the target region and one or a plurality of peripheral regions in the input image by acquiring a motion search result of each region by the motion search unit With the saliency calculator
Have
The motion saliency calculating unit obtains the motion saliency value based on a sum of difference amounts between a motion vector of a target region in the input image and motion vectors of a plurality of surrounding regions. An image processing apparatus. The motion saliency calculating unit assigns a weight according to the distance between the target area in the input image and each peripheral area to the difference amount between the motion vector of the target area and the motion vector of each peripheral area. The image processing apparatus according to claim 1 , wherein a motion saliency value is obtained. When the distance between the target area and the peripheral area in the input image exceeds a threshold, the motion saliency calculating unit deletes the difference amount between the motion vector of the target area and the motion vector of the peripheral area The image processing apparatus according to claim 1 . In an image processing method for determining the saliency of a moving image,
A saliency calculating means for obtaining a saliency value based on a feature amount of each pixel in the input image constituting the moving image; and
A time-change saliency calculating step for obtaining a time-change saliency value that changes with time based on the saliency value of the input image and the saliency value of the past image;
A motion saliency calculating unit that calculates a motion saliency value indicating the saliency of the motion of each region obtained by dividing the input image with reference to a past image;
The integrated saliency calculation unit reflects the movement of the object in the moving image based on the saliency value obtained by the saliency calculation unit, the time change saliency value, and the motion saliency value. possess the integration saliency calculating step of obtaining a value,
The motion saliency calculating means is
Dividing the input image into a plurality of regions, referring to past images, a motion search step of performing a motion search of each region in the input image;
A motion for obtaining a motion saliency value of a target region based on a motion vector difference amount between the target region and one or a plurality of peripheral regions in the input image by acquiring a motion search result of each region by the motion search unit Saliency calculation process and
Have
An image characterized in that, in the motion saliency calculating step, the motion saliency value is obtained based on a sum of differences between a motion vector of a target region in the input image and motion vectors of a plurality of surrounding regions. Processing method.

Images