JP2013093667A - Image reproducing device, program and method for optimizing reproduction time for each photographic image - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image reproducing device, a program and a method capable of reproducing a photographic image in a reproduction time optimal to a viewer according to a content of a subject in a photographic image.SOLUTION: An image reproducing device comprises: region of interest detection means which extracts feature quantity from a photographic image and detects a region of interest by a viewer and region of interest number N; first parameter calculation means which calculates an average region of interest size by dividing the entire region of interest by the region of interest number N, and calculates a region of interest ratio value Rof an average region of interest size for an image region size; first extension reproduction time calculation means which calculates a first extension reproduction time tbased on region of interest number N/region of interest ratio value R; reproduction time calculation means which calculates a reproduction time t by adding/subtracting the extension reproduction time tto/from an image reference reproduction time t; image storage means which stores a reproduction time t for each image, and reproduction is performed so that an image is displayed on a display by only a reproduction time t.

Description

  The present invention relates to a slide show technique for continuously displaying a plurality of photographic images on a display visually recognized by a viewer.

  The slide show technique is capable of continuously displaying a plurality of images designated by the user one by one, and is executed by a digital photo frame or a personal computer. For example, there is a technique for displaying only images whose check boxes are checked among a plurality of images displayed as thumbnails in an arbitrary order in a slide show (see, for example, Patent Document 1). According to this technology, the reproduction time (display time) of each image can be set in advance by a user operation. However, the reproduction time is constant.

  There is also a technique for changing the reproduction time of each photographic image by weighting based on the total number of reproductions or the total reproduction time (see, for example, Patent Document 2). According to this technique, it is assumed that the cumulative number of playback times or the total playback time for the photograph changes according to the viewer's interest.

JP 2003-288068 A JP 2009-246989 A

OpenCV, [online], [Search September 18, 2011], Internet <URL: http://opencv.jp/> 1.Ming-Ming Cheng, Guo-Xin Zhang, Niloy J. Mitra, Xiaolei Huang, Shi-Min Hu.Global Contrast based Salient Region Detection.IEEE CVPR, p.409-416, Colorado Springs, USA, June 21-23 , 2011.

  However, according to the technique described in Patent Document 1, the reproduction time for each photographic image is constant, and the reader who continuously browses often has a monotonous feeling as a whole. Further, according to the technique described in Patent Document 2, since the playback time changes based on the user's past cumulative number of playback times or cumulative playback time, the playback time in a photographic image that is played back for the first time is always constant. I must.

  On the other hand, in the case of a photographic image taken at home, the inventors can change the playback time according to the content of the subject in the photographic image so that the viewer can easily understand the content. I thought that. In particular, I thought that the more complex the subject that appears in the photographic image, the easier it is for the viewer to understand the content when the playback time is longer.

  Accordingly, an object of the present invention is to provide an image reproduction apparatus, a program, and a method capable of reproducing a photographic image in an optimal reproduction time for a viewer in accordance with the contents of a subject appearing in the photographic image.

According to the present invention, in an image playback device that continuously displays a plurality of images on a display visually recognized by a viewer,
A gaze area detecting means for extracting a feature amount from the image and detecting a gaze area by the viewer and the number of gaze areas _NH ;
A first parameter calculating unit that calculates an average gaze area size obtained by dividing the total area of the gaze area by the number of gaze areas N _H , and calculates a gaze area ratio value R _H of the average gaze area size with respect to the image area size;
First extended playback time calculating means for calculating a _first extended playback time t ₁ based on the number of watched areas N _H / gaze area ratio value R _H ;
Reproduction time calculating means for calculating a reproduction time t obtained by adding and subtracting the extended reproduction time t ₁ to the reference reproduction time t ₀ of the image;
Image storage means for storing the reproduction time t for each image;
The image is reproduced such that the image is displayed on the display only for the reproduction time t.

According to another embodiment of the image playback device of the present invention,
The image is a photographic image,
The gaze area is the face area of the subject person,
The gaze area detecting means is preferably a face area detecting means.

According to another embodiment of the image playback device of the present invention,
Saliency area detecting means for extracting a feature amount from a photographic image and detecting a saliency area other than the subject person;
A second parameter calculating means for calculating a saliency area R _O combining all the saliency areas;
A second extended playback time calculating means for calculating a _second extended playback time t ₂ based on 1 / significant area R _O ;
It is also preferable that the playback time calculating means further adds / subtracts the _second extended playback time t ₂ to the playback time t.

According to another embodiment of the image playback device of the present invention,
A saliency area detecting means for extracting a feature amount from a photographic image and detecting a saliency area other than the subject person and the saliency area number _NH ;
A second parameter calculating means for calculating an average saliency area size obtained by dividing the area of all the saliency areas by the number of saliency areas _NH , and calculating a saliency area ratio value _RH of the average saliency area size to the image area size;
A second extended reproduction time calculating means for calculating a _second extended reproduction time t ₂ based on the number of remarkable areas N _H / the remarkable area ratio value R _H ;
Further comprising
It is also preferable that the playback time calculating means further adds / subtracts the _second extended playback time t ₂ to the playback time t.

According to another embodiment of the image playback device of the present invention,
A scene analysis means for extracting feature amounts from a photographic image and detecting a background scene;
Third parameter calculating means for calculating a scene recognition value R _S having a higher value as it is outdoors and having a lower value as it is indoors;
And third extended playback time calculating means for calculating a _third extended playback time t ₃ based on the scene recognition value R _S ,
It is also preferable that the reproduction time calculation means further add / subtract the _third extended reproduction time t ₃ to the reproduction time t.

According to another embodiment of the image playback device of the present invention,
Photo similarity detection means for detecting whether or not the photo shooting time is continuous within a predetermined time range and the similarity of the continuous photo images is similar to a predetermined threshold value or more,
A fourth parameter calculating means for calculating a continuous similarity number N _S of photographic images similar in succession,
A fourth extended playback time calculating means for calculating a _fourth extended playback time t ₄ based on the 1 / continuous similar number N;
It is also preferable that the reproduction time calculation means further add / subtract the _fourth extended reproduction time t ₄ to the reproduction time t.

According to another embodiment of the image playback device of the present invention,
Reproduction history storage means for recording the average reproduction time C _L and the number of reproductions C _T in the reproduced photographic image;
A fifth parameter acquisition means for acquiring an average reproducing time in the image C _L and Views C _T,
And 5th extended reproduction time calculation means for calculating a _fifth extended reproduction time t ₅ based on 1 / (average reproduction time C _L × number of reproductions C _T ),
It is also preferable that the reproduction time calculation means further add / subtract the _fifth extended reproduction time t ₅ to the reproduction time t.

  According to another embodiment of the image reproduction apparatus of the present invention, it is also preferable that the extended reproduction time calculation means further multiplies a normalization coefficient that is normalized by a predetermined unit time.

According to the present invention, in a program for causing a computer mounted on a device that continuously displays a plurality of images to a display visually recognized by a viewer,
A gaze area detecting means for extracting a feature amount from the image and detecting a gaze area by the viewer and the number of gaze areas _NH ;
A first parameter calculating unit that calculates an average gaze area size obtained by dividing the total area of the gaze area by the number of gaze areas N _H , and calculates a gaze area ratio value R _H of the average gaze area size with respect to the image area size;
First extended playback time calculating means for calculating a _first extended playback time t ₁ based on the number of watched areas N _H / gaze area ratio value R _H ;
Reproduction time calculating means for calculating a reproduction time t obtained by adding and subtracting the extended reproduction time t ₁ to the reference reproduction time t ₀ of the image;
Image storage means for storing the reproduction time t for each image;
The image is reproduced such that the image is displayed on the display only for the reproduction time t.

According to the present invention, in an image display method in an apparatus for continuously displaying a plurality of images on a display visually recognized by a viewer,
A first step of extracting a feature amount from an image and detecting a viewer's gaze area and the number of gaze areas _NH ;
A second step of calculating an average gaze area size obtained by dividing the area of all gaze areas by the number of gaze areas N _H and calculating a gaze area ratio value R _H of the average gaze area size with respect to the image area size;
A third step of calculating a _first extended reproduction time t ₁ based on the number of gaze areas N _H / gaze area ratio value R _H ;
A fourth step of calculating a reproduction time t obtained by adding and subtracting the extended reproduction time t ₁ to the reference reproduction time t ₀ of the image;
A fifth step of accumulating the reproduction time t for each image, and reproducing the image so that the image is displayed on the display only for the reproduction time t.

  According to the image reproducing apparatus, program, and method of the present invention, a photographic image can be reproduced in an optimal reproduction time for the viewer according to the contents of the subject in the photographic image. In particular, without setting the playback time in advance, the content can be played back in a playback time that is easy for the viewer to understand according to the complexity of the subject in the photographic image.

It is a functional block diagram of the image reproduction apparatus in this invention. It is explanatory drawing showing the reproduction time based on the face area | region in this invention. It is explanatory drawing showing the reproduction time based on the remarkable area | region in this invention. It is explanatory drawing showing the reproduction time based on the scene analysis in this invention. It is explanatory drawing showing the reproduction time based on the photograph similarity in this invention. It is explanatory drawing showing the reproduction time based on the reproduction history in this invention. It is explanatory drawing showing the example where reproduction time becomes short and the example which becomes long.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a functional configuration diagram of an image reproducing apparatus according to the present invention.

  The image reproducing apparatus 1 according to the present invention is a digital photo frame for home use, for example, and continuously displays images on a display. In the following description, it is assumed that the image reproduction device 1 is a digital photo frame. The image displayed on the display of the photo frame 1 is a still image such as a photograph taken at home. Of course, the program is not limited to a photo frame, and may be a slide show program executed on a personal computer.

  According to FIG. 1, a digital photo frame 1 as an image reproducing device has a memory card 101 for storing images and a display 102 for displaying these images.

  The memory card 101 is an SD (Secure Digital) card, for example, and is often detachable from the digital photo frame 1. The memory card 101 can be inserted into a digital camera, for example, and can store a large number of images taken by the digital camera. The digital photo frame 1 continuously displays a large number of images stored in the memory card 101 on the display 102 like a slide show.

  The photographic images reproduced from the memory card 101 may be all photographic image groups below an arbitrary folder, or photographic image groups designated by the user. In general, a large number of photographic images can be arranged in ascending order in chronological order with respect to the shooting date.

The digital photo frame 1 in FIG. 1 has the following functional configuration.
(1) A face area detection unit 111, a first parameter calculation unit 112, and a first extended reproduction time calculation unit 113 for calculating a reproduction time based on a face area.
(2) A saliency area detection unit 121, a second parameter calculation unit 122, and a second extended reproduction time calculation unit 123 for calculating the reproduction time based on the saliency area.
(3) A scene analysis unit 131, a third parameter calculation unit 132, and a third extended reproduction time calculation unit 133 for calculating a reproduction time based on scene analysis.
(4) A photo similarity detector 141, a fourth parameter calculator 142, and a fourth extended playback time calculator 143 for calculating a playback time based on the photo similarity.
(5) A reproduction history storage unit 151, a fifth parameter calculation unit 152, and a fifth extended reproduction time calculation unit 153 for calculating a reproduction time based on the reproduction history.
(6) Playback time calculation unit 16
(7) Image storage unit 17
(8) Image display unit 18
These functional components are realized by executing a program that causes a computer mounted on the image reproduction apparatus to function.

[(1) Calculation of playback time based on gaze area (face area) detection]
The “gaze area” refers to an area where the viewer visually pays attention. The detection of the gaze region includes, for example, tracking of eye movements using an infrared camera. According to the present invention, it refers to a portion that matches a predetermined condition obtained from an image. For example, it may be a face area of a subject person, a dense area of irregular color change, or a salient area in the center of an image. In the case of a face region, a portion that matches a pattern characterized by the relative position and size of the facial parts, the shape of eyes, nose, cheekbones, and jaws is detected. In the following description, the gaze area is described as a “face area”, but may be other gaze areas.

  According to FIG. 1, in order to calculate the reproduction time based on the face area, the face area detection unit 111, the first parameter calculation unit 112, and the first extended reproduction time calculation unit 113 are provided.

  The face area detection unit 111 extracts a feature amount for each photographic image stored in the memory card 101, and detects the face area of the subject person and the number N of the face areas. Here, the face area detection technique is detected by applying a face detector library based on a cascade classifier using OpenCV Haar Feature features to images converted into a plurality of resolutions (for example, non-detection). Patent Document 1).

A specific process of face area detection will be described. First, the input color image is converted into a luminance image, and an integral image is generated with the upper left coordinate of the image as the origin (0, 0).
I (x, y): luminance value of coordinates (x, y) of luminance image П (x, y): integral image П (x + 1, y) = П (x, y) + I (x + 1, y ) + I (x + 1, y-1)
П (x, y + 1) = П (x, y) + I (x, y + 1) + I (x-1, y + 1)
П (0,0) = I (0,0)
П (x, -1) = 0
П (-1, y) = 0

  A filter operation of a known Haar type feature amount is applied to the integral image П (x, y) (see, for example, Non-Patent Document 2). The Haar type feature amount is represented by the difference between the average luminance of adjacent white rectangular areas and the average luminance of black rectangular areas. The integrated value of this filter is calculated as R (a, b) within a certain rectangular size, and it is determined whether or not each rectangular area is a face image according to a face area determination unit calculated with a known ada boost. By performing this process in order from the upper left of the pixel, a face area with a certain rectangular size R (a, b) is detected. In addition, the size of the rectangle R (a, b) is changed depending on the original image, and this process is repeatedly scanned to detect a face region of an arbitrary size.

Next, based on the result of detecting the face area, the original color area below the face area on the image and the skin color area in the vicinity of the face area are set as “person areas”. Assuming that a person is often upright, the lower part of the face region, that is, the region in the range l _t (generally l _t > 3l _f ) centered on the face is defined as the “body”. Here, the definition of the unique color indicates a color having a large distance from other colors existing in the image. When the two colors expressed in RGB are (R ₁ , G ₁ , B ₁ ) (R ₂ , G ₂ , B ₂ ), for example, the color distance Dc is expressed as follows.
Dc = √ {(R ₁ -R ₂ ) ² + (G ₁ -G ₂ ) ² + (B ₁ -B ₂ ) ² }

  Here, the color space is not limited to RGB, and the following general color spaces (L * a * b *, L * u * v, HSV, HSL, YCrCb, etc.) may be used. The color distance is not limited to the Euclidean distance as described above, and a general distance scale may be used. The definition of the unique color represents a color in which this Dc is calculated for a color other than itself in the image and the sum Dsum is a value greater than a certain value. The skin color area is detected by searching for similar colors in the vicinity of the face area using the color information of the face area.

  Note that the face area detection unit 111 may use a face detection function in an image management program that automatically detects a subject person from a photographic image. Typical image management programs include Picasa (registered trademark) provided by Google (name tag function after v3.5), Microsoft's Windows Live Photo Gallery (registered trademark), Apple's iPhoto ' 09 (registered trademark).

The first parameter calculation unit 112 calculates an average face area size obtained by dividing the area of all face areas by the number of face areas N _H , and calculates a face area ratio value R _H of the average face area size with respect to the image area size.

The first extended reproduction time calculation unit 113 calculates a _first extended reproduction time t ₁ based on the number of face areas N _H / face area ratio value R _H. The calculated first extended reproduction time t ₁ is output to the reproduction time calculation unit 16.

  FIG. 2 is an explanatory diagram showing the reproduction time based on the face area in the present invention.

The playback time of the photograph shown in FIG. 2A is set as follows. In the photograph, two face areas are detected with respect to the image area size. The unit of size is not clear and means a relative size.
Face area size = 20
First face area size = 5
Second face area size = 3
At this time, the average face area size obtained by dividing the area of all face areas by the number of face areas N _H is as follows.
Average face area size = (5 + 3) / 2 = 4
Further, the face area ratio value R _H of the average face area size with respect to the image area size is as follows.
Face area ratio value R _H = average face area size / image area size
= 4/20 = 0.2
In this case, the first extension time t ₁ is as follows.
First extension time t ₁ = normalization coefficient ω ₁ · number of face areas N _H / face area ratio value R _H
= Ω ₁・ 2 / 0.2 = ω ₁・ 10

Here, the normalization coefficient ω is adjusted so that each term including the ω falls within a predetermined unit time (for example, −2 seconds to +2 seconds). That is, the normalization coefficient ω is adjusted so that the minimum value is assigned to −2 and the maximum value is assigned to +2 for the number of face areas N _H / face area ratio value R _H.

The playback time is set for the photograph shown in FIG. 2B as follows. In the photograph, four face areas are detected with respect to the image area size.
Face area size = 20
First face area size = 2
Second face area size = 3
Third face area size = 2
4th face area size = 1
At this time, the average face area size obtained by dividing the area of all face areas by the number of face areas N _H is as follows.
Average face area size = (2 + 3 + 2 + 1) / 4 = 2
Further, the face area ratio value R _H of the average face area size with respect to the image area size is as follows.
Face area ratio value R _H = average face area size / image area size
= 2/20 = 0.1
In this case, the first extension time t ₁ is as follows.
First extension time t ₁ = normalization coefficient ω ₁ · number of face areas N _H / face area ratio value R _H
= Ω ₁・ 4 / 0.1 = ω ₁・ 40

  According to FIG. 2, the extended reproduction time of the photographic image in FIG. 2B is longer than the extended reproduction time of the photographic image in FIG.

[(2) Calculation of playback time based on remarkable area]
In order to calculate the reproduction time based on the saliency area, the saliency area detection unit 121, the second parameter calculation unit 122, and the second extended reproduction time calculation unit 123 are included.

  The saliency area detection unit 121 extracts a feature amount from the photographic image and detects a saliency area other than the subject person. The target photographic image is preferably an image obtained by removing the face area detected by the face area detection unit 111. The saliency area detection unit 121 detects an object candidate from the image from which the face area has been removed (see, for example, Non-Patent Document 2).

  A specific process for detecting the saliency area will be described. First, the image is smoothed to remove image noise and reduce artifacts. For the smoothing, for example, a median filter or a Gaussian filter is used.

  Subsequently, color quantization is performed to reduce the color before calculating the color uniqueness. In a general 24-bit color, 8 bits (0 to 255) are assigned to each RGB color, and this is reduced to, for example, 12 stages for each color. In this case, it is expressed by 12 × 12 × 12 = 1728 colors.

A frequency distribution (histogram) in colors is calculated for the image reduced in color by the color quantization unit. Then, as a result of sorting in order of color frequency, for example, the subsequent processing is executed for the top 90% colors, thereby reducing the amount of calculation. For each color c _i of the image sorted in order of color frequency, a sum S (c _i ) of similarities with other color regions in the image is calculated.
Nc: Total number of colors in the image after color reduction and sorting cj (1≤j≤Nc-1): Other colors in the image for any color c _i fc _j : Ratio of cj in the image S (c _i ) = Σ ^Nc−1 _{j = 1} f _j D (c _i , c _j )
This calculates the ratio of the area of another color and the sum of the differences between the colors for an arbitrary color. This S (c _i ) is sorted in descending order, and is selected as a saliency area candidate in this order. For example, binarization is performed using a fixed threshold, the maximum area is detected, and noise is removed by performing expansion / reduction processing. Is a remarkable area.

The second parameter calculation unit 122 calculates a saliency area R _O that includes all the saliency areas.

The second extended reproduction time calculation unit 123 calculates a _second extended reproduction time t ₂ based on 1 / significant region area R _O. The calculated second extended playback time t ₂ is output to the playback time calculation unit 16.

  FIG. 3 is an explanatory diagram showing the reproduction time based on the salient region in the present invention.

The playback time of the photograph shown in FIG. 3A is set as follows. In the photograph, two salient areas are detected in addition to the face area.
First salient area size = 2
Second salient area size = 3
At this time, the saliency area R _O including all the saliency areas is as follows.
Saliency area R _O = first saliency area size + second saliency area size
= 2 + 3 = 5
In this case, the second extension time t ₂ is as follows.
Second extension time t ₂ = normalization coefficient ω ₂ · 1 / salience area R _O
= Ω ₂・ 1/5

The playback time of the photograph shown in FIG. 3B is set as follows. In the photograph, two salient areas are detected in addition to the face area.
First salient area size = 0.6
Second salient region size = 1.6
At this time, the saliency area R _O including all the saliency areas is as follows.
Saliency area R _O = first saliency area size + second saliency area size
= 0.6 + 1.6 = 2.2
In this case, the second extension time t ₂ is as follows.
Second extension time t ₂ = normalization coefficient ω ₂ · 1 / salience area R _O
= Ω ₂ 1/2.

  According to FIG. 3, the extended reproduction time of the photographic image in FIG. 3B is longer than the extended reproduction time of the photographic image in FIG.

In other embodiments, the reproduction time can be calculated for the saliency area in the same manner as the face area described above.
The saliency area detection unit 121 extracts a feature amount from the photographic image, and detects a saliency area other than the subject person and the number of saliency areas _NH .
The second parameter calculation unit 122 calculates an average saliency area size obtained by dividing the area of all saliency areas by the number of saliency areas N _H , and calculates a saliency area ratio value R _H of the average saliency area size to the image area size.
The second extended reproduction time calculation unit 123 calculates a _second extended reproduction time t ₂ based on the number of saliency areas N _H / the saliency area ratio value R _H.
Then, the playback time calculation unit 16 can further add / subtract the _second extended playback time t ₂ to the playback time t.

[(3) Calculation of playback time based on scene analysis]
In order to calculate the playback time based on the scene analysis, a scene analysis unit 131, a third parameter calculation unit 132, and a third extended playback time calculation unit 133 are provided.

  The scene analysis unit 131 extracts a feature amount from the photographic image and roughly detects a background scene. As a scene, for example, it is possible to distinguish between outdoor and indoor. In addition, it is possible to distinguish between nature (mountain, sea, etc.) or a sightseeing spot outdoors, and to distinguish a house or a party indoors. According to this technique, a local feature amount such as a SIFT (Scale-Invariant Feature Transform) feature amount is extracted from an image classified by scene, and the well-known BoF (Bags-of based on the distribution of features obtained by vector quantization of the feature is used. -Features) technology (for example, see Non-Patent Document 1). Specifically, local feature quantities (hundreds to thousands of feature vectors per image) are extracted from manually classified scene images, and local feature quantities of all scenes are set to about 2000, for example. Classify into clusters and create a codebook by vector quantization. A histogram is created from local feature amounts in each scene image using this codebook. The generated histogram becomes a feature amount unique to the scene, and a local feature amount is similarly generated for a photograph specifying the scene, and a histogram for the code book is created. The scene of the inputted photograph is determined according to the similarity of the histogram.

Based on the result of the scene analysis unit 131, the third parameter calculation unit 132 calculates a scene recognition value R _S having a higher value as it is outdoors and having a lower value as it is indoors.

The third extended playback time calculation unit 133 calculates a _third extended playback time t ₃ based on the scene recognition value R _S. The calculated third extended reproduction time t ₃ is output to the reproduction time calculation unit 16.

  FIG. 4 is an explanatory diagram showing the reproduction time based on the scene analysis in the present invention.

The reproduction time of the photograph in FIG. 4A is set as follows. The photo shows an indoor scene, and the scene recognition value R _S is detected as follows.
Scene recognition value R _S = 1 (low)
In this case, the third extension time t ₃ is as follows.
Third extension time t ₃ = normalization coefficient ω ₃ · 1
= Ω ₃ · 1

The playback time of the photograph shown in FIG. 4B is set as follows. The photograph shows an outdoor scene, and the scene recognition value R _S is detected as follows.
Scene recognition value R _S = 5 (high)
In this case, the third extension time t ₃ is as follows.
Third extension time t ₃ = normalization factor ω ₃ · 5
= Ω ₃・ 5

  According to FIG. 4, the extended reproduction time of the photographic image in FIG. 4B is longer than the extended reproduction time of the photographic image in FIG.

[(4) Calculation of playback time based on photo similarity]
In order to calculate the reproduction time based on the photo similarity, the photo similarity detection unit 141, the fourth parameter calculation unit 142, and the fourth extended reproduction time calculation unit 143 are provided.

  The photo similarity detection unit 141 detects whether or not the photo shooting times are continuous within a predetermined time range (for example, 60 seconds) and the similarity of the continuous photo images is similar to a predetermined threshold value or more. The similarity of a photographic image is determined by dividing the entire image into blocks and calculating a histogram of color information for each block. If the similarity is less than a predetermined threshold, it is determined that they are similar.

The fourth parameter calculator 142 calculates a continuous similarity number N _S of photographic images similar in succession.

The fourth extension reproduction time calculation unit 143 calculates a fourth extension reproduction time t ₄ based on 1 / Continuous similar number N _S. The calculated fourth extended playback time t ₄ is output to the playback time calculation unit 16.

  FIG. 5 is an explanatory diagram showing the reproduction time based on the photo similarity in the present invention.

According to FIG. 5A, it is determined that the three photographic images are similar. In this case, the reproduction time is set as follows.
Continuous similar number N _S = 3
In this case, the fourth extension time t ₄ is as follows.
Fourth extension time t ₄ = normalization coefficient ω ₄ · 1 / similar number of similar images N _S
= Ω ₄・ 1/3

According to FIG. 5B, it is determined that the two photographic images are similar. In this case, the reproduction time is set as follows.
Continuously similar number N _S = 2
In this case, the fourth extension time t ₄ is as follows.
Fourth extension time t ₄ = normalization coefficient ω ₄ · 1 / similar number of similar images N _S
= Ω ₄ 1/2

According to FIG. 5C, it is determined that there is no similar photographic image. In this case, the reproduction time is set as follows.
Continuously similar number N _S = 1
In this case, the fourth extension time t ₄ is as follows.
Fourth extension time t ₄ = normalization coefficient ω ₄ · 1 / similar number of similar images N _S
= Ω ₄ · 1/1

  According to FIG. 5, the extended reproduction time of the photographic image is long in the order of FIG. 5 (c)> FIG. 5 (b)> FIG. 5 (a).

[(5) Calculation of playback time based on playback history]
In order to calculate the reproduction time based on the reproduction history, the reproduction history storage unit 151, the fifth parameter calculation unit 152, and the fifth extended reproduction time calculation unit 153 are included.

The reproduction history storage unit 151 records the average reproduction time C _L and the number of reproductions C _T for each reproduced photographic image.

The fifth parameter calculator 152 obtains an average reproducing time in the image C _L and Views C _T. As for a photographic image that is not recorded in the reproduction history storage unit 151, both the average playing time C _L and Views C _T, a value of 0.

The fifth extended playback time calculation unit 153 calculates a _fifth extended playback time t ₅ based on 1 / (average playback time C _L × playback count C _T ). The calculated fifth extended reproduction time t ₅ is output to the reproduction time calculation unit 16.

  FIG. 6 is an explanatory diagram showing the reproduction time based on the reproduction history in the present invention.

For the photographic image in FIG. 6A, the playback time is set as follows.
Number of reproductions C _T = 7 Average reproduction time C _L = 5 seconds In this case, the fifth extension time t ₅ is as follows.
Fifth extension time t ₅ = normalization coefficient ω ₅ · 1 / (average reproduction time C _L × number of reproductions C _T )
= Ω ₅・ 1 / (5 × 7)

For the photographic image in FIG. 6B, the playback time is set as follows.
Number of reproductions C _T = 4 Average reproduction time C _L = 3 seconds In this case, the fifth extension time t ₅ is as follows.
Fifth extension time t ₅ = normalization coefficient ω ₅ · 1 / (average reproduction time C _L × number of reproductions C _T )
= Ω ₅・ 1 / (3 × 4)

For the photographic image in FIG. 6C, the playback time is set as follows.
Number of reproductions C _T = 1 Average reproduction time C _L = 2 seconds In this case, the fifth extension time t ₅ is as follows.
Fifth extension time t ₅ = normalization coefficient ω ₅ · 1 / (average reproduction time C _L × number of reproductions C _T )
= Ω ₅・ 1 / (2 × 1)

  According to FIG. 6, the extended reproduction time of the photographic image is longer in the order of FIG. 6 (c)> FIG. 6 (b)> FIG. 6 (a).

[(6) Playback time calculation unit 16]
The reproduction time calculation unit 16 calculates a reproduction time t obtained by adding (or subtracting) at least the first extended reproduction time t ₁ to the reference reproduction time t ₀ of the photographic image. Further, the playback time calculation unit 16 further adds (or subtracts) any one of the second extended playback time t ₂ to the fifth extended playback time t ₅ or a combination thereof to the playback time t. For example, the standard reproduction time t ₀ = 3 seconds and each extended reproduction time may be normalized from −2 seconds to +2 seconds.

[(7) Image Accumulator 17]
For each photographic image, a reproduction time t is associated and stored.

[(8) Image display unit 18]
The image display unit 18 reproduces each photographic image so as to be displayed on the display 102 for a reproduction time t corresponding to the photographic image.

  FIG. 7 is an explanatory diagram illustrating an example in which the reproduction time is shortened and an example in which the reproduction time is increased.

  FIG. 7A shows an example in which the reproduction time is shortened based on FIGS. 2 to 6 described above. According to the photographic image in FIG. 7A, the number of face areas is small, the face area size is large, the saliency area size is large, there are many photographic images with high similarity in indoor scenes, and past playback times. Is long.

  FIG. 7B shows an example in which the reproduction time becomes longer based on FIGS. 2 to 6 described above. According to the photographic image in FIG. 7B, the number of face areas is large, the face area size is small, the saliency area size is small, the outdoor scene has few photographic images with high similarity, and the past reproduction time Is short.

  As described above in detail, according to the image playback device, program, and method of the present invention, it is possible to play back a photographic image in a playback time that is optimal for the viewer in accordance with the content of the subject in the photographic image. . In particular, without setting the playback time in advance, the content can be played back in a playback time that is easy for the viewer to understand according to the complexity of the subject in the photographic image.

  Various changes, modifications, and omissions of the above-described various embodiments of the present invention can be easily made by those skilled in the art. The above description is merely an example, and is not intended to be restrictive. The invention is limited only as defined in the following claims and the equivalents thereto.

DESCRIPTION OF SYMBOLS 1 Image reproduction apparatus 101 Memory card 102 Display 111 Face area detection part 112 1st parameter calculation part 113 1st extended reproduction time calculation part 121 Remarkable area detection part 122 2nd parameter calculation part 123 2nd extended reproduction time calculation Unit 131 scene analysis unit 132 third parameter calculation unit 133 third extended playback time calculation unit 141 photo similarity detection unit 142 fourth parameter calculation unit 143 fourth extended playback time calculation unit 151 playback history storage unit 152 5 parameter calculation unit 153 5th extended reproduction time calculation unit 16 reproduction time calculation unit 17 image storage unit 18 image display unit

Claims (10)

In an image reproduction device that continuously displays a plurality of images on a display visually recognized by a viewer,
A gaze area detecting means for extracting a feature amount from the image and detecting a gaze area by the viewer and the number of gaze areas _NH ;
A first parameter calculating means for calculating an average gaze area size obtained by dividing the area of all gaze areas by the number of gaze areas N _H and calculating a gaze area ratio value R _H of the average gaze area size with respect to the image area size;
First extended playback time calculating means for calculating a _first extended playback time t ₁ based on the number of watched areas N _H / gaze area ratio value R _H ;
Reproduction time calculating means for calculating a reproduction time t obtained by adding or subtracting the extended reproduction time t ₁ to the reference reproduction time t ₀ of the image;
Image storage means for storing the reproduction time t for each image;
An image reproduction apparatus, wherein the image is reproduced so as to be displayed on the display for the reproduction time t. The image is a photographic image;
The gaze area is a face area of a subject person,
The image reproduction apparatus according to claim 1, wherein the gaze area detection unit is a face area detection unit. A salient region detecting means for extracting a feature amount from the photographic image and detecting a salient region other than the subject person;
A second parameter calculating means for calculating a saliency area R _O combining all the saliency areas;
A second extended playback time calculating means for calculating a _second extended playback time t ₂ based on 1 / significant area R _O ;
3. The image reproducing apparatus according to claim 2, wherein the reproduction time calculating means further adds / subtracts a _second extended reproduction time t2 to / from the reproduction time t. A saliency area detecting means for extracting a feature amount from the photographic image and detecting a saliency area other than the subject person and the saliency area number _NH ;
Wherein calculating the average saliency area size obtained by dividing the salient region all areas in the salient region number N _H, and the second parameter calculating means for calculating a salient region ratio value R _H of the average salient region size to the image region size ,
A second extended reproduction time calculating means for calculating a _second extended reproduction time t ₂ based on the number of remarkable areas N _H / the remarkable area ratio value R _H ;
Further comprising
3. The image reproducing apparatus according to claim 2, wherein the reproduction time calculating means further adds / subtracts a _second extended reproduction time t2 to / from the reproduction time t. A scene analysis means for extracting a feature amount from the photographic image and detecting a background scene;
Third parameter calculating means for calculating a scene recognition value R _S having a higher value as it is outdoors and having a lower value as it is indoors;
A third extended reproduction time calculating means for calculating a _third extended reproduction time t ₃ based on the scene recognition value R _S ;
5. The image reproduction apparatus according to claim 2, wherein the reproduction time calculation unit further adds or subtracts a _third extended reproduction time t ₃ to the reproduction time t. 6. Photo similarity detection means for detecting whether or not the photo shooting time is continuous within a predetermined time range and the similarity of the continuous photo images is similar to a predetermined threshold value or more,
A fourth parameter calculating means for calculating a continuous similarity number N _S of photographic images similar in succession,
A fourth extended playback time calculating means for calculating a _fourth extended playback time t ₄ based on the 1 / continuous similar number N;
6. The image reproduction apparatus according to claim 2, wherein the reproduction time calculation unit further adds / subtracts a _fourth extended reproduction time t4 to / from the reproduction time t. Reproduction history storage means for recording the average reproduction time C _L and the number of reproductions C _T in the reproduced photographic image;
A fifth parameter acquisition means for acquiring an average reproducing time in the image C _L and Views C _T,
And 5th extended reproduction time calculation means for calculating a _fifth extended reproduction time t ₅ based on 1 / (average reproduction time C _L × number of reproductions C _T ),
The image reproduction device according to any one of claims 2 to 6, wherein the reproduction time calculation unit further adds / subtracts a _fifth extended reproduction time t5 to the reproduction time t.   The image reproduction device according to claim 1, wherein the extended reproduction time calculation unit further multiplies a normalization coefficient that is normalized by a predetermined unit time. In a program for functioning a computer mounted on a device that continuously displays a plurality of images on a display visually recognized by a viewer,
A gaze area detecting means for extracting a feature amount from the image and detecting a gaze area by the viewer and the number of gaze areas _NH ;
A first parameter calculating means for calculating an average gaze area size obtained by dividing the area of all gaze areas by the number of gaze areas N _H and calculating a gaze area ratio value R _H of the average gaze area size with respect to the image area size;
First extended playback time calculating means for calculating a _first extended playback time t ₁ based on the number of watched areas N _H / gaze area ratio value R _H ;
Reproduction time calculating means for calculating a reproduction time t obtained by adding or subtracting the extended reproduction time t ₁ to the reference reproduction time t ₀ of the image;
Image storage means for storing the reproduction time t for each image;
An image display program for reproducing the image so as to be displayed on the display for the reproduction time t. In an image display method in an apparatus for continuously displaying a plurality of images on a display visually recognized by a viewer,
A first step of extracting a feature amount from the image and detecting a gaze area by a viewer and the number of gaze areas N _H ;
A second step of calculating an average gaze area size obtained by dividing the area of all gaze areas by the number of gaze areas N _H and calculating a gaze area ratio value R _H of the average gaze area size with respect to the image area size;
A third step of calculating a _first extended reproduction time t ₁ based on the number of gaze areas N _H / gaze area ratio value R _H ;
A fourth step of calculating a reproduction time t obtained by adding and subtracting the extended reproduction time t ₁ to the reference reproduction time t ₀ of the image;
And a fifth step of accumulating the reproduction time t for each image, wherein the image is reproduced so as to be displayed on the display only for the reproduction time t.

Images