JP2006140559A - Image reproducing apparatus and image reproducing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image reproducing apparatus for reproducing a slide show while providing a visual effect in response to the relation between images on the basis of the relation between the images, so as to allow a user to be able to easily grasp the relation between the images thereby allowing the user to feel an interest in the slide show. <P>SOLUTION: The image reproducing apparatus includes: an image input section 101 for receiving image data; an image feature extract section 102 for extracting an image feature quantity; an image storage section 103 for storing image packets; a music input section 104 for receiving music data; a music feature extract section 105 for extracting a feature quantity of music from the music data; a music storage section 106 for storing music packets; a menu entry section 107 whereby the user carries out menu operation; a slide show generating section 108 for generating the slide show in matching with a condition entered from the menu entry section 107; a display section 109 for a screen on which the slide show is displayed; a relation analysis section 112 for calculating the relation between the images on the basis of meta information included in the images; and a scenario analysis section 113. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image reproducing apparatus for recording and reproducing digital photographs, and more particularly to an image reproducing apparatus having a slide show function for continuously displaying a plurality of digital photographs.

  Conventionally, as a method for continuously reproducing still images, there is a display method called a slide show in which images are sequentially displayed at predetermined time intervals. For example, it is realized by a PC software such as a photo cinema or a movie shaker, a mobile phone with a camera, or the like. ing.

  Here, the slide show is a so-called digital picture-story show in which a digital still image to be displayed is switched by a constant time interval or a trigger such as a user instruction as in the case of a slide projector. In recent years, depending on the application, BGM is added, or transitions such as fade, wipe, and rotation are added when switching between digital still images to further improve the entertainment property.

  As an example of a conventional slide show apparatus, there is one that continuously reproduces using a CD-ROM (see, for example, Patent Document 1). FIG. 16 is a configuration diagram of a conventional slide show apparatus 1600 described in Patent Document 1. In FIG.

In the slide show apparatus 1600, the CD-ROM 1606 is set in the CD-ROM reading unit 1605, and the start instruction of the slide show execution program 1606a is issued by the input instruction unit 1602. The arithmetic control unit 1601 reads the slide show execution program 1606a and functions as a slide show execution unit. When executing the slide show, the arithmetic control unit 1601 first reads the order table 1606b, extracts image files from the image file group 1606c in the order defined in the order table 1606b, and causes the display unit 1603 to display them. The hard disk 1604 stores various data as necessary.
JP 2001-103415 A (pages 3 to 5, FIG. 1)

  However, in the conventional configuration, since the image file is only reproduced according to the order table, there is a problem that it is not interesting and surprising as viewed from the user. Such a problem becomes more serious especially when the number of images increases and many similar images are accumulated. For example, when a plurality of people photograph similar images at the same event such as watching sports or weddings, they simply feel bored by simply reproducing them according to the order table.

  In recent years, the use of digital photographs using digital cameras, mobile phones, and the like has become widespread, and the number of digital images possessed by individuals has increased, and a large number of images taken by users have been stored. These images include many similar or related images. Thus, how to use and select a large number, similar, or related digital images that are captured by the user and stored in the terminal device is a problem. ing.

  Further, in the conventional slide show playback device, communication using images between the slide show devices is not taken into consideration, and it is an issue to provide a slide show corresponding to the recent networking.

  The present invention is to solve the above-described conventional problems, and provides an image reproducing apparatus that performs recreational image reproduction while grasping the relationship between images using a plurality of captured images. The purpose is to do. It is another object of the present invention to provide an image reproducing apparatus that can realize communication with other photographers using still images.

  In order to solve the above-described conventional problems, an image reproduction device of the present invention is an image reproduction device that performs reproduction using image data, the image feature extraction unit that extracts an image feature amount from the image data, Based on the image information related to the image data and the image feature amount, the relevance analysis means for analyzing the relevance, which is information indicating similarity with other images, and using the image data based on the relevance And a slide show generating means for reproducing the slide show.

  Further, the slide show generating means of the image reproducing apparatus according to the present invention includes an image selecting unit that selects an image group that is a collection of images to be reproduced based on the relevance level, and an image selection unit that selects the image group based on the relevance level. A visual effect determination unit that determines a visual effect at the time of reproduction; and a scenario generation unit that generates a scenario that is a reproduction order of images included in the image group based on the relevance level, and the selected image group The determined visual effects are used to reproduce the scenarios in order.

  With these configurations, even when a slide show is reproduced using a large amount of similar image data, the image feature extraction unit extracts the image feature amount, and the analysis is performed using the image feature amount and the image information. Based on the degree, the image selection unit can select the image, the visual effect determination unit can determine the visual effect at the time of slide show playback, and the scenario generation unit can generate the scenario. This makes it possible to perform slide show reproduction that is easy and interesting for the user.

  In addition, the visual effect determination unit of the image reproduction device according to the present invention includes, as the visual effect, divided display for dividing and displaying a screen into one or more small screens, display time for the image data, and how to move the image data It is characterized by determining. The visual effect determination unit may determine to perform the divided display during reproduction between the image data according to the degree of association.

  With these configurations, the visual effect determination unit can generate a slide show based on the relevance level in order to determine to give a visual effect such as split display during reproduction between images with low relevance level. To do.

  Further, the image selection unit of the image reproduction device according to the present invention selects a set of some images as a subset from the entire image group based on the relevance degree, and the relevance analysis means includes It is characterized by analyzing the degree of association between images. The image selection unit may select a set of image data with close photographing times as the subset as the subset.

  With these configurations, the image selection unit selects a part of the entire image group as a subset, and analyzes the relevance of the image group in the subset, so that the shooting time is close, etc. It is possible to appropriately play back the slide show using image data having similar degrees of association.

  The image selection unit of the image reproduction device according to the present invention selects an N-sequence image group from an M-sequence image group when M and N are integers of 1 or more, and the visual effect determination unit includes a screen Is divided into N windows, and the N series of images are determined to be divided and displayed. Further, the M series is divided by a photographer.

  With these configurations, it is possible to reproduce and display a slide show while performing divided display based on the degree of association using image data taken by another photographer divided for each series.

  In addition, the image selection unit of the image reproduction device according to the present invention, when L is an integer greater than or equal to 1, based on at least one part of the image information or the degree of association of the L-th selected image group, The second image group is selected.

  With this configuration, the image selection unit selects the image group to be reproduced next in the slide show based on the image information or the degree of association of the image group selected immediately before. It is possible to realize a slide show generation that feels interesting in both recall and surprise.

  In order to achieve the above object, the present invention can be realized as an image reproducing method using characteristic constituent means of the image reproducing apparatus as steps, or as a program including all these steps. The program is not only stored in a ROM or the like provided in the image reproducing apparatus, but can also be distributed via a recording medium such as a CD-ROM or a communication network.

  According to the image reproducing device of the present invention, a plurality of related images are divided and displayed, for example, between images having a low degree of association according to the strength of the relation between images, or continuous shooting with a high degree of association. In this case, the slide show can be reproduced while providing a visual effect such as split display. For this reason, it is easy to grasp the relationship between images, and it is possible to reproduce images that are interesting for the user.

  Embodiments of an image reproducing apparatus according to the present invention will be described below with reference to the drawings.

(Embodiment 1)
The image reproducing apparatus according to the first embodiment is characterized in that the degree of association between images is extracted and a slide show is reproduced while providing a visual effect based on the degree of association. The slide show is a method in which digital photos are read from the image storage unit, music is read from the music storage unit, and images are reproduced according to the music. Rather than simply displaying digital photos in sequence, a visual effect with movement such as panning, zooming, enlargement, reduction, and color change is applied to the image in accordance with the music, and a dynamic feeling is produced. Other visual effects include transition effects such as fade-in, slide-in, and wipe when switching images, and 3D effects such as pasting images on cubes and cylinders, or drawing reflections by applying light to images. is there. Note that the image reproduction apparatus according to the first embodiment divides and displays an image using the degree of association between images, which is also an example of a visual effect.

  The image data used in the image reproducing apparatus according to the present invention is a digital photograph mainly taken with a digital still camera or a mobile phone, but may be a moving picture, CG, or the like. The images need not be limited to those obtained by digital still cameras. The source is not limited to commercially available content, download from the Web, broadcast programs, etc., and the image data is JPEG, bitmap, etc. It doesn't matter. Further, the music data may be either a wave format such as AAC or MP3, or a data format such as MIDI, and the format is not particularly limited.

FIG. 1 is a block configuration diagram of an image reproduction device 100 according to the first embodiment.
The image input unit 101 is a processing unit for inputting image data.

  The image feature extraction unit 102 is a processing unit that analyzes image data and extracts an image feature amount from an image. In addition, the image feature quantity extracted by the image feature extraction unit 102 is added to data composed of an image ID and image data to generate an image packet 110.

The image storage unit 103 is a database that stores the generated image packets 110.
The music input unit 104 is a processing unit for inputting music data.

  The music feature extraction unit 105 extracts music feature values from the music data. In addition, the music feature quantity extracted by the music feature extraction unit 105 is added to data composed of a music ID and music data to generate a music packet 111.

The music storage unit 106 is a database that stores the generated music packets 111.
The menu input unit 107 is an interface such as a keyboard for a user to perform a menu operation for generating a slide show. This menu operation includes selection of styles such as “formal”, “party”, and “travel”.

  The slide show generation unit 108 creates a slide show according to the conditions input from the menu input unit 107.

  The display unit 109 is created by the slide show generation unit 108, and the scenario analysis unit displays the analyzed slide show on the screen.

  The relevance analysis unit 112 reads an image from the image storage unit 103 and calculates the relevance between the images from meta information included in the image.

  The scenario analysis unit 113 analyzes the scenario generated by the slide show generation unit 108 based on the degree of association between images, and analyzes the image ID and visual effect at the time of the slide show created as the scenario.

  In FIG. 1, the image packet 110 and the music packet 111 are stored in the image storage unit 103 and the music storage unit 106 as follows.

  The image data is input from the image input unit 101, and the image feature extraction unit 102 extracts the image feature amount. Then, the image feature extraction unit 102 converts the image ID 20, the photographer ID 21, the camera information 22, the subject information 203 as the extraction result, the physical feature 24, and the original image data 25 into the form of the image packet 110, and the image storage unit 103. To accumulate.

  The music data is input from the music input unit 104, and the feature extraction is performed by the music feature extraction unit 105. Then, the music feature extraction unit 105 stores the music ID 30, the rhythm information 31 as the extraction result, the music sensitivity information 32, and the original music data 33 in the form of the music packet 111 in the music storage unit 106.

Next, the flow of slide show generation will be described from the viewpoint of user operation.
First, the user selects a style with the menu input unit 107. Here, the style determines the atmosphere of the entire slide show, and is presented in terms of “formal”, “party”, and “travel”. Here, if the style is “formal”, the calm visual effect (movement) is mainstream, and if it is “travel”, the light visual effect is mainstream, so that the slide show suitable for the image is reproduced.

  Next, the user selects an image for which a slide show is requested. The image is roughly specified by the user. For example, when a folder containing the image is specified, the slide show generation unit 108 selects an appropriate image from the folder. Music is automatically selected based on rhythm information and music sensitivity information according to the style. For example, select “Formal” as the style, and specify the image folder containing the photo of the reception. Music that matches the image of “formal”, for example, classical music is selected. Of course, the user may specify both images and music. The slide show generation unit 108 generates a slide show scenario according to an instruction from the menu input unit 107, and the scenario analysis unit 113 decodes the scenario and reproduces and displays it on the display unit 109. Note that it is possible that the user selects a style after selecting an image, and therefore the procedure of selecting an image and selecting a style may be reversed.

  FIG. 2 is a data configuration diagram of an image packet 110 including data extracted by the image feature extraction unit 102 according to the first embodiment.

  The image packet 110 includes an image ID 200 that is a unique number of the image, a photographer ID 201 that is an ID given to each photographer, camera information 202 that is information obtained by the camera, subject information 203 that is subject information, Physical feature information 204 that is physical information based on luminance, color, and the like in the image, and image data 205 that is the actual data of the image are included.

  The camera information 202 includes a shooting time (June 30, 2004 in FIG. 2), a shooting location (135 degrees 10 minutes 20 seconds east longitude, 33 degrees 33 minutes 40 seconds north latitude), a focal length (3 m), and an aperture value (f5 And 6) physical shooting conditions.

  The subject information 203 includes the number of persons in the photo (one person), the name of the person (Mr. A), the position (the center coordinates of the face in the image), and the size (the upper left and lower right coordinate values of the circumscribed rectangle of the face). Etc. are included.

  The physical feature information 204 includes, for example, an average luminance, an average color, a center frequency, and the like in each area obtained by dividing an image into a plurality of areas. For example, the average luminance is a numerical value represented by 0 or more and 255 or less in the case of 256 gradations, the average color is RGB display (in the case of 256 steps, (204,0,0) is red, (183,65,14 ) Indicates a red rust color), and the center frequency is the center value of the spatial frequency components in the x and y directions of the luminance distribution in the region.

  As indicated by the arrows in the figure, the image ID 200, the photographer ID 201, the camera information 202, and the image data 205 are information acquired from the image input unit 101, and the subject information 203 and the physical feature information 204 are image features. This is information acquired from the extraction unit 102.

  Further, the image ID 200, the photographer ID 201, the camera information 202, the subject information 203, and the physical feature information 204, which are additional information other than the image data 205, are meta information 206.

  FIG. 3 is a data configuration diagram of the music packet 111 including the extraction information extracted by the music feature extraction unit 105 according to the first embodiment.

  The music packet 111 includes a music ID 300 that is a unique identification number for each song, rhythm information 301, music sensitivity information 302, and music data 303 that is music entity data.

  The rhythm information 301 is a tempo, rhythm, etc. indicating the speed of music. The music sensitivity information 302 indicates the sensitivity extracted from the music data. The sensibility of music is expressed by intensity and warmth by scoring (numerizing) the degree. When the maximum score is 50 in the music sensitivity information 302 of FIG. 3, the intensity = −40 and the warmth = −40 indicate “pretty static and cold”.

  Prior to slide show generation, image feature amounts need to be extracted in advance. FIG. 4 shows an operation procedure of subject information extraction in the image feature extraction unit 102 of the image reproduction apparatus 100 according to the first embodiment. It is a flowchart.

  First, the image feature extraction unit 102 receives an input of an image from the image input unit 101 (S401).

  Then, the image feature extraction unit 102 performs a person detection process to determine whether or not a person is detected (S402). When a person is detected (Y in S402), it is further determined whether or not a person can be specified (S403). Person identification is performed by comparison with registered person images. If there is a high similarity to the registered image, there is a high possibility that it can be specified. If a person can be specified (Y in S403), a person name is assigned (S404). If a person cannot be specified (N in S403), it is simply determined that there is a person (S405).

  The image feature extraction unit 102 writes these determination results in the subject information 203 of the image packet 110 and records them in the image storage unit 103 (S407). It should be noted that the number of people and the position may be simultaneously performed when detecting a person. Further, in the person detection, when no person is detected (N in S402), it is determined that the person is not photographed (S406), so it is not necessary to write the subject information 203.

  FIG. 5 is a flowchart showing an operation procedure of physical quantity extraction in the image feature extraction unit 102 of the image reproduction device 100 according to the first embodiment.

  First, the image feature extraction unit 102 receives an input of an image from the image input unit 101 (S501).

  Next, the image feature extraction unit 102 divides the image into a plurality of regions (S502). This division processing is performed with a predetermined pattern, but the division pattern may be decided after a rough analysis.

  Next, the image feature extraction unit 102 extracts luminance information, color information, and frequency information for each of the divided regions (S503), calculates a physical feature amount, and writes it to the physical feature information 204 of the image packet 110 ( In step S <b> 504, the image packet 110 is recorded in the image storage unit 103. It should be noted that the extraction of subject information and physical feature values shown in FIGS. 4 and 5 may be performed in any order.

  FIG. 6 is an explanatory diagram of a subset that is a set that is a calculation target of the degree of association between images in the image reproduction device according to the present invention. In FIG. 6, the vertical series and the horizontal time axis are shown, and the captured images are shown as a1, a2,... On the time axis.

  In the description of the first embodiment, the series means a photographer, the A series is an image taken by Mr. A, the B series is taken by Mr. B, the C series is taken by Mr. C, and the D series is an image taken by Mr. D. That is, in the description of the first embodiment, the image group is divided into four series of A, B, C, and D for each photographer, and each image is photographed along time.

  In FIG. 6, the A series photographed by Mr. A is photographed when a1, a2, and a3 are adjacent to each other, and a4 and a5 are photographed after some time. In the B series photographed by Mr. B, b1 and b2 are photographed at a time close to a2 and a3, and b3 is photographed at a time close to a4 with a little time. The C and D series have the same view.

  Then, the degree-of-association analysis unit 112 automatically detects a set of sequences that are close in time as a subset. In this figure, the portions surrounded by dotted lines are subsets, a1, a2, b1, c1, c2, and d1 are subset 1, a2, a3, b1, b2, c2, and d2 are subset 2, a4, a5, b3, c3, d3 and d4 are images included in the subset 3. It should be noted that the time for creating the subset can be different depending on the type of event to be photographed, for example, it is on the order of a few minutes for weddings and date / time orders for travel etc. It is.

  FIG. 7 is a flowchart showing an operation procedure of the relevance analysis unit 112 included in the image reproduction device 100 according to the first embodiment.

  First, when the degree-of-association analysis unit 112 first acquires two images to be compared from the image storage unit 103 (S701), the degree-of-relationship for each image is extracted. In the present invention, the degree of association is basically calculated by comparing two images.

  Next, the relevance analysis unit 112 calculates each evaluation value from various similarities such as time, and determines that the relevance is higher as the evaluation value is larger. It is the time (S702), the place (S703), the subject (S704), and the physical feature (S705) that become similar to the evaluation value calculation target. If the time and place match, it is the highest point (for example, 100 points), and the evaluation value is lowered as the distance increases. Regarding the time, not only all the similarities of the year, month, and day, but also the similarities of only the date, time, and part of information may be evaluated. The location information can be acquired using a GPS or the like provided in the mobile phone. The time information needs to match the time of each terminal that takes a digital image using radio waves or the like. In addition, for the subject and physical information, the absolute value is not as clearly as the time, so the evaluation value is adjusted so that the range from 0 to the highest point is distributed to some extent. The object information evaluates the similarity of the object in the image, and the physical information evaluates the layout similarity such as the color / luminance distribution.

  Then, the relevance analysis unit 112 weights and adds these evaluation values (S706). The weighting coefficient may be a fixed value or may be changed dynamically depending on the purpose. For example, in the case of an event such as a reception, the time and subject are emphasized. In general, the weights are weighted in the order of time, place, subject, and physical quantity.

  Finally, the relevance analysis unit 112 determines whether the evaluation value after weighted addition is larger than a preset threshold value (S707), and determines whether the evaluation value is larger or smaller than the relevance threshold value. In this case, when the degree of association is larger than the threshold (Y in S707), the degree of association is determined to be strong (S709). When the degree of association is smaller than the threshold (N in S707), the degree of association is weak. Determination is made (S708), and the operation of the series of relevance analysis units 112 is terminated. In the description of this figure, the degree of relevance uses two divisions of large and small, but the present invention is not limited to this, and the division may be performed more finely by points.

  In addition, in the first embodiment, since the relevance analysis unit 112 calculates the relevance level in a series of processing flows, the relevance level is temporarily held dynamically by software. Since this is inefficient, it may be calculated at the time of image registration and stored in the meta information 206. In this case, it is inefficient to leave the degree of association with all images, so for example, the image ID and the evaluation value for the top ten sheets with the highest degree of association may be left in the meta information 206 in the image packet 110 (FIG. 2). (Not shown).

  Furthermore, it is ideal to calculate the degree of association with all images, but in order to reduce the calculation cost, primary screening is performed at the time, and a detailed evaluation value is obtained after narrowing the range to a certain degree. Is preferred. Here, the time close to the time is not limited to the time close to the date, but may be close to the date. In this way, images of the same month in different years are evaluated as having a high degree of relevance, and are displayed side by side on the screen, etc., which is effective for recalling memories and improving entertainment when users play back slideshows. Is possible.

Also, in the description of this figure, the comparison of two images has been described in order to obtain the degree of association. However, in the case where there are a plurality of images, it can be used for slide show scenario generation in the following manner. For example, in three images a, b, and c, it is assumed that the degree of association has been evaluated between two images, a and b, b and c, and c and a. The relevance threshold is 10, and if it is greater than this, the relevance is high.
(1) In the case of the relevance degree 20 of a, b, the relevance degree 30 of b, c, and the relevance degree 50 of c, a: In this case, the image set (a, b, c) is a group having a high relevance degree. deal with.
(2) When the degree of association 20 is a, b, degree 30 is b, c, and degree 1 is c, a: In this case, the image set (a, b) (b, c) It may be handled as a high group, and (a, b, c) may be handled as a highly related group.
(3) When the degree of association 20 is a, b, the degree of association 5 is b, c, and the degree of association 1 is c, a: In this case, the image set (a, b) is treated as a group having a high degree of association.

  FIG. 8 is a flowchart showing an operation procedure of the slide show generation unit 108 of the image reproduction apparatus 100 according to the first embodiment. The operation procedure at the time of slide show generation and playback is roughly divided into “image income, relevance analysis”, “slide show generation”, and “slide show playback”.

  First, the slide show generation unit 108 acquires A, B, C, and D series images from the image storage unit 103 (S801).

  Then, the slide show generating unit 108 performs subset detection from each of the A, B, C, and D series (S802).

  Next, the slide show generation unit 108 checks the degree of association between images (S803). Here, it is determined whether or not the degree of association is strong.

  Then, the slide show generation unit 108 gives a visual effect according to the style input from the user via the menu input unit 107, but when it is determined that the degree of association is weak (N in S803). When it is determined that the degree of relevance is high (Y in S803), the visual effect for the image with high degree of relevance is given (S805). In the description of the present embodiment, the high degree of association means that the shooting date and time, the shooting location, the photographer, the subject information, the luminance information, and the color information are similar, and that the degree of association is weak. The shooting date / time, shooting location, photographer, subject information, luminance information, and color information are not similar. In addition, the visual effect has a difference in the screen division method, the image display time, and the image moving method according to the degree of relevance. This will be described later with reference to FIGS.

  In addition, the slide show generation unit 108 determines whether or not a length scenario determined for the above processing has been generated (S807). If no scenario has been generated up to the determined length (N in S807) ), The steps from S801 to S806 are repeated, and when a scenario is generated up to a predetermined length (Y in S807), the scenario is output to the scenario analysis unit 113 in the form of a scenario, and is decoded by the scenario analysis unit 113. Then, the slide show is reproduced by outputting to the display unit 109 (S808). A scenario is a description of an image and visual effects over time.

  FIG. 14 is a reference diagram showing a scenario data configuration example 1400 created based on the degree of association between images in the slide show generation unit 108 of the image playback device of the present invention. As shown in this figure, the scenario 1400 includes, for example, (1) playback time (2) image allocation (3) visual effects (for example, zoom to make it flashy, messed up based on relevance) (4) BGM is described.

  Next, specific slide show generation assuming a reception scene will be described with reference to FIGS.

  FIG. 9 is a time chart showing the state of image shooting. As shown in the figure, in the description of the present embodiment, the reception party is assumed to proceed in the order of four sections “before start”, “cake sword”, “joy chat”, and “candle service”.

  And in this Embodiment 1, four photographers image | photograph an image and let each image group be A, B, C, and D series. Here, before the start of “Section 1”, two pictures a1 and a2 in the A series, one each of b1, c1 and c1 in the B, C, and D series are photographed, and the cake in “Section 2” is entered. A large number of images are taken in a short time in the candle service of the sword and “Section 4”, and in the chat of “Section 3”, they are taken discretely as before the start.

  In the figure, the subset is indicated by a dotted line. For example, the relevance analysis unit 112 can automatically detect a subset by setting a subset that is close in time based on the shooting time. In the present invention, in preparation for a discrete case, that is, when shooting frequency is low, such as during a chat, if a certain number of sheets are collected, it is set as a subset.

  FIG. 10 and FIG. 11 are time chart diagrams showing a state of image shooting, and screen image diagrams showing a state of image reproduction on a multi-screen corresponding to this time chart diagram. FIG. 10 shows screen image diagrams corresponding to the first “section 1” and “section 2”, and FIG. 11 shows screen image diagrams corresponding to “section 3” and “section 4”.

  First, as shown in FIG. 10, before the start of “Section 1”, the state of the venue, participants, etc. are photographed, so there is a large difference in subjects and the time varies, so the degree of association is low It is evaluated. As a result, the screen is divided into four, and four images are displayed there. Here, four sheets with the closest shooting times are arranged as much as possible. In principle, an A-series image is displayed at the upper left, a B-series image is displayed at the upper right, a C-series image is displayed at the lower left, and a D-series image is displayed at the lower right. It is also possible to give a quiet visual effect and simply switch the image.

  Next, when the cake sword of “Section 2” shown in FIG. 10 is reached, scenes where the bride and groom enter the cake are photographed from slightly different angles in all series. For this reason, the similarity of the subject is increased and the time interval is narrow, so that the similarity of the photographing time is also increased. As a result, it is evaluated that the degree of association between images taken in this “section 2” is high. Therefore, a difference is made with “Section 1” before the start, and a flashy effect is displayed on the screen.

  In the first embodiment, first, the screen is switched from four screens to one screen, a3, a4, a5 are sequentially displayed on the full screen, and then divided into four screens again, a3, b2, c2, and so on. Four images taken at approximately the same time such as d2 are displayed simultaneously. As a result, an impact at the time of slide show reproduction on a large screen can be given, and an effect can be obtained as if the same scene was shot at multiple angles by dividing the screen. In addition, visual effects with zooming and panning are added.

  Next, it enters the “section 3” of the chat shown in FIG. In “Section 3”, the degree of relevance is evaluated to be low because there is a large difference in the subject and the shooting time is also discrete as before the start. As a result, the screen is divided into four, and four images are selected and displayed side by side with the closest shooting time. The visual effect may be quiet and simply switch images.

  Lastly, in “Section 4” of the candle service shown in FIG. 11, similar to “Section 2” of cake swords, a similar subject is photographed, and therefore, the degree of association is evaluated to be high. As for the screen division, for example, the order of the candle service and the visual effect is switched, and four images are displayed first, and then a10, b8, c7, and d7 are displayed on the entire screen. Use flashy visual effects.

  The point here is to make a difference in the visual effect between the “Before start, chatting” section and the “Cake sword, candle service” section, and make the latter stand out. Need not be. Also, the display time of the image should be longer. Also, since the shooting time is almost the same between images taken using the continuous shooting function provided in the portable camera, the degree of association is high, and these images can be displayed prominently or dividedly displayed. .

  Next, slide show generation when there is one photographer will be described with reference to FIG. FIG. 12 is a reference diagram showing a state of image shooting and reproduction on a multi-screen when there is one series. 10 and FIG. 11, the reception will be described as a model. In this figure, the reception is mainly held in two sections “before start” and “cake sword”.

  Then, in the first “Section 1”, two images a1 and a2 are photographed, and in “Section 2” the cake is inserted, five images a3, a4, a5, a6, and a7 are photographed. A large amount of images are taken in a short time in “Section 2” of the cake cutting blade, and discretely in “Section 1” before the start. In the figure, the subset is indicated by a dotted line.

  Next, the slide show generation will be described. First, in “Section 1” before the start, the state of the venue, participants, etc. are photographed, and there is a large difference in subjects, so it is evaluated that the relevance is low. As a result, the screen is divided into four, and four images are displayed there. However, since there are only two images here, some of them are displayed overlappingly. Further, it is not necessary to forcibly divide the screen into four, and the screen may be divided into two. The visual effect may be quiet and simply switch images. Similarly, BGM is automatically selected as gentle music.

  Next, when it comes to “Section 2” of the cake sword, the scene where the bride and groom enter the cake is continuously shot, so the similarity of the subject is high and the time interval is narrow, so the shooting time is similar The degree will be higher. Therefore, it is evaluated that the relevance between images taken here is high. Therefore, it is displayed in a flashy direction with a difference from the start. Here, first, a3, a4, a5, and a6 are sequentially displayed on the entire screen, and then four images of a4, a5, a6, and a7 are divided and displayed. Thereby, in the image reproducing device 100 according to the first embodiment, it is possible to produce an effect as if a continuous scene was photographed at multiple angles. A visual effect is also added with movement such as zooming and panning. Similarly, for BGM, intense types of music are automatically selected.

  As described above, in the image reproducing apparatus according to the present invention, even when one person shoots, the difference in visual effect between “before start” and “cake sword” is the same as when four persons shoot. It is possible to reproduce the latter so that it stands out.

  FIG. 13 is a reference diagram illustrating an example of image display during slide show playback displayed on the screen of the image playback device 100 according to the first embodiment.

  FIG. 13A shows a slide show while switching the sequential display of the above-mentioned wedding split screen (Mr. A photographed image, Mr. B photographed image, Mr. C photographed image, Mr. D photographed image), and the representative image. Is to play. By displaying in this way, it is possible to compare and display related images having a high degree of relevance, and to increase variations in screen display using a large amount of similar digital images.

  FIG. 13B is a goal scene division screen. In this way, the same goal scene (from the back of the net, from the left, the state of the audience seat, the photographer's viewpoint) is divided and displayed. In this case, it is possible to reproduce images from the other person's viewpoint as a slide show by using the degree of relevance based on time, place, or the like. Also, by acquiring images with high relevance from others through communication via a network, etc., and playing slideshows using these images, it is possible to play slideshows with message characteristics and higher entertainment. It can be carried out.

  FIG. 13C is a split screen showing the past and present of Naramachi (50 years ago, 10 years ago, a slightly different place, now). In this case, by using the relevance level based on the location, it is possible to reproduce videos of the same location at different times as a slide show.

  As described above, in the image reproduction device 100 according to the first embodiment, the relevance analysis unit 112 extracts the relevance between images, and the slide show generation unit 108 determines the visual effect based on this relevance. A slide show scenario is generated. Therefore, in the image reproducing apparatus according to the present invention, the images with low relevance are divided and displayed on one screen between images with high relevance, using a large number of photographed digital images. Therefore, it is possible to realize an interesting slide show reproduction while grasping the relationship between images.

  Also, by acquiring images with similar degrees of relevance from images held by others via a network and reproducing them as a slide show, it is possible to realize a slide show using images from the other person's viewpoint.

  In the image reproducing apparatus 100 according to the first embodiment, the case where the photographers are divided as a series is shown. However, it is not necessary, and if divided by the shooting year, the same time of each year is displayed on the divided screen. Images can be displayed.

  Also, the number of screen divisions need not be four, and may be selected depending on the number of image sequences and the size of the screen to be displayed. Further, the number of divisions and the number of series do not need to match.

  In the first embodiment, the operation after accumulating images has been described. However, a photographer may generate a slide show while sending images to each other. For example, by sending images taken by people sitting at different seats in a soccer venue, scenes from different angles in the goal scene can be displayed side by side. Also, the ceremony participation and browsing can be performed in real time in the above-described slide show reproduction of the reception. To supplement the transmission and reception of images, recent mobile phones have built-in high-performance digital cameras, and the mobile phone modules themselves are also inexpensive, so it is easy to provide equipment that combines shooting and communication functions. It is.

  Furthermore, the meta information 206 included in the image packet 110 need not be limited to that shown in FIG. The event name, information on any object other than a person, or the like may be used. If automatic extraction is difficult, it may be manually added. Based on these, for example, when the degree of relevance of collected items is high, they can be displayed in association with each other.

  In the first embodiment, a case where one screen is physically divided into a plurality of windows as a plurality of screens has been described. However, the plurality of screens may be physically different. In this case, the large screen display means that a part of an image is displayed on each of a plurality of display devices so that the entire image can be seen as one large image.

  In addition, the description has been made centering on an example in which images with weak relevance are displayed in a divided manner, and images with strong relevance are displayed on one screen, but the present invention is not limited to this, and in short, the degree of relevance is large or small. For example, in the case of continuous shooting that is judged to have a high degree of relevance, it is also effective to divide the screen and display it in a single screen at high speed. It is.

  Although only the case of still images as slide show playback has been described, the present application is not limited to still images, and moving images are a collection of still images. It can be considered to play.

(Embodiment 2)
Next, an image reproduction apparatus according to Embodiment 2 of the present invention will be described. The second embodiment is characterized in that an image set is selected by selecting the (L + 1) th image set based on the Lth image set. In addition, since the functional configuration of the image reproduction apparatus according to the second embodiment is the same as that of the first embodiment, detailed description thereof is omitted.

  FIG. 15 is a flowchart showing an operation procedure during slide show generation and playback of the slide show generation unit 108 of the image playback device according to Embodiment 2 of the present invention. Further, in the image reproduction apparatus according to the second embodiment, the degree of association between images has already been calculated, and an image similar to the degree of association of the currently slid image is reproduced next.

  First, the slide show generation unit 108 selects an image set from the image storage unit 103 (S1501), examines the degree of association, and gives a visual effect according to the result.

  Next, processing for selecting the next image set is performed based on the meta information 206 of the selected image (S1502). This selection process is normally performed once.

  For example, a case where there are two image sets is taken as an example. If the first images are a1 and a2, the meta information 206 is as follows.

a1: Time = 2004/06/20, Place = Tokyo, Person = Mr. A a2: Time = 2004/06/21, Place = Osaka, Person = B Next, the slide show generation unit combines this meta information 206 Or search for the next image with a partial change. Next, assuming that the selected images are b1 and b2, the meta information 206 is as follows.

b1: Time = 2004/06/22, place = Tokyo, person = A b2: time = 2004/06/20, place = Tokyo, person = B b1 changes time in meta information 206 of a1, b2 The meta information 206 is extracted from the a1 as the time and place and as the person is related to the image being reproduced at the present time from a2.

  Then, the slide show generation unit 108 checks the degree of association between images (S1503). Here, it is determined whether or not the degree of association is strong.

  Then, the slide show generation unit 108 gives a visual effect according to the style input from the user via the menu input unit 107, but when it is determined that the degree of association is weak (N in S1503). When it is determined that the degree of relevance is strong (Y in S1503), the visual effect for the image having a high degree of relevance is given (S1505).

  Next, the slide show generation unit 108 determines whether or not a length scenario determined for the above processing has been generated (S1507). If no scenario has been generated up to the determined length (S1507). N) Steps S1501 to S1506 are repeated, and when a scenario is generated up to a predetermined length (Y in S1507), the scenario is output to the scenario analysis unit 113 in the form of a scenario and decoded by the scenario analysis unit 113 Then, the slide show is reproduced by outputting to the display unit 109 (S1508).

  As described above, in the image reproduction device 100 according to the second embodiment, the meta information 206 is used to sequentially search for images having some relevance. In addition, since there are some elements to be changed, the next image reproduced during the slide show reproduction may jump to an unexpected and unexpected image. In other words, in order to select the next set of images according to the association between images, the display of an associative game image in which variation sometimes occurs is realized, and for the user, a slide show generation that feels interesting in both memory recall and unexpectedness is realized. It can be realized.

  The image reproduction apparatus according to the present invention can be used as an apparatus such as a digital camera, a mobile phone, a car navigation system, a DVD recorder, or a PDA having an image reproduction function, and the image reproduction method according to the present invention is digitally performed on a PC. Useful as photo browsing software.

Block configuration diagram of an image reproduction apparatus according to Embodiment 1 Data structure diagram of image packet including data extracted by image feature extraction unit in embodiment 1 Data configuration diagram of music packet including extraction information extracted by music feature extraction unit in Embodiment 1 7 is a flowchart showing an operation procedure of subject information extraction in the image feature extraction unit of the image reproduction device according to the first embodiment. 8 is a flowchart showing an operation procedure of physical quantity extraction in the image feature extraction unit of the image reproduction device according to the first embodiment. Explanatory drawing of the subset which is one set used as the relevance calculation object between each image in the image reproduction apparatus which concerns on this invention 7 is a flowchart showing an operation procedure of a relevance analysis unit included in the image reproduction device according to Embodiment 1. 6 is a flowchart showing an operation procedure of a slide show generation unit of the image reproduction apparatus according to Embodiment 1; Time chart showing the state of image shooting Screen image corresponding to the first “Section 1” and “Section 2” Screen image corresponding to “Section 3” and “Section 4” Reference diagram showing image capture and playback on multiple screens when there is a single sequence Reference diagram showing an image display example during slide show playback displayed on the screen of the image playback device according to the first embodiment Reference diagram showing a data configuration example of a scenario created based on the degree of association between images in the slide show generating unit of the image reproducing device of the invention 10 is a flowchart showing an operation procedure during slide show generation and playback of the slide show generation unit of the image playback apparatus according to the second embodiment. Configuration diagram of a conventional slideshow device

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 Image input part 102 Image feature extraction part 103 Image storage part 104 Music input part 105 Music feature extraction part 106 Music storage part 107 Menu input part 108 Slide show generation part 109 Display part 110 Image packet 111 Music packet 112 Relevance analysis part 113 Scenario Analysis unit 200 Image ID
201 Photographer ID
202 Camera information 203 Subject information 204 Physical feature information 205 Image data 206 Meta information 300 Music ID
301 Rhythm information 302 Music sensitivity information 303 Music data

Claims (23)

An image playback apparatus that performs playback using image data,
Image feature extraction means for extracting an image feature amount from the image data;
Relevance analysis means for analyzing relevance, which is information indicating similarity with other images, based on image information related to the image data and the image feature amount;
An image reproduction apparatus comprising: a slide show generation unit configured to reproduce a slide show using the image data based on the degree of association. The slide show generation means selects an image group that is a group of images to be reproduced based on the relevance level, and a visual that determines a visual effect at the time of reproduction of the image group based on the relevance level. An effect determination unit, and a scenario generation unit that generates a scenario that is a reproduction order of images included in the image group based on the degree of association;
The image playback device according to claim 1, wherein the selected image group is played back in the order of the scenario using the determined visual effect. The image reproduction apparatus according to claim 1, wherein the image information includes at least one information of a photographing time, a photographing place, and a photographer who photographed the image data. The image feature amount includes physical feature amount that is subject information and physical information related to a subject included in the image data,
The image reproduction apparatus according to claim 1, wherein the image feature extraction unit extracts the subject information and the physical feature amount. The physical feature amount includes at least one of luminance information regarding luminance of the image data, color information regarding color, and frequency information,
The image reproducing apparatus according to claim 4, wherein the subject information includes at least one of the number of subjects, the name of the subject, the location of the subject, and the size of the subject. The visual effect determination unit determines, as the visual effect, a divided display for dividing a screen into one or more small screens, a display time for the image data, and a method for moving the image data. Item 3. The image reproducing device according to Item 2. The image reproduction device according to claim 6, wherein the visual effect determination unit determines to perform the divided display during reproduction between the image data according to the degree of association. The visual effect determination unit determines the visual effect so that the display time is long, the image data is displayed largely or slowly when the image data having a high degree of association is reproduced. 6. The image reproducing device according to 6. The image playback device further includes:
A music feature extraction means for extracting music feature values from the music data;
The image reproduction device according to claim 2, wherein the visual effect determination unit determines the visual effect based on the relevance and the music feature amount. The relevance analysis means includes a weighting determination unit that determines the weighting of the photographing time, the photographing location, the photographer, the subject information, and the physical feature information,
The image reproduction apparatus according to claim 3, wherein the relevance analysis unit analyzes the relevance using a value after being weighted by the weight determination unit. The image selection unit selects a set of some images as a subset from the entire image group based on the degree of association,
The image reproduction device according to claim 2, wherein the relevance analysis unit analyzes relevance between images in the subset. The image reproduction device according to claim 11, wherein the image selection unit selects, as the subset, a set of image data whose shooting times are close as the relevance. The image selection unit selects an N-sequence image group from an M-sequence image group when M and N are integers of 1 or more,
The image reproduction device according to claim 2, wherein the visual effect determination unit determines to divide a screen into N windows and to display the N series of images in a divided manner. The image reproduction apparatus according to claim 13, wherein the M series is divided by a photographer. The visual effect determination unit, when the M series is one series, determines to divide the window according to a shooting location of the image data, a part of or a combination of shooting times. Image playback device. The image reproduction apparatus according to claim 13, wherein the M series is divided according to photographing equipment. The image selecting unit selects the L + 1th image group based on the image information of the Lth selected image group or at least one part of the relevance when L is an integer equal to or greater than 1. The image reproducing apparatus according to claim 2, wherein: The image selecting means selects, as the L + 1th image group, an image group having image information or a degree of association obtained by changing at least a part of the image information or the degree of association of the Lth selected image group. The image reproducing device according to claim 17. The image playback device further includes:
Scenario analysis means for analyzing the scenario;
The image reproducing apparatus according to claim 2, further comprising display means for reproducing and displaying the slide show generated by the slide show generating means. An image reproduction method for use in an image reproduction apparatus that performs reproduction using image data,
An image feature extraction step for extracting an image feature amount from the image data;
A relevance analysis step of analyzing relevance, which is information indicating similarity to other images, based on image information related to the image data and the image feature amount;
A slide show generation step of playing back a slide show using the image data based on the degree of association. The slide show generation step further includes an image selection step of selecting an image group that is a collection of images to be reproduced based on the relevance level, and a visual effect at the time of reproduction of the image group based on the relevance level. A visual effect determination step for determining, and a scenario generation step for generating a scenario that is a reproduction order of images included in the image group based on the relevance level,
21. The image reproducing method according to claim 20, wherein, in the slide show generating step, the selected image group is reproduced in the order of the scenario using the determined visual effect. A program used in an image reproduction method for reproducing using image data,
An image feature extraction step for extracting an image feature amount from the image data;
A relevance analysis step of analyzing relevance, which is information indicating similarity to other images, based on image information related to the image data and the image feature amount;
A slide show generation step of reproducing a slide show using the image data based on the degree of association. The slide show generation step further includes an image selection step of selecting an image group that is a collection of images to be reproduced based on the relevance level, and a visual effect at the time of reproduction of the image group based on the relevance level. A visual effect determination step for determining, and a scenario generation step for generating a scenario that is a reproduction order of images included in the image group based on the relevance level,
23. The program according to claim 22, wherein in the slide show generating step, the selected image group is reproduced in the order of the scenario using the determined visual effect.

Images