JP5499605B2 - Image playback device - Google Patents

Description

  The present invention relates to an image reproducing apparatus capable of reproducing still images, moving images, and sounds.

  A playback device that plays back a moving image including a still image and sound recorded on a recording medium, associating the still image with a predetermined shooting point of the moving image, and recording management information indicating the predetermined shooting point, There has been proposed a playback device that extracts a predetermined portion from a moving image based on this management information, plays back a sound corresponding to the predetermined portion of the moving image while playing a still image (for example, Patent Documents). 1).

JP 2005-117083 A

  By the way, in the playback apparatus described in Patent Document 1, the time when a still image is shot is set as a predetermined shooting time point, and a predetermined portion in a moving image recorded at the predetermined shooting time point is extracted. Then, when reproducing a still image, audio corresponding to a predetermined portion of the extracted moving image is also reproduced at the same time.

  However, for example, when a plurality of still images taken at different times (different days) are sequentially reproduced and displayed at predetermined time intervals, the time at which each still image was taken is set as a predetermined shooting time and recorded at the predetermined shooting time. When the sound corresponding to each moving image is sequentially played back, the sound is interrupted and sometimes lacks smoothness. In addition, since a part of the moving image recorded at a time other than the predetermined shooting time cannot be extracted as a portion related to the still image, the sound corresponding to the moving image recorded at a time other than the predetermined shooting time is simultaneously reproduced with the reproduction of the still image. Could not play.

  An object of the present invention is to provide an image reproduction apparatus capable of extracting an optimum reproduction sound to be reproduced together with the still image from the sounds included in the moving image when reproducing and displaying the still image. is there.

Image reproducing apparatus of the present invention, moving images recorded on the recording medium body, a reproducing unit sound is recorded together with the moving image, and a still image associated with the moving image reproducing, respectively, said recording medium body from among the still images recorded on the a selection unit for selecting a more still image to be reproduced to the reproduction unit, an analysis unit for performing image analysis of the selected the still image by the selection unit, the analyzing unit based on the analysis result by the out of the recording medium body the sound recorded in an extraction unit for extracting a reproduced sound more reproduction to the reproduction unit, the still image and said selected by the selection unit of the reproduced sound extracted by the extracting unit, a reproduction control section for controlling the reproduction by the reproducing unit, a reference image associated with each theme, and registration unit for registering a reference sound associated with the reference image Be equipped The extraction unit performs the reproduction based on at least one of the reference image and the reference sound extracted based on the analysis result by the analysis unit from the reference image and the reference sound registered in the registration unit. It is characterized by extracting sound .

  According to the image reproduction apparatus of the present invention, when reproducing and displaying a still image, it is possible to extract the optimum reproduction sound to be reproduced together with the still image from the sounds included in the moving image.

It is a block diagram which shows the system configuration | structure of the image reproduction apparatus which concerns on embodiment. FIG. 10 is a flowchart for explaining processing when a playback sound to be played back by an audio playback unit is extracted at the same time that a still image is played back and displayed on an image display unit in the image playback device according to the embodiment. It is a figure which shows the example of a registration image. It is a figure which shows the example of the selected still image. It is a time chart which shows the relationship between a still image and the related moving image. It is a time chart which shows the relationship between a still image and the related moving image.

  Hereinafter, an image reproduction apparatus according to an embodiment of the present invention will be described with reference to the drawings. The image reproducing apparatus according to the present embodiment is a liquid crystal panel for reproducing and displaying still images or moving images (including continuous still images recorded together with sound) recorded on a recording medium or the like. An image display unit and a sound reproduction unit such as a speaker for reproducing sound recorded on a recording medium or the like are provided. As an example, a personal computer (hereinafter referred to as a PC) such as a notebook personal computer or a desktop personal computer. Is mentioned.

  FIG. 1 is a block diagram showing a system configuration of an image reproduction apparatus 2 according to the embodiment. As shown in FIG. 1, the image reproduction device 2 includes a control unit 4, a recording medium 6, an image display unit 8, an audio reproduction unit 10, an operation unit 12, an analysis unit 14, and a registration unit 16. . The control unit 4 is configured by a CPU or the like, and comprehensively controls each unit of the image reproduction device 2. In this embodiment, when a still image is reproduced and displayed on the image display unit 8, a reproduction sound to be reproduced by the audio reproduction unit 10 is extracted, and reproduction of the still image and the reproduction sound is controlled.

  The recording medium 6 is a portable recording medium that is detachably attached to a card slot (not shown) provided in the image reproducing apparatus 2, and for example, a CF card, an SD card, smart media, or the like is used. . The recording medium 6 records still images, moving images including sound, and information relating to shooting such as shooting dates and times added to the still images and moving images. In addition, information indicating the association with the moving image (for example, the file name of the moving image) is added to the still image captured while the moving image is recorded. Information indicating the association with the moving image is also recorded together with the still image. Other moving images can also be associated with still images by, for example, user operation, and the reproduced sound reproduced in the audio reproducing unit 10 when the still image is reproduced and displayed on the image display unit 8 Extracted from the sound contained in the image.

  The image display unit 8 reproduces and displays still images and moving images recorded in the recording medium 6, registered images registered in the registration unit 16 described later, and the like. The audio reproduction unit 10 reproduces the audio included in the moving image recorded on the recording medium 6, the registered sound registered in the registration unit 16, and the like. The operation unit 12 includes an operation member operated by a user, for example, a keyboard and a pointing device. The analysis unit 14 analyzes a still image to be reproduced and displayed on the image display unit 8 and a moving image associated with the still image and recorded on the recording medium 6.

  In the registration unit 16, a registration image and a registration sound that are used as a reference when the reproduction sound reproduced by the audio reproduction unit 10 is simultaneously reproduced and displayed on the image display unit 8 are registered in association with each theme. Has been. The registered image is registered in advance. For example, a frame in a still image or a moving image already recorded by a camera or the like can be registered as a registered image by a user operation. The registered sound is registered in advance, for example, a sound already recorded with a moving image by a camera or the like, a sound recorded by a recordable device, or keywords related to various themes (scenes) (for example, a theme “festival” The keyword “Washoi” and the keyword “Cheers” for the theme “Party”, etc.) can be registered as registered sounds by user operation. In addition, a still image for registration as a registered image can be captured by a camera or the like and registered as a registered image, and a moving image and sound for registration as a registered image and registered sound can be simultaneously recorded and registered by a camera or the like. An image and a registered sound can be registered in association with each other. The theme can be created by the user via the operation unit 12, and one registered image or one registered sound can be associated with a plurality of themes.

  Next, referring to the flowchart shown in FIG. 2, when reproducing the still image on the image display unit 8 and extracting the reproduction sound to be reproduced by the audio reproduction unit 10 in the image reproduction apparatus 2 according to this embodiment. Processing will be described.

  In this embodiment, as shown in FIG. 3, the theme “Festival” includes a child image (image including a face) A, a shrine image B related to a festival, a yukata image C, and a fan image D. It is assumed that the registered image is registered in advance in the registration unit 16. Further, it is assumed that keywords reminiscent of a festival such as a child's voice of image A, a sound of a festival lion, and a washer are registered in advance in the registration unit 16 as a registered sound of the theme “festival”.

  First, when a still image to be reproduced and displayed by the image display unit 8 is selected from among still images recorded on the recording medium 6 through the operation unit 12, the control unit 4 selects the selected still image. Is acquired from the recording medium 6 (step S10). In this embodiment, it is assumed that a still image E of a child wearing a yukata and a fan is selected with a shrine as shown in FIG.

  Next, the control unit 4 causes the analysis unit 14 to analyze the still image E acquired in step S10 (step S11). That is, face recognition, face identification, object recognition, scene (theme) recognition, and the like of the still image E are performed. From the still image E, there is one person (face recognition), the person is a “child” (face identification), the “fan” (object recognition) in the child's hand, and the “shinto” behind the child. ”(Object recognition), children are dressed in“ yukata ”(thing recognition), and it is analyzed that“ festivities ”(scene recognition) from“ fan ”,“ shinto ”and“ yukata ” .

  Next, the control unit 4 extracts a reference image from the registered images registered in the registration unit 16 based on the analysis result analyzed by the analysis unit 14 in step S11 (step S12). That is, among all the registered images registered in the registration unit 16, images related to “children”, “fan”, “shrine”, “yukata”, “festival” analyzed from the still image E (in this embodiment) , Images A to D) are extracted as reference images.

Next, the control unit 4 selects a frame (reference image A to D) related to the reference image A to D extracted in step S12 from all the frames constituting the predetermined moving image associated with the still image E. At least one frame region including a similar frame or a frame including the reference images A to D is extracted (step S13). The predetermined moving image was recorded from a moving image recorded in a time zone including a time when the still image was captured, a moving image designated and operated by the user, and a predetermined time before the still image capturing time. A moving image and a moving image recorded from a still image capturing time to a preset time. The time before and after the preset time can be changed and set independently. For example, the time can be set in a day unit such as the previous day and two days later, or in a time unit such as several hours before and after several hours. In this embodiment, as shown in FIG. 5, the moving image M (recorded from time t ₁ to t ₂ ) recorded by the same camera at the time t ₃ of capturing the still image E is a predetermined moving image. still image and those associated with the E, the frame region F ₁ to F ₃ including the frame f ₁ ~f ₃ is assumed to have been extracted as a frame associated with the reference image A~D as. In this embodiment, the frame regions F ₁ and F ₃ other than the frame region F ₂ are always extracted regardless of whether or not the frame region F ₂ including the frame f ₂ corresponding to the still image E is extracted. It is assumed that it is set to

Next, the control unit 4 extracts sounds corresponding to the frame regions F _{1 to} F ₃ extracted in step S13 as reproduction sounds (step S14). That is, the sound recorded when the frame regions F _{1 to} F ₃ extracted from all the frames constituting the predetermined moving image are recorded is extracted.

  On the other hand, the control unit 4 also extracts a reference sound from the registered sounds registered in the registration unit 16 based on the analysis result analyzed by the analysis unit 14 in step S11 (step S15). That is, among all the registered sounds registered in the registration unit 16, sounds related to “kid”, “fan”, “shrine”, “yukata”, “festival” analyzed from the still image E (in this embodiment, , Children's voice, ritual sound, keywords such as washes) are extracted as reference sounds.

  Next, the control unit 4 selects a sound related to the reference sound extracted in step S15 from the sounds included in the predetermined moving image associated with the still image E (a sound similar to the reference sound or a reference sound). Are extracted as reproduced sounds (step S16). At this time, the same reproduced sound as the reproduced sound extracted in step S14 may be extracted, or a different reproduced sound may be extracted. For human voices, the voiceprint of the reference sound, for environmental sounds and music, the reference sound features (frequency change over time, frequency periodicity, frequency continuity, time length of frequency level, frequency direction) And the temporal direction), the similarity is determined, and the reproduced sound similar to the reference sound is extracted. In this embodiment, the control unit 4 is set such that sounds other than the sound recorded at the same time as the still image capturing time are always extracted as the reproduced sound.

  And the control part 4 reproduces | regenerates and displays the still image E on the image display part 8, and reproduces | regenerates the reproduction sound extracted in step S14 and S16 in the audio | voice reproduction | regeneration part 10 (step S17). When a plurality of reproduced sounds are extracted, the reproduced sounds are reproduced in the order in which the recorded times are earlier, the recorded times are closer to the imaging time of the still image E, or randomly.

  When one still image E is selected by the user, the reproduction sound is repeatedly reproduced in the audio reproduction unit 10 while the still image E is displayed on the image display unit 8. In addition, when a plurality of still images are selected by the user and the playback display time of the still image is not set, the still image E is displayed as the image display unit 8 while the playback sound is played back by the audio playback unit 10. To display. That is, when the display of the still image E is started, the playback of the playback sound is also started. When the playback of the playback sound is ended, the display of the still image E is ended and the display of the next still image is started. In this case, for each selected still image, the reproduction sound is extracted from the sound in the associated moving image and reproduced. Therefore, the playback and display times of still images are different, and the playback and display time of still images is longer when there are many extracted playback sounds, and the playback and display time of still images is shorter when there are few extracted playback sounds. Become.

  In addition, when a plurality of still images are selected by the user and the reproduction display time of each still image is set to a predetermined time, the recorded time is earlier during the predetermined time for displaying the still image E. In order, the recorded sound is played back in order of recording time of the still image E, or randomly. Then, after a predetermined time, the display of the still image E and the playback of the playback sound are terminated, and the display of the next still image is started. Also in this case, for each selected still image, the reproduction sound is extracted from the sound in the associated moving image and reproduced. Therefore, when the playback time of the extracted playback sound is shorter than the predetermined time, the playback sound is repeatedly played back. When the playback sound playback time is longer than the predetermined time, the playback sound is shortened to the predetermined time and played back. To do. Note that when a part of the playback sound is played back, the playback sound other than the playback sound recorded at the same time as the still image E is captured is set to be played back.

  In addition, when a plurality of still images are selected by the user, it is also possible to extract common reproduction sounds for a plurality of selected still images, instead of extracting reproduction sounds for each of the still images. In this case, a common theme, person or thing is found in a plurality of still images from the analysis result by the analysis unit 14, and a reference image and reference sound common to the plurality of still images are obtained based on the common theme, person or thing. Extract. Then, a sound related to the reference sound is extracted as a common reproduction sound for the plurality of still images from the sounds included in the moving image associated with at least one of the plurality of still images. The common reproduction sound is reproduced while a plurality of still images are sequentially reproduced, and is not switched when the still images are switched.

  Although the theme “festival” has been described as an example, as shown in FIG. 3, for example, the registration unit 16 includes themes “car race”, “air show”, “family trip”, etc. other than the theme “festival”. And various registered images (images A, G to N, etc.) and registered sounds are registered. The child image A is registered in the registration unit 16 as registered images of a plurality of themes “festival”, “automobile race”, and “air show”. In the following, a process for extracting reproduced sound will be described by taking the theme “car race”, “air show”, and “family trip” other than the theme “festival” as examples.

  First, the theme “car racing” will be described. The theme “car race” includes a child image A, a racing car image G, a racing motorcycle image H, a checker flag image I, etc. as registered images. Sounds and the like are registered in advance in the registration unit 16 as registered sounds. When the user selects a still image (a still image of a child with a racing car in the back), the control unit 4 causes the analysis unit 14 to analyze the still image. That is, from the selected still image, there is one person (face recognition), the person is a “kid” (face identification), and there is a “racing car” (object recognition) behind the child, It is analyzed from “racing car” to “car racing” (scene recognition). Next, the control unit 4 extracts, as reference images, images related to the analyzed “children”, “racing cars”, and “automobile races” from all the registered images registered in the registration unit 16. Next, the control unit 4 extracts a frame region related to the reference image from the associated predetermined moving images, and extracts a sound corresponding to the extracted frame region as a reproduction sound. On the other hand, the control unit 4 relates to the analyzed “kid”, “racing car”, and “automobile race” from all the registered sounds registered in the registration unit 16 based on the analysis result by the analysis unit 14. Sound (in this embodiment, a child's voice, exhaust noise sound, etc.) is extracted as a reference sound. Then, a sound related to the reference sound is extracted as a playback sound from the sounds included in the associated predetermined moving image.

  Next, the theme “Air Show” will be described. In the theme “air show”, a child image A, an airplane image J, and the like are registered in the registration unit 16 in advance, and a child voice, a jet engine sound, and the like in the image A are registered as registered sounds. . First, when a still image (a still image of a child with an airplane back) is selected by the user, the control unit 4 causes the analysis unit 14 to analyze the still image. That is, from the selected still image, there is one person (face recognition), the person is “child” (face identification), and there is “airplane” (object recognition) behind the child, It is analyzed from “airplane” to “air show” (scene recognition). Next, the control unit 4 extracts, as reference images, images related to the analyzed “children”, “airplanes”, and “airshows” from all the registered images registered in the registration unit 16. Next, the control unit 4 extracts a frame region related to the reference image from the associated predetermined moving images, and extracts a sound corresponding to the extracted frame region as a reproduction sound. On the other hand, based on the analysis result by the analysis unit 14, the control unit 4 selects the sound related to the analyzed “kid”, “plane”, and “air show” from all the registered sounds registered in the registration unit 16. (In this embodiment, a child's voice, a jet engine sound, etc.) are extracted as a reference sound. Then, a sound related to the reference sound is extracted as a playback sound from the sounds included in the associated predetermined moving image.

  When analyzing the image, analyze the color, shape, number, etc. of the racing car or airplane, and only if the racing car or airplane in the selected still image is the same as the racing car or airplane in the registered image The registered image can be extracted as a reference image.

  Next, the theme “family trip” will be described. For the theme “family trip”, the family image K, the theme park character image L, the theme park attraction image N, etc. are registered images, and the family voice, parade music, etc. are registered in the registration unit 16 in advance. It is registered. First, when a still image (a family still image surrounding a theme park character) is selected by the user, the control unit 4 causes the analysis unit 14 to analyze the still image. That is, there are a plurality of persons from the selected still image (face recognition), the plurality of persons are the user's “family” (face identification), and “theme park character” (object recognition) in the center. It is analyzed that “family trip” (scene recognition) from “family” and “theme park character”. Next, the control unit 4 extracts, as reference images, images related to the analyzed “family”, “theme park character”, and “family trip” from all the registered images registered in the registration unit 16. . Next, the control unit 4 extracts a frame region related to the reference image from the associated predetermined moving images, and extracts a sound corresponding to the extracted frame region as a reproduction sound. On the other hand, the control unit 4 selects the analyzed “family”, “theme park character”, and “family trip” from all the registered sounds registered in the registration unit 16 based on the analysis result by the analysis unit 14. Related sounds (in this embodiment, family voice, parade music, etc.) are extracted as reference sounds. Then, a sound related to the reference sound is extracted as a playback sound from the sounds included in the associated predetermined moving image.

  According to the image reproduction device 2 according to this embodiment, when reproducing and displaying a still image, the optimum reproduction sound to be reproduced together with the still image is automatically extracted and reproduced from the sounds included in the moving image. can do. In addition, since the optimum reproduction sound can be reproduced when the still image is reproduced and displayed, the still image can be viewed with a sense of presence similar to that of moving image reproduction.

  In this embodiment, the reproduction sound is extracted based on the reference image and the reference sound. However, the reproduction sound may be extracted from only the reference image without extracting the reference sound, and the reference image is extracted. The reproduced sound may be extracted from only the reference sound without doing so.

  In this embodiment, the frame region of the moving image related to the reference image is extracted, and the sound corresponding to the frame region is extracted as the reproduction sound. However, one frame of the moving image related to the reference image is extracted. And a sound region including a sound corresponding to the extracted frame can be extracted as a reproduced sound.

In the above embodiment, in step S13 in FIG. 2, the frame region related to the still image E (reference images A to D) from the moving image M recorded in the time zone including the time when the still image E was captured. Although F _{1 to} F ₃ are extracted, the present invention is not limited to this, and as described above, a moving image recorded outside the recording time of the still image E (as shown in FIG. 6, before and after the moving image M 2 Related frame regions F ₂ and F ₃ may be extracted from the moving images M 1 and M 3).

  DESCRIPTION OF SYMBOLS 2 ... Image display apparatus, 4 ... Control part, 6 ... Recording medium, 8 ... Image display part, 10 ... Sound reproduction part, 12 ... Operation part, 14 ... Analysis part, 16 ... Registration part.

Claims (8)

A playback unit for playing back a moving image recorded on a recording medium, a sound recorded together with the moving image, and a still image associated with the moving image;
A selection unit for selecting a still image to be reproduced by the reproduction unit from the still images recorded on the recording medium;
An analysis unit that performs image analysis of the still image selected by the selection unit;
Based on the analysis result by the analysis unit, an extraction unit that extracts a reproduction sound to be reproduced by the reproduction unit from the sound recorded on the recording medium;
A reproduction control unit for controlling reproduction by the reproduction unit of the still image selected by the selection unit and the reproduction sound extracted by the extraction unit;
A reference image associated with each theme, and a registration unit for registering a reference sound associated with the reference image,
The extraction unit is configured to generate the reproduction sound based on at least one of a reference image and a reference sound extracted based on an analysis result by the analysis unit from the reference image and the reference sound registered in the registration unit. An image reproducing apparatus characterized by extracting the image. When the reference image is extracted based on the analysis result, the playback sound corresponding to the frame region of the moving image similar to the reference image is extracted;
The analysis in the case where the reference sound is extracted based on the result, the image reproducing apparatus according to claim 1, wherein the extracting the reproduced sound which is similar to the reference sound. The selection unit selects a plurality of the still images,
The extraction unit extracts at least one reproduction sound based on the analysis result of the plurality of still images by the analysis unit;
The reproduction control unit, while playing the plurality of still images, the image reproducing apparatus according to claim 1 or claim 2, characterized in that for reproducing the reproduced sound. The reproduction control unit, while reproducing the reproduced sound extracted by the extraction unit, any of claims 1 to 3, characterized in that for reproducing the still image selected by the selection unit The image reproducing device according to claim 1. The reproduction control unit reproduces the still image selected by the selection unit for a predetermined time,
When the reproduction sound extracted by the extraction unit is shorter than the predetermined time, the reproduction sound is reproduced repeatedly while the still image is being reproduced,
When the reproduction sound extracted by the extraction unit is longer than the predetermined time, the reproduction sound is shortened to the predetermined time and reproduced based on a predetermined condition while the still image is being reproduced. The image reproducing device according to any one of claims 1 to 4 . The still image, the image reproducing apparatus according to any one of claims 1 to 5, characterized in that having been captured while the moving image is recorded. The image reproducing apparatus according to claim 6 , wherein the extraction unit extracts a frame other than a frame corresponding to the still image of the moving image, and extracts the reproduction sound corresponding to the extracted frame. . The still image, the image reproducing apparatus according to any one of claims 1 to 5, wherein the moving picture is one that was captured at the time other than the time recorded.

Images