JP5293102B2 - Image reproducing apparatus and program - Google Patents

Description

  The present invention relates to an image reproducing apparatus capable of reproducing image data and a program executable by the image reproducing apparatus.

  There are various image playback apparatuses that play back images recorded on a recording medium such as a digital camera or a computer on a display unit as a slide show or a movie (movie), and also play audio from a speaker or the like (for example, patents) Reference 1). According to Japanese Patent Application Laid-Open No. 2004-228620, images are played back as a slideshow along with BGM (Back Ground Music) based on the shooting date.

For example, Patent Document 2 discloses a technique for recognizing sound data of an image with sound data and using it for image retrieval.
JP 2006-164229 A JP 2005-346259 A

  By the way, when reproducing a large number of image data in order such as a slide show, various reproduction orders such as a recording date order, a file name order, and a random reproduction can be considered. However, when playing back in date order or file name order, the order of playback is always constant and is less interesting. On the other hand, when playing back at random, the playback order is not predicted, and the user's desire about the playback order is not reflected.

  On the other hand, if it is played back in the order according to the criteria specified by the user, it is thought that the interest will increase. In particular, when the voice of a person who is the subject is added to the image data as voice data such as a voice memo, the voice of the person is analyzed, the subject is classified into gender and other characteristics, and reflected in the playback order. If this is the case, it can be considered that interest is enhanced.

  However, the technique described in Patent Document 1 is to add sound data to image data based on the shooting date, and is not a technique for processing image data associated with sound data.

  In the technique described in Patent Document 2, since voice data associated with image data is recognized by voice recognition, the subject can be classified in detail, and may be reflected in the reproduction order. . However, the load of voice recognition is high, and the processing capability of the image playback device is consumed greatly.

  In view of such a problem, the present invention reproduces a large number of image data by classifying them according to the characteristics of audio data caused by the subject or arranging them in order of remarkable characteristics by simple processing. An object of the present invention is to provide an image reproducing apparatus and a program that can reflect image data in an order and reproduce image data in an interesting reproduction order.

In order to solve the above problems, a typical configuration of an image reproduction device according to the present invention includes a frequency analysis unit that analyzes the frequency of audio data associated with each of one or more image data, and an analyzed frequency. Accordingly, a voice determination unit that determines characteristic information of voice data including the gender of the speaker, an association unit that associates the characteristic information with each of the one or more image data, and one or more of the image data, respectively. An image selection unit that selects reproduction target image data to be reproduced based on the associated characteristic information, and a reproduction control unit that reproduces the selected reproduction target image data on the display unit in ascending or descending order of the analyzed frequency. And.

According to the above configuration, the image data subjected to gender determination is further given a characteristic of frequency (voice color), and the image data can be reproduced in the order of high voice color (descending order of frequency) or low order (ascending order of frequency). is there. This makes it possible to play an interesting slide show such as preferentially reproducing image data in which a bright and bright person is captured as a subject.

  The reproduction control unit may sequentially switch and reproduce the selected reproduction target image data. According to such a configuration, so-called slide show reproduction is executed.

  According to the above configuration, the gender-determined image data is further given a characteristic of frequency (voice color), and the image data can be reproduced in the order of high voice color (descending order of frequency) or low order (frequency aiming). is there. This makes it possible to play an interesting slide show such as preferentially reproducing image data in which a bright and bright person is captured as a subject.

  The associating unit may add audio data characteristic information to each of one or more pieces of image data. Thereby, the characteristic information of the sound data is associated with the image data by being stored integrally with the image data.

  The associating unit may create associating data indicating an associating state between the audio data characteristic information and each of the one or more pieces of image data, and store the associating data as a file.

  According to the configuration described above, it is possible to determine which image data the audio data characteristic information is associated with by referring to the association data.

In order to solve the above-described problem, a typical configuration of a program according to the present invention includes a frequency analysis unit that analyzes the frequency of audio data associated with each of one or more image data, and a frequency that is analyzed. A voice determination unit for determining characteristic information of the voice data including the gender of the speaker, an association unit for associating the characteristic information with each of the one or more image data, and one or more of the image data. An image selection unit that selects reproduction target image data to be reproduced based on the characteristic information, and a reproduction control unit that reproduces the selected reproduction target image data on the display unit in ascending or descending order of the analyzed frequency. The playback device is made to function.

  The components corresponding to the technical idea of the image reproduction apparatus described above and the description thereof can be applied to the program.

  According to the present invention, a large number of image data can be classified according to the characteristics of audio data caused by the subject by simple processing, or arranged in order of remarkable characteristics, and reflected in the reproduction order. Image data can be reproduced in an interesting reproduction order.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. The dimensions, materials, and other specific numerical values shown in the embodiments are merely examples for facilitating understanding of the invention, and do not limit the present invention unless otherwise specified. In the present specification and drawings, elements having substantially the same function and configuration are denoted by the same reference numerals, and redundant description is omitted, and elements not directly related to the present invention are not illustrated. To do. Further, signals and currents are expressed by the codes of the lines through which they pass.

(Digital camera)
FIG. 1 is a block diagram of a digital camera which is an embodiment of an image playback apparatus according to the present invention. FIG. 2 is a diagram of image data played back by the digital camera of FIG. 1, and audio data and characteristic information associated with the image data. FIG.

(Frequency analysis part)
The digital camera 100 includes a frequency analysis unit 106 that analyzes the frequencies of audio data 104A to 104D associated with each of the one or more image data 102A to 102D.

  2A exemplifies image data 102A, audio data 104A, and characteristic information 108A that are associated with each other and recorded on the recording medium 370 as representatives of the image data 102A to 102D. FIG. 2B is a reference table showing the association of the image data 102A to 102D, the audio data 104A to 104D, and the characteristic information 108A, and is recorded on the recording medium 370 in FIG. This is a file generated by the association unit 112 described later, and may be a text file.

  The image data 102A to 102D are still images in the present embodiment, but may be moving images. In the present embodiment, only four pieces of image data 102A to 102D are shown for convenience of explanation as shown in FIG. 2B, but the number is not limited. The audio data 104A to 104D may be, for example, voice memos acquired by the voice memo function in which the digital camera 100 records audio in association with still images. The association between the still image and the sound in the voice memo function may be performed by recording and recording the sound at the time of shooting the still image, or when the still image recorded on the recording medium 370 is being reproduced. You may carry out by recording and recording an audio | voice.

  The file format of the image data 102A to 102D is JPEG (Joint Photographic Experts Group) in this embodiment, but in addition, TIFF (Tagged Image File Format), GIF (Graphics Interchange Format), PNG (Portable Network Graphics), bit It may be an image file of any format such as a map (Bitmap). Moreover, it is good also as Exif (Exchangeable Image File Format) which added various information at the time of imaging | photography to the compression data of JPEG format.

  The file format of the audio data 104A to 104D is WAV (WAVeform Audio Format, which is a kind of RIFF (Resource Interchange File Format) for storing audio and images together) in this embodiment, but is AVI (Audio Video Interleave. RIFF). 1 type), WMA (Windows (registered trademark) Media Audio), MP3 (MPEG Audio Layer-3), etc.

(Voice judgment part)
The digital camera 100 includes a voice determination unit 110 that determines the characteristic information 108D to 108D of the voice data 104A to 104D including the gender (male or female) of the speaker according to the analyzed frequency.

  For example, if the frequency analyzed by the frequency analysis unit 106 is 100 to 150 Hz, the voice determination unit 110 refers to a Japanese average value as the fundamental frequency of vocal cord vibration. If it is in between, it may be determined that it is a child. Such a criterion may be freely changed.

  Thus, if the frequency is used, gender can be determined by a simple process. Therefore, the characteristic that can be a reference of the image data reproduction order, such as gender, is applied to the voice data 104A without applying a load as much as voice recognition. ~ 104D characteristic information 108D to 108D.

(Volume detector)
The digital camera 100 further includes a volume detector 111 that detects the volume of the audio data 104A to 104D associated with each of the one or more image data 102A to 102D and adds the detected volume to the characteristic information 108D to 108D.

(Association section)
The digital camera 100 includes an association unit 112 that associates the characteristic information 108D to 108D with each of the one or more pieces of image data 102A to 102D.

  The associating unit 112 may add the characteristic information 108D to 108D of the audio data 104A to 104D to each of the one or more image data 102A to 102D. In this case, for example, the characteristic information 108A in FIG. 2A may be a part of the image data 102A, for example, header information. Accordingly, the characteristic information 108D to 108D of the audio data 104A to 104D is associated with the image data by being stored together with the image data 102A to 102D.

  The associating unit 112 creates association data indicating the association state between the characteristic information 108D to 108D of the audio data 104A to 104D and each of the one or more image data 102A to 102D, and stores the association data as a file. Good.

  FIG. 2B is a reference table 114 showing an example of the association data. When the reference table 114 is created, the characteristic information 108D to 108D is stored as an independent file different from the image data 102A to 102D.

  According to the above configuration, the characteristic information 108 </ b> A to 108 </ b> D of the audio data 104 </ b> A to 104 </ b> D can be identified with which of the image data 102 </ b> A to 102 </ b> D by referring to the reference table 114.

  In the present embodiment, the associating unit 112 associates the characteristic information 108A to 108D with the image data 102A to 102D, but may associate with the audio data 104A to 104D.

(Image selection part)
The digital camera 100 selects (extracts) reproduction target image data to be reproduced based on the characteristic information 108A to 108D associated with each of the one or more pieces of image data 102A to 102D. Is provided.

  The method of image selection by the image selection unit 116 can be freely set by the user through a predetermined interface. For example, of the image data 102A to 102D, only a male image (image data 102A, 102B) can be selected and reproduced, or only a female image (image data 102C, 102D) can be selected and reproduced. It is also possible to select and reproduce both male and female images (image data 102A to 102D).

(Reproduction control unit)
The digital camera 100 includes a reproduction control unit 118 that reproduces the selected reproduction target image data on a display unit (LCD (Liquid Crystal Display) 380). The reproduction control unit 118 also reproduces the audio data associated with the reproduction target image data from the speaker 320 at the same time.

(Slideshow playback)
The reproduction control unit 118 sequentially switches and reproduces the selected reproduction target image data. That is, so-called slide show reproduction is executed. The playback time of each playback target image may be the playback time of the audio data associated therewith. According to such a method, there is no need to set the playback time of the playback target image that is a still image. However, the playback time may be set to a fixed time regardless of the playback time of the audio data, or may be changed according to an arbitrary rule. For example, an image of a female subject may have a longer reproduction time than a male image.

(Playback order according to gender)
According to the above configuration, it is possible to classify a large number of image data according to the characteristics (gender) of the audio data caused by the subject by simple processing, and to reproduce the image data in an interesting reproduction order. is there.

  For example, consider a case where a congratulatory message from a person is recorded by a voice memo function while photographing a large number of persons at a wedding. It is assumed that audio data 104A to 104D are obtained together with the image data 102A to 102D, and further that characteristic information 108A to 108D is obtained. Then, for example, the image data 102C and 102D in which the subject is determined to be female can be reproduced first, and the image data 102A and 102B determined to be male can be reproduced later.

(Playback order according to volume)
The reproduction control unit 118 may reproduce the selected reproduction target image data in ascending or descending order of the volume detected by the volume detection unit 111.

  According to the above configuration, the gender-determined image data 102A to 102D is further given a volume characteristic, and the image data 102A to 102D can be reproduced in ascending or descending volume order. This makes it possible to play an interesting slide show such as preferentially reproducing image data in which a bright person and a bright person appear as subjects.

  For example, the above-described reproduction order according to gender may be combined with the reproduction order according to the volume. If the reproduction control unit 118 is set to the reproduction order of female → male voice, and the volume is ascending order, the image data Playback is performed in the order of 102C, 102D, 102A, and 102B.

  In addition, the image data is classified according to a predetermined threshold of volume, and is classified into groups such as “voice is loud”, “normal”, and “voice is low”, and image data belonging to any group is displayed in the image selection unit 116. You may select and play.

(Playback according to voice)
The reproduction control unit 118 may reproduce the selected reproduction target image data in ascending or descending order of the analyzed frequencies.

  According to the above configuration, the gender-determined image data 102 </ b> A to 102 </ b> D is further given a characteristic of frequency (voice color), and the image data 102 </ b> A is ordered in descending order of voice color (descending order of frequency) or decreasing order (rising order of frequency). -102D can be reproduced. This makes it possible to play an interesting slide show such as preferentially reproducing image data in which a bright and bright person is captured as a subject.

  For example, the playback order according to gender may be combined with the playback order according to the voice, and the playback control unit 118 may be configured to change the order of female → male voice and descending frequency (priority is given to a person with a bright voice sound of high frequency). In the order of image data 102D, 102C, 102A, and 102B.

  In addition, the image data is classified according to a predetermined threshold of frequency, and is classified into groups such as “voice sound is bright”, “normal”, and “voice sound is dark”, and the image data belonging to any group is displayed in the image selection unit 116. You may select and play.

(Classification and ordering by other characteristics)
In addition, the image data may be classified or ordered according to other characteristics of the audio data 104A to 104D, or the image data may be classified or ordered according to the characteristics of the image data itself. For example, image recognition is performed, and image data determined to be a smile can be classified into a group with a bright expression.

(Digital camera configuration)
3 is a diagram showing an example of the appearance of the digital camera of FIG. 1, FIG. 3 (a) is a diagram seen from the front, and FIG. 3 (b) is a diagram seen from the back. Other configurations of the digital camera will be described below with reference to FIGS. 3 and 1. The imaging unit 220 may be a CCD (Charge Coupled Device) and is an imaging unit that captures an object scene and generates an electronic image signal, and has, for example, 1600 × 1200 pixels. The imaging unit 220 converts the light image of the subject formed by the imaging lens 210 into image signals of R (red), G (green), and B (blue) color components for each pixel (pixels received by each pixel). A signal comprising a signal sequence of signals) and output.

  An image signal (analog signal) obtained from the imaging unit 220 is given to the analog signal processing circuit 230. The analog signal processing circuit 230 is a circuit that performs predetermined analog signal processing on the image signal. The analog signal processing circuit 230 includes at least a correlated double sampling circuit (Correlated Double Sampling: CDS, not shown) and an auto gain control (Auto Gain Controlled: AGC, not shown) circuit. Noise reduction processing of the image signal is performed by the correlated double sampling circuit, and the level of the image signal is adjusted by adjusting the gain by the auto gain control circuit.

The A / D converter 240 converts each pixel signal of the image signal into, for example, a 12-bit digital signal. The converted digital signal is given to a central processing unit (CPU) 250 and temporarily stored in RAM (Random Access Memory) as image data.
260. The image data stored in the RAM 260 is subjected to color correction processing or the like by the image processing unit 280 and then subjected to compression processing or the like by the compression / decompression unit 290.

  An audio signal such as an environmental sound obtained from the directional microphone 300 is input to the audio processing unit 310. The audio signal input to the audio processing unit 310 is converted into a digital signal by an A / D converter (not shown) provided in the audio processing unit 310 and temporarily stored in the RAM 260. The digitized audio signal is sent again to the audio processing unit 310 and can be reproduced from the speaker 320.

  The operation unit 330 includes a power button 330 </ b> A, various operation buttons 330 </ b> B, a shutter release button 330 </ b> C, and the like, and is used when the user changes the setting of the digital camera 100 or performs an imaging operation.

  The CPU 250 comprehensively controls the above-described units by executing predetermined programs recorded in a RAM 260 and a ROM (Read Only Memory) 350. The RAM 260 is a high-speed accessible semiconductor memory, and the ROM 350 is configured as a non-volatile semiconductor memory (for example, a flash ROM) that cannot be electrically rewritten. A part of the area in the RAM 260 functions as a temporary storage buffer area, and temporarily stores image data and audio data.

  Each processing unit 280, 290, 310 of the CPU 250 is a functional part realized by the microcomputer executing a predetermined program.

  The image processing unit 280 is a processing unit that performs various digital image processing such as WB (white balance) processing and γ correction processing. The WB process is a process of adjusting the color balance by performing level conversion of each of R, G, and B color components based on the calculated white balance correction value, and the γ correction process corrects the gradation of the pixel data. It is processing. The compression / decompression unit 290 further compresses the image data that has been subjected to color correction processing and the like by the image processing unit 280. As the compression method, for example, a JPEG (Joint Photographic Experts Group) method or the like is adopted. The audio processing unit 310 is a processing unit that performs various digital processes on audio data.

  The CPU 250 having such a configuration performs processing in an imaging mode and a reproduction mode. For example, in the imaging mode, first, the imaging unit 220 is set to an operation mode for viewfinder (LCD 380) image output, imaging is performed at a predetermined cycle (for example, 30 frames / second), and image data corresponding to the imaging is sequentially obtained. Output (preliminary imaging). The image data output from the imaging unit 220 is displayed as a viewfinder image on the LCD 380 via the analog signal processing circuit 230, the A / D converter 240, and the image processing unit 280. In this state, when the shutter release button 330C is half-pressed (S1 state) by the user, the CPU 250 performs AE (Auto Exposure) evaluation value and AF (Auto Focus) Find the evaluation value. The CPU 250 obtains the in-focus position based on the AF evaluation value by a known hill-climbing method, for example, and moves the imaging lens 210 to the in-focus position using the AE / AF unit 360. Further, the CPU 250 determines the shutter speed (charge accumulation time in the imaging unit 220) at the time of actual imaging, the aperture value of the imaging lens 210, and the gain value of auto gain control in the analog signal processing circuit 230 based on the AE evaluation value.

  When the shutter release button 330C is fully pressed (S2 state) in the imaging mode, the CPU 250 sets the operation mode of the imaging unit 220 to the operation mode for main imaging, captures the subject, and is acquired by the imaging unit 220. A compression / decompression unit 290 generates a compressed image based on the captured image data. The compressed high resolution image data is recorded on the recording medium 370. The recording medium 370 may be, for example, an SD card (registered trademark).

  Furthermore, the digital camera 100 can also import a program operated by the digital camera 100 from the recording medium 370. For example, a control program recorded on the recording medium 370 can be taken into the RAM 260 or the ROM 350 of the CPU 250. Thereby, the control program can be updated.

  The LCD 380 is a display device that displays an object scene, displays various menus for the user, and provides information for operation. The flash 390 is a light-emitting device that emits photographing auxiliary light and irradiates a subject while light emission is permitted.

(program)
A typical configuration of the program according to the present embodiment includes a frequency analysis unit 106 that analyzes the frequency of the audio data 104A to 104D associated with each of the one or more pieces of image data 102A to 102D, and a frequency that is analyzed. A voice determination unit 110 that determines the characteristic information 108A to 108D of the voice data 104A to 104D including the gender of the speaker, and an association unit 112 that associates the characteristic information 108A to 108D with each of the one or more image data 102A to 102D, Based on the characteristic information 108A to 108D associated with each of the one or more pieces of image data 102A to 102D, an image selection unit 116 that selects reproduction target image data to be reproduced, and the selected reproduction target image The digital camera 100 serves as a playback control unit 118 that plays back data on the LCD 380. Characterized in that to function.

(Slideshow execution process)
FIG. 4 is a flowchart showing a slide show execution process performed by each element of the CPU 250 of FIG. 1, and FIG. 5 is a subroutine showing in detail the gender / voice sound determination step S500 of FIG.

  First, the frequency analysis unit 106 and the sound determination unit 110 determine the gender and the sound of the image data 102A to 102D associated with the sound data 104A to 104D in the sex / voice sound determination step S500.

  As shown in FIG. 5A, the sex / voice determination step S500 is repeated by the number of image data. The frequency analysis unit 106 measures the frequency of the audio data 104A to 104D associated with the image data 102A to 102D, and the audio determination unit 110 compares the magnitude relationship with the gender threshold value (step S600). Determination is made (step S602 or S702). When the voice frequency is between 150 Hz and 220 Hz, it is possible to determine that the child is a child. However, in this embodiment, the voice determination unit 110 uses, for example, a median value of 185 Hz as a gender threshold, and male or female. Only one of them is judged.

  Hereinafter, since the process when it is determined that it is male and the case where it is determined that it is female is the same, both will be described in parallel. FIG. 5B is a diagram showing the magnitude relationship between the threshold values used in FIG.

  The sound determination unit 110 further compares the magnitude of the image data determined to be male or female with the male high frequency threshold or the female high frequency threshold (steps S604 and S704). (Step S606, S706), if smaller, the male low frequency threshold or the female low frequency threshold is further compared with the magnitude relationship (steps S608, S708). The sound determination unit 110 classifies the image data as “normal” if it is larger than the low frequency threshold (steps S610 and S710), and classifies it as “dark” if it is smaller (steps S612 and S712).

  As described above, the gender and voice classification determined in step S500 are all recorded on the recording medium 370 as the characteristic information 108A to 108D, and are associated with the image data 102A to 102D by the associating unit 112.

  Returning to FIG. 4, the volume detector 111 detects the volume of the audio data 104 </ b> A to 104 </ b> D (step S <b> 502), and these are also recorded in the characteristic information 108 </ b> A to 108 </ b> D.

  Next, the playback method is set by the user (step S504). That is, the CPU 250 selects a male image, a female image, or both images, and displays image data classified into “bright”, “normal”, or “dark” among them. Etc. are set by the user through a predetermined interface. Also, settings such as playback in ascending order and descending order of volume can be performed here.

  The image selection unit 116 selects the reproduction target image data to be reproduced according to the user setting (step S506).

  Depending on whether the selected reproduction target image data is classified as “bright”, “normal”, or “dark”, the reproduction control unit 118 selects BGM to be reproduced simultaneously with reproduction of the reproduction target image data. (Step S508).

  Finally, the playback control unit 118 plays back the playback target image data as a slide show according to the playback method set in step S504 and together with the BGM selected in step S508 (step S510). The playback time of each playback target image may be the playback time of the audio data associated therewith. According to such a method, there is no need to set the playback time of the playback target image that is a still image.

  It should be noted that each step in the slide show execution of this specification does not necessarily have to be processed in time series in the order described in the flowchart, and may include parallel or subroutine processing.

  As mentioned above, although preferred embodiment of this invention was described referring an accompanying drawing, it cannot be overemphasized that this invention is not limited to the example which concerns. It will be apparent to those skilled in the art that various changes and modifications can be made within the scope of the claims, and these are naturally within the technical scope of the present invention. Understood.

  The present invention can be used for an image reproducing apparatus capable of reproducing image data and a program executable by the image reproducing apparatus.

1 is a block diagram of a digital camera that is an embodiment of an image playback device according to the present invention. FIG. FIG. 2 is a diagram schematically illustrating image data reproduced by the digital camera of FIG. 1, audio data associated with the image data, and characteristic information. It is a figure which shows an example of the external appearance of the digital camera of FIG. It is a flowchart which shows the slide show execution process which each element of CPU of FIG. 1 performs. It is a subroutine showing in detail the gender / voice sound determination step of FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 ... Digital camera 102A-102D ... Image data 104A-104D ... Audio | voice data 106 ... Frequency analysis part 108A-108D ... Characteristic information 110 ... Audio | voice determination part 111 ... Volume detection part 112 ... Correlation part 114 ... Reference table 116 ... Image selection part 118... Playback control unit 320... Speaker 380.

Claims (5)

A frequency analyzer for analyzing the frequency of audio data associated with each of the one or more image data;
A voice determination unit for determining characteristic information of the voice data including gender of the speaker according to the analyzed frequency;
An association unit associating the characteristic information with each of the one or more image data;
An image selection unit that selects target reproduction target image data based on characteristic information associated with each of the one or more pieces of image data;
A reproduction control unit that reproduces the selected reproduction target image data on a display unit in ascending or descending order of the analyzed frequency ;
An image reproducing apparatus comprising: The image reproduction apparatus according to claim 1,
The reproduction control unit reproduces the selected reproduction target image data by sequentially switching and reproducing the selected reproduction target image data. The image reproduction apparatus according to claim 1 or 2,
The associating unit adds characteristic information of the audio data to each of the one or more pieces of image data . The image reproduction apparatus according to any one of claims 1 to 3 ,
The image reproducing apparatus , wherein the associating unit creates association data indicating an association state between the characteristic information of the audio data and each of the one or more image data, and stores the association data as a file . A frequency analyzer for analyzing the frequency of audio data associated with each of the one or more image data;
A voice determination unit for determining characteristic information of the voice data including gender of the speaker according to the analyzed frequency;
An association unit associating the characteristic information with each of the one or more image data;
An image selection unit that selects reproduction target image data to be reproduced based on characteristic information associated with each of the one or more pieces of image data;
A program that causes an image reproduction device to function as a reproduction control unit that reproduces the selected reproduction target image data on a display unit in ascending or descending order of the analyzed frequency .