JP2011010007A - Electronic apparatus, image output method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an electronic apparatus, an image output method and a program, capable of displaying a still image relevant to a life at present or in future of a user on a photo frame while effectively using a moving image photographed with a video camera or the like.SOLUTION: A PVR 100 detects a predetermined feature from moving image content photographed with the video camera 300 to extract a key frame, outputs the key frame to a digital photo frame 200 to be displayed, and when specific time arrives, in order to change the still image to other image relevant to the specific time or other image relevant to the specific time and relevant to the key frame, outputs such other image to the digital photo frame 200.

Description

  The present invention relates to an electronic device that can output and display an image on a display device such as a digital photo frame, and an image output method and program in the electronic device.

  Conventionally, there is a digital photo frame that can display a digital image, imitating a photo frame (photo frame) for viewing a photograph printed on a paper medium. The following documents can be cited as prior documents relating to digital photo frames.

JP 2009-045139 A JP 2000-31635 A

  However, a conventional digital photo frame generally reads out still images taken by a digital camera or the like from a recording medium such as a memory card attached to the digital photo frame, and simply performs a slide show. Therefore, the digital photo frame has no advantage other than that a still image can be displayed without operating a device such as a digital camera or a computer. In addition, the user can, for example, recall past events by appreciating the still image displayed by the digital photo frame, but cannot associate it with the current or future life.

  On the other hand, for example, if a moving image taken by a user with a digital video camera or the like is viewed once after shooting, then it will not be viewed unless there is something special. It was.

  In view of the circumstances as described above, an object of the present invention is to display a still image related to a user's current or future life on a photo frame while effectively using a moving image taken by a video camera or the like. It is to provide an electronic device, an image output method, and a program.

In order to achieve the above object, an electronic apparatus according to an embodiment of the present invention includes a storage unit, a first communication unit, and a control unit. The storage unit stores a moving image and predetermined date and time information. The first communication unit communicates with a digital photo frame. The control unit extracts a predetermined feature from the moving image, extracts a first still image based on the extracted feature, and displays the first still image in order to display the first still image. The unit outputs the first still image to the digital photo frame. The control unit acquires a second still image related to the date information when the date corresponding to the date information arrives, and changes the first still image to the second still image. Therefore, the second communication image is output to the digital photo frame by the first communication unit.
With this configuration, the electronic device can extract the first still image including the specific object from the moving image and display the first still image on the digital photo frame, and at the date and time related to the specific object, The still image can be changed to the second still image. Therefore, the electronic device not only allows the viewer of the photo frame to browse each scene in the moving image as a still image, but also at a specific date and time, reminds the user of information related to the date and time, and the user's current and future behavior Can be used as a guideline.
Here, the moving image is, for example, a moving image taken by a digital video camera or the like, and the predetermined features include, for example, moving features such as pan, tilt, and zoom, and people (faces), animals, buildings, and the like. This is a feature indicating an object. The electronic device is, for example, a digital video camera, a recording / reproducing apparatus, other AV devices, a PC (Personal Computer), a mobile phone, or the like.

In the electronic device, the date information may be date information related to a specific object that can be included in the moving image. The control unit may recognize the specific object from the moving image and extract a still image including the specific object as the second still image. Then, based on the date and time information related to the specific object, the control unit transmits the second still image to the digital photo by the first communication unit when the date and time corresponding to the date and time information arrives. You may output to a frame.
Thus, the electronic device displays the second still image extracted from the moving image on the digital photo frame at the date and time related to the object in the second still image, and the event related to the object is related to the date and time. Remind users. For example, if the object is a child and the date of birth of the child is stored as date / time information, the photo frame includes an image not related to the child as a first still image at a date / time other than the birthday. Even when displayed, the child image as the second still image is displayed when the date and time arrives. Therefore, the user can recall the child's birthday by browsing the child image at the date and time.
In this case, the moving image may have audio information, and the digital photo frame may be able to output the audio information. In this case, the control unit may extract audio information related to a specific object included in the extracted second still image from the moving image. The control unit may output the audio information to the digital photo frame in order to output the extracted audio information when the date and time corresponding to the date and time information arrives.
Thus, the electronic device can display a still image related to the specific date and time on the photo frame at a specific date and output sound (music) related to the specific date and time. It can be provided more impressively by the user on the photo frame.

The electronic device may further include a second communication unit connected to the network. In this case, the date information may be a predetermined time of the day. In this case, the control unit may recognize a specific object from the moving image and extract a still image including the specific object as the first still image. The control unit retrieves and receives related information related to the first still image from the network by the second communication unit, and when the predetermined time arrives, the received related information The second still image may be output to the digital photo frame by the first communication unit.
As a result, the electronic device not only allows the object included in the still image extracted from the moving image to be browsed on the photo frame but also provides information related to the object in a timely manner. Can be improved. For example, if the object is a user's pet, information about pet food can be displayed at a time when the user will go shopping, such as at 4 pm.

In addition, the control unit searches for advertisement information of a product related to the object included in the first still image and available only on a day to which the time of executing the search belongs as the related information. Also good.
As a result, the electronic device displays relevant information on the photo frame at a predetermined time, thereby invoking actions to be taken in the user's daily lifestyle based on the relevant information and providing guidelines on the actions. Can be. For example, when a user goes to a supermarket for shopping at 4 pm every day, the electronic device can search and display advertisements (flyers) of products for sale on that day in the supermarket as related information.

  An image output method according to another aspect of the present invention includes storing a moving image and predetermined date / time information. A predetermined feature is extracted from the moving image, and a first still image is extracted based on the extracted feature. The extracted first still image is output to a digital photo frame in order to display the first still image. Then, when the date corresponding to the date information arrives, a second still image related to the date information is acquired. The second still image is output to the digital photo frame in order to change the first still image to the second still image.

  A program according to still another aspect of the present invention includes a storage step, an extraction step, a first output step, an acquisition step, and a second output step in the electronic device. The storing step stores a moving image and predetermined date / time information. The extraction step extracts a predetermined feature from the moving image and extracts a first still image based on the extracted feature. The first output step outputs the first still image to a digital photo frame in order to display the extracted first still image. The acquisition step acquires a second still image related to the date information when the date corresponding to the date information arrives. The second output step outputs the second still image to the digital photo frame in order to change the first still image to the second still image.

  As described above, according to the present invention, a still image related to a user's current or future life can be displayed on a photo frame while effectively using a moving image taken by a video camera or the like.

It is a figure which shows the outline | summary of the image output system in one Embodiment of this invention. It is a figure which shows the hardware constitutions of PVR which concerns on one Embodiment of this invention. It is the figure which showed the structure of the said characteristic extraction part of PVR which concerns on one Embodiment of this invention. It is the figure which showed notionally the mode of the object recognition process of PVR which concerns on one Embodiment of this invention. It is the flowchart which showed the flow of the still image output process of PVR which concerns on one Embodiment of this invention. It is the flowchart which showed the flow of the still image output process of PVR in the 1st modification of this invention. It is the flowchart which showed the flow of the specific process of the still image change process of PVR in the 1st modification of this invention. It is the figure which showed notionally the mode of the still image change process in the 1st modification of this invention. It is the flowchart which showed the flow of the still image output process of PVR in the 2nd modification of this invention. It is the flowchart which showed the flow of the specific process of the still image output process of PVR in the 2nd modification of this invention. It is the flowchart which showed the flow of the specific process of the still image change process of PVR in the 2nd modification of this invention.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[Outline of PVR]
FIG. 1 is a diagram showing an outline of an image output system according to an embodiment of the present invention.
As shown in the figure, a PVR (Personal Video Recorder) 100 can communicate with a photo frame 200, a video camera 300, and a portable terminal 400 by wire or wireless, and can also be connected to a network 50 such as the Internet. .

  The PVR 100 receives and stores moving image content captured by the user stored in the video camera 300 from the video camera 300. The PVR 100 also stores time codes included in the moving image content as date / time information. Further, the PVR 100 receives the user's schedule information from the mobile terminal and stores the schedule information as date / time information.

  The PVR 100 extracts a still image from the moving image content and outputs the still image to the digital photo frame 200 for display. The PVR 100 can change a still image displayed on the digital photo frame 200 to another still image based on the date / time information. The other still images include those received via the network 50. Further, the PVR 100 can transmit and output audio information to the digital photo frame 200 together with a still image.

  The digital photo frame 200 includes a frame 201, an LCD (Liquid Crystal Display) 202, and a communication interface such as a USB (Universal Serial Bus) interface (not shown), a wireless LAN, and Bluetooth (registered trademark). The digital photo frame 200 (hereinafter also simply referred to as the photo frame 200) can receive a still image output from the PVR 100 through the communication interface and display the still image on the LCD 202. The digital photo frame 200 has a speaker (not shown), and can also output audio information received from the PVR 100.

[Hardware configuration of PVR]
FIG. 2 is a diagram showing a hardware configuration of the PVR 100. As shown in FIG.
As shown in the figure, the PVR 100 includes a digital tuner 1, a demodulator 2, a demultiplexer 3, a decoder 4, a recording / reproducing unit 5, an HDD (Hard Disk Drive) 8, an optical disk drive 9, and a communication unit 11. The PVR 100 includes a CPU (Central Processing Unit) 12, a flash memory 13, and a RAM (Random Access Memory) 14. Further, the PVR 100 includes an operation input unit 15, a graphic control unit 16, a video D / A (Digital / Analog) converter 17, an audio D / A (Digital / Analog) converter 18, a wireless interface (I / F) 19, and an external interface 20. And a feature extraction unit 21.

  Under the control of the CPU 12, the digital tuner 1 selects a specific channel for digital broadcasting via the antenna 30 and receives a broadcast signal including program data. The demodulator 2 demodulates the modulated broadcast signal.

  The demultiplexer 3 separates the multiplexed broadcast signal into a video signal and an audio signal and supplies them to the decoder 4. The demultiplexer 3 can also supply each separated signal to the feature extraction unit 21. The decoder 4 decodes the video signal and the audio signal separated by the demultiplexer 3, respectively. Each decoded signal is supplied to the recording / reproducing unit 5.

  The recording / reproducing unit 5 includes a recording unit 6 and a reproducing unit 7. The recording unit 6 temporarily stores the video signal and the audio signal decoded and input by the decoder 4 and outputs and records them on the HDD 8 and the optical disk drive 9 while controlling the timing and the data amount. The recording unit 6 can also read the content recorded on the HDD 8, output it to the optical disc drive 9, and record it on the optical disc 10. The playback unit 7 reads the video signal and audio signal of the video content recorded on the HDD 8 or the optical disk 10 and outputs them to the decoder 4 for playback while controlling the timing and the data amount.

  The HDD 8 stores a program received via the digital tuner 1 and various contents received via the network 50 such as the Internet by the communication unit 11 in a built-in hard disk. The HDD 8 also stores the moving image content and the date / time information input from the digital camera 300 or the portable terminal 400 via the external interface 20 or the like. The date information includes, for example, events related to a specific person (birthday, anniversary, etc.) and dates of past events (travel, wedding, etc.) experienced by the user. This date / time information is stored in association with information identifying the event or event. Further, the HDD 8 also stores learning data for recognizing a specific object (for example, a face) from the moving image content. When these stored contents are reproduced, the HDD 8 reads these data from the hard disk and outputs them to the recording / reproducing unit 5.

  The HDD 8 may store various programs such as applications, other data, and the like. These programs and data are read from the HDD 8 according to a command from the CPU 12 and executed in the RAM 14 at the time of execution and reference.

  Similar to the HDD 8, the optical disk drive 9 can record various data such as the program content on the mounted optical disk 10, and can read the recorded data. The various programs may be recorded on a general-purpose recording medium such as the optical disk 10 and installed in the PVR 100 by the optical disk drive 9.

  The communication unit 11 is a network interface for connecting to the network 50 and exchanging data with other devices on the network 50 using a protocol such as TCP / IP (Transmission Control Protocol / Internet Protocol). When the data received by the communication unit 11 is multiplexed, the data is supplied to the demultiplexer 3. At least a part of the received data is also supplied to the feature extraction unit 21 as necessary.

  The wireless interface 19 includes a module conforming to a wireless communication standard such as a wireless LAN and Bluetooth (registered trademark), for example, and can transmit still images and audio information to the photo frame 200 wirelessly.

  The external interface 20 includes a USB interface, for example, and receives moving image content from the video camera 300 and schedule information from the mobile terminal 400. The moving image content and schedule information may be received from the video camera 300 and the portable terminal 400 via the wireless interface 19.

  The CPU 12 accesses the RAM 14 or the like as necessary, and comprehensively controls processing of each block of the PVR 100 such as data recording and playback processing by the recording / playback unit 5.

  The flash memory 13 is, for example, a NAND type, and is a non-volatile memory in which firmware such as an OS, a program, and various parameters executed by the CPU 12 is fixedly stored. In addition, the flash memory 13 is a program used for extracting still images and audio information from moving image content, changing a still image and audio information to be output to the photo frame 200, various other programs, various EPG data, and the like. Store the data.

  The RAM 14 is a memory that is used as a work area of the CPU 12 and temporarily stores an OS, a program, processing data, and the like during the above-described still image and audio information extraction processing and change processing.

  The operation input unit 15 inputs, for example, various setting values and commands by a user operation from a remote controller 22 having a plurality of keys (hereinafter referred to as a remote controller 22), and outputs it to the CPU 12. Of course, the operation input unit 15 may be configured by a keyboard or mouse connected to the PVR 100, a switch mounted on the PVR 100, or the like without using the remote controller 22.

  The graphic control unit 16 performs graphic processing such as OSD (On Screen Display) processing on the video signal output from the decoder 4 and other video data output from the CPU 12, for example, a display device such as a television device (TV). A video signal to be displayed on the screen is generated.

  The video D / A converter 17 converts the digital video signal input from the graphic control unit 16 into an analog video signal and outputs the analog video signal to the display device.

  The audio D / A converter 18 converts the digital audio signal input from the decoder 4 into an analog audio signal and outputs the analog audio signal to a TV or the like.

  The feature extraction unit 21 extracts predetermined video features and audio features from the moving image content received and stored from the video camera 300, and based on the features, a still image and a still image for output to the photo frame 200 are extracted. Extract voice information. The predetermined video features are, for example, motion features such as pan, tilt, and zoom, and features that indicate objects such as people (faces), animals, and buildings. The predetermined voice feature is, for example, a feature showing a human voice, more specifically, a feature showing a duration of voice power.

[Configuration of Feature Extraction Unit]
FIG. 3 is a diagram illustrating a configuration of the feature extraction unit 21 of the PVR 100.
As shown in the figure, the feature extraction unit 21 includes a key frame extraction unit 211, an image feature detection unit 212, and an image memory 213 as blocks for performing processing relating to an image. The feature extraction unit 21 includes a speech feature extraction unit 214, a speech feature detection unit 215, and a speech memory 216 as blocks that perform processing related to speech.

  The user uses, for example, the remote controller 22 to display a still image extracted from the moving image content stored in the PVR 100 on the photo frame 200 among the operation modes of the PVR 100 (broadcast program viewing mode, recorded program viewing mode, etc.). Select. When the mode selection operation is input, the mode selection information is transmitted to the CPU 12 via the operation input unit 15, and feature extraction processing from the moving image content is started.

  The CPU 12 inputs the moving image content transferred from the video camera 300 among the content stored in the HDD 8 to the feature extraction unit 21. Image data of the moving image content is input to the key frame extraction unit 211 and the image feature detection unit 212.

  The image feature detection unit 212 detects camera features such as pan, tilt, and zoom, and image features such as an object (such as a face) from the moving image content, and transmits the results to the CPU 12. Based on the detection result, the CPU 12 controls the key frame extraction unit 211 so as to extract, from the moving image content, the frame from which the image feature is extracted as a key frame. The extracted key frames are stored in the image memory 213 and are output to the photo frame 200 through processing described later.

  Here, camera features and objects are extracted as image features because the camera feature section is a section where the user intentionally operates the camera and is taken as an important feature for the user. This is because an object such as a face is likely to be the focus of user attention. The PVR 100 allows a user to view an image that has a strong impression for the user by outputting and displaying a still image including these features on the photo frame 200.

  The audio feature detection unit 215 detects, for example, audio in the preceding and following sections of the key frame in the audio data of the moving image content as audio features and transmits the detection result to the CPU 12. . Based on the detection result, the CPU 12 controls the voice feature extraction unit 214 so as to extract the voice data of the section in which the voice feature is detected from the moving image content as the voice feature section data. The extracted voice feature section data is stored in the voice memory 216, and is output to the photo frame 200 by the processing described later.

[Operation of PVR]
Next, the operation of the PVR 100 configured as described above will be described. In the following description, the CPU 12 of the PVR 100 will be described as the main operating subject. However, this operation is also performed in cooperation with the feature extraction unit 21 and a program executed under the control of the CPU 12.

(Object discrimination processing)
First, an object recognition process necessary for changing a still image displayed on the photo frame 200 in the present embodiment will be described. FIG. 4 is a diagram conceptually showing the state of the object recognition process.

  As shown in the figure, the HDD 8 of the PVR 100 stores still images indicating various objects as learning data. In the present embodiment, assuming that the moving image captured by the digital camera 300 is in the home, for example, an image showing the user's family and pets (for example, baby, dog, eldest daughter, mother, father, etc.) It is stored as learning data. The feature extraction unit 21 extracts feature quantities such as edges, colors, textures, and the like from the learning data, converts them into multidimensional vectors, and generates a discriminant function by statistical machine learning based on the feature vectors. The generated discriminant function is stored in, for example, the HDD 8 or the flash memory 13, and used for discriminant analysis processing necessary for detecting an object from moving image content. For example, based on feature data extracted from unknown image data included in the moving image content, it is determined that the unknown object is a known object (a dog in the figure) by being input to the discriminant function. The result is output.

  Further, instead of the discriminant analysis process using the discriminant function, a discriminant analysis process using a machine learning method such as a support vector machine (SVM), Ada-boost, or a neural network may be executed. In this case, instead of the discriminant function, a processing module that executes the discriminating process is incorporated in the PVR 100.

  For example, when recognizing a human face instead of the entire object, the feature extraction unit 21 extracts face feature data using, for example, a feature filter, luminance distribution information, skin color information, and the like according to the positional relationship of each part of the face. To generate a feature vector.

(Still image output processing to photo frame)
Next, still image output processing to a photo frame by the PVR 100 will be described. FIG. 5 is a flowchart showing the flow of the still image output process.

  As shown in the figure, first, the CPU 12 of the PVR 100 extracts video features and audio features from the moving image content input from the video camera 300 (step 41). When a key frame is detected by the video feature extraction process (Yes in step 42), the CPU 12 extracts the key frame (step 43). As the key frame, a frame having camera characteristics such as pan, tilt, and zoom, and a frame including an object recognized based on the discriminant function are extracted. The extraction process is started when the user selects the mode, for example.

  In order to recognize camera features, for example, the feature extraction unit 21 detects a motion vector by performing block matching processing for each pixel between a plurality of frames, and calculates an affine coefficient by multiple regression analysis based on the motion vector. To do. The feature extraction unit 21 determines whether the affine coefficient indicates a pan, tilt, or zoom feature based on, for example, a threshold corresponding to each camera feature, and detects the presence or absence of the camera feature.

  Further, the CPU 12 extracts, for example, a section including a human voice as voice feature section data from the voice data of the section from which the key frame is extracted. Among the voice sections included in the moving image data, the human voice includes consonants, vowels, breath breaths, etc., so that the duration of the predetermined power or higher is shorter than voice other than voice such as music. is there. Using this feature, the feature extraction unit 21 sets, for example, a threshold value related to time, and when the average duration time equal to or greater than a predetermined power is smaller than the threshold value, the segment is set as a voice segment. If it is larger, it is determined that the section is a non-voice section.

  Subsequently, the CPU 12 stores the extracted key frame in the image memory 213, and stores the extracted voice feature section data in the voice memory 216 (step 44). At this time, information for identifying the object (person) included in the stored key frame is also stored.

  The CPU 12 repeats the feature extraction / memory processing for all moving image contents input from the video camera 300, for example (step 45).

  Subsequently, the CPU 12 outputs the stored key frame to the photo frame 200, and starts image display processing in the photo frame 200 (step 46). For example, the CPU 12 gives priority to the extracted key frames according to the extracted features and sequentially outputs them to the photo frame 200 at predetermined intervals. The predetermined interval is, for example, 30 seconds, but is not limited thereto. The photo frame 200 displays the key frame at each time interval. For example, the priority order may be set in the order of (1) zoom, (2) face, (3) pan, and (4) tilt.

  Subsequently, the CPU 12 acquires the date and time (including year) information from the HDD 8 or the like (step 47), and compares the date and time information with the current date and time indicated by the built-in clock (step 48). If the CPU 12 determines that the specific date and time corresponding to the stored date and time information has arrived (Yes in step 48), the CPU 12 acquires a still image related to the date and time information. The related still image is, for example, a frame including an object related to date / time information corresponding to the date / time of arrival among the frames extracted as the key frame. The CPU 12 outputs a frame related to the date and time information to the photo frame 200, and changes the frame displayed so far to the new frame (step 49). At this time, if there is audio information related to the new frame, the CPU 12 also outputs the audio information.

  For example, when the stored date and time information is the birthday of the child of the user, the CPU 12 extracts and outputs a frame including the image of the child from the image memory 213 when the birthday of the child arrives. Then, it is displayed on the photo frame 200 in place of the previous frame. Thus, the user can recognize that the day is a child's birthday and can make a plan such as preparing a present or a special meal. At this time, for example, the CPU 12 may output the audio information of the music “Happy Birthday” to the photo frame 200 and reproduce it. This will make the birthday atmosphere more exciting.

  If the stored date and time information is the user's own wedding anniversary, the CPU 12 extracts a frame including the user and the spouse when the anniversary arrives, and sends the frame to the photo frame 200. It may be output. As a result, the user can recognize the anniversary and make some preparations, and can soak in memories while browsing the photo frame 200 on the table of the day, for example.

  The CPU 12 repeats the above processing every time the date and time corresponding to the stored date and time information arrives until the end of the processing is instructed by the user.

[Summary]
As described above, according to the present embodiment, a still image related to a user's current or future life can be displayed on a photo frame while effectively using a moving image taken by a video camera or the like.

[Modification]
The present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the scope of the present invention.

(Modification 1)
In the above-described embodiment, when changing the still image displayed in the photo frame 200, the key frame related to the date / time information stored in the HDD 8 among the key frames extracted from the moving image content is output. However, the PVR 100 acquires, for example, information related to the key frame that has been displayed in the photo frame 200 from the outside, and outputs the acquired related information to the photo frame 200 as a new still image for display. You can also Hereinafter, specific processing of the PVR 100 in this case will be described.

  FIG. 6 is a flowchart showing the flow of still image output processing of the PVR 100 in the present modification. In the following, description of the output processing of audio information to the photo frame 200 by the PVR 100 is omitted, but it is assumed that the audio information is also extracted from the key frame and output to the photo frame 200 as in the above embodiment.

  As shown in the figure, first, the CPU 12 initializes a detection flag k (n) to 0 when detecting a key frame from the moving image content (step 61). Here, n is 0 ≦ n ≦ kmax.

  Subsequently, the CPU 12 sets a counter m indicating the number of detected key frames to 0 (step 62). Subsequently, as in the above-described embodiment, the CPU 12 extracts image features such as camera features and object features for each frame from the moving image content (step 63).

  Subsequently, the CPU 12 determines whether or not the processing target frame is a key frame by the feature extraction process (step 64). If the frame is a key frame (Yes), the mth frame is determined. Is set to k (m) = 1 (step 65), and m is incremented (step 66).

  Then, the CPU 12 extracts the key frame (step 67) and stores it in the image memory 213 (step 68). The CPU 12 repeats the above processing for all frames of all moving image contents captured from the video camera 300 (step 69).

  When the key frame extraction process ends (Yes in step 69), the CPU 12 executes an output process of the key frame to the photo frame 200.

  First, the CPU 12 initializes the counter n of the key frame output process to 0 (step 70). Subsequently, the CPU 12 starts a process of sequentially outputting and displaying the extracted key frames to the photo frames in accordance with, for example, the priority order in the embodiment (step 71).

  Subsequently, the CPU 12 acquires the date and time (including year) information from the HDD 8 or the like (step 72), and compares the date and time information with the current date and time indicated by the built-in clock (step 73). When the CPU 12 determines that the current date / time has reached a specific date / time corresponding to the stored date / time information (Yes in step 73), the CPU 12 changes the image displayed on the photo frame 200 until then. Is executed (step 74). The date and time information in this case is the date and time (time) at which the user's habitual behavior will start, such as the time when the user goes out for shopping every day.

  FIG. 7 is a flowchart showing a specific flow of the image change process. As shown in the figure, the CPU 12 sends related information related to the key frame k (n% m) from the network 50 by the communication unit 11 together with life information such as a supermarket flyer which is a reference for the shopping, for example. Search is performed (step 731).

  Here, the CPU 12 prevents the counter n of the key frame output process from overflowing by the arithmetic process of mod m (modulo m). That is, the result of the modulo operation is 0 when n is divisible by m, and the remainder when it is not divisible, so that the number of detected key frames (m) is equal to the number of detected key frames. Subsequent output (display) processing is repeated.

  When the related information exists (Yes in Step 732), the CPU 12 displays the related information following the life information such as the leaflet (Step 733). When the related information does not exist (No), the current key frame display is displayed. Is maintained (step 734). The related information is information related to the date and time information and related to the key frame that has been displayed in the photo frame 200 until then.

  In the search, for example, an object name included in a key frame that has been displayed in the photo frame 200 is searched as a keyword. In this search, the CPU 12 stores the object name when extracting the key frame. The CPU 12 also stores the URL of a website used as the search engine. Further, the CPU 12 may transmit the key frame itself including the object to the search engine on the network 50 regardless of the keyword, and acquire related information by image search.

  The CPU 12 repeats the above processing by the number of key frames in accordance with the counter n (step 76), and ends the processing when a display end instruction is issued from the user (Yes in step 75).

  FIG. 8 is a diagram conceptually showing related information display processing as the image change processing. As shown in the figure, when a frame including a baby is extracted and displayed, for example, as a key frame, the CPU 12 adds information related to the frame, such as a diaper, in addition to life information such as the above-mentioned grocery flyer. Search and display advertisement information (sale information) about baby items. When a frame including a pet dog is extracted and displayed as a key frame, the CPU 12 searches and displays advertisement information related to pet food as related information. Further, the CPU 12 may search and display not only advertisement information but also area information (map information) that can be used as a reference for setting a new walk course for the dog.

  Further, the CPU 12 may display a clock image instead of the key frame when a fixed time such as 12:00, 15:00, or the like has come. The clock image is stored in advance in, for example, the HDD 8 or the like.

  After the life information and the related information are displayed for a predetermined time or a predetermined number of times, the CPU 12 returns to the key frame display process again.

  Through the above processing, the PVR 100 not only allows the key frame extracted from the moving image content to be browsed on the photo frame, but also the relevant information of the object included in the key frame, such as the time when the user usually goes to shopping, etc. Can be provided at any time. Therefore, the user can efficiently execute subsequent actions such as shopping with reference to the related information.

(Modification 2)
In the first modification, related information of each key frame is retrieved when a specific date corresponding to the stored date information arrives, and these are repeatedly displayed for each key frame in the same time zone. However, in this modification, the PVR 100 periodically checks whether or not related information of any key frame exists, and displays related information when related information exists at the time of the confirmation. I am going to do that. FIG. 9 is a flowchart showing the flow of image change processing of the PVR 100 in this modification.

  As shown in the figure, the processing of the PVR 100 in this modification is the same as that of Modification 1 except for the specific processes in Steps 91 and 94. FIG. 10 is a flowchart showing a specific processing flow of step 91, and FIG. 11 is a flowchart showing a specific processing flow of step 94.

As shown in FIG. 10, at the start of the frame output process to the photo frame 200, the CPU 12 initializes the counter t of the related information detection process according to the number of extracted key frames to 0, and the related information Is set to 0 (step 911).
Subsequently, the CPU 12 searches for related information for the t-th key frame k (t) (step 912). If the related information exists (Yes in step 913), the CPU 12 sets the detection flag f to 1 ( Step 914). Then, the CPU 12 outputs and displays the key frame k (t) for which the relevant information is retrieved to the photo frame 200 (step 915).
On the other hand, if the related information does not exist (No in Step 913), the CPU 12 increments the counter t (Step 916), and if t is less than the total number m of extracted key frames (No in Step 917), the next step The related information about the key frame is searched (step 912). If t> m (Yes in step 917), the extracted key frames are sequentially and repeatedly displayed by the modulo calculation (step 918).

  As shown in FIG. 11, when a specific periodic date and time has arrived (step 93), the CPU 12 determines whether there is related information for the key frame displayed at that time (whether the detection flag is on or not). ) (Step 941), if it exists (Yes), the relevant information is displayed (step 942). If it does not exist (No), the display of the previous key frame is maintained (step 943). .

  Through the above processing, the presence / absence of related information of key frames is periodically checked and displayed whenever there is related information. Therefore, the user always browses the latest related information for any key frame. can do.

[Others]
In the above-mentioned embodiment, the example which applied this invention to PVR100 was shown. However, the present invention is not limited to PVR, and can be applied to any electronic device as long as it can store moving image content, such as a PC, a mobile phone, and other AV devices. Further, a still image may be directly output from the video camera 300 to the photo frame 200 without passing through another device such as the PVR 100. Furthermore, not only the moving image content photographed by the user but also the key frame may be extracted from the moving image content such as a broadcast program and output to the photo frame 200.

  In the above-described embodiment, the related information output to the photo frame by the image change process is the advertisement information of the product. For example, other related information such as news information and program information related to the key frame is included. It may be retrieved and output. Thus, the user can obtain a guideline for subsequent actions.

  In the above-described embodiment, the image changing process is executed by outputting an image different from the key frame. However, for example, when the specific time arrives, the key frame is enlarged or reduced. The image change process may be executed by giving This also allows the user to remember any event or schedule related to the key frame.

11. Communication unit 12 ... CPU
DESCRIPTION OF SYMBOLS 19 ... Wireless interface 20 ... External interface 21 ... Feature extraction part 50 ... Network 100 ... PVR
DESCRIPTION OF SYMBOLS 200 ... Digital photo frame 211 ... Key frame extraction part 212 ... Image feature detection part 213 ... Image memory 214 ... Audio | voice feature extraction part 215 ... Audio | voice feature detection part 216 ... Audio | voice memory 300 ... Video camera 300 ... Digital camera 400 ... Portable terminal

Claims (7)

A storage unit for storing moving images and predetermined date and time information;
A first communication unit that communicates with the digital photo frame;
A first feature is extracted from the moving image, a first still image is extracted based on the extracted feature, and the first communication unit displays the first still image. Is output to the digital photo frame, and when the date and time corresponding to the date and time information arrives, a second still image related to the date and time information is obtained, and the first still image is acquired as the second image. An electronic device comprising: a control unit that outputs the second still image to the digital photo frame by the first communication unit in order to change to a still image. The electronic device according to claim 1,
The date and time information is date and time information related to a specific object that can be included in the moving image,
The control unit recognizes the specific object from the moving image, extracts a still image including the specific object as the second still image, and based on the date and time information related to the specific object. An electronic device that outputs the second still image to the digital photo frame by the first communication unit when the date and time corresponding to the date and time information arrives. The electronic device according to claim 2,
The moving image has audio information,
The digital photo frame can output the audio information;
The control unit extracts audio information related to a specific object included in the extracted second still image from the moving image, and extracts the audio information when the date and time corresponding to the date and time information arrives. An electronic device that outputs the audio information to the digital photo frame in order to output the audio information. The electronic device according to claim 1,
A second communication unit connected to the network;
The date and time information is a predetermined time of the day,
The control unit recognizes a specific object from the moving image, extracts a still image including the specific object as the first still image, and relates related information related to the first still image to the first image. When the predetermined time arrives, the received related information is retrieved and received as the second still image by the first communication unit to the digital photo frame. Output electronic equipment. The electronic device according to claim 4,
The control unit searches for advertisement information of a product related to the object included in the first still image and available only on a day to which the time of executing the search belongs, as the related information. . Stores moving images and predetermined date and time information,
Extracting a predetermined feature from the moving image, extracting a first still image based on the extracted feature,
In order to display the extracted first still image, the first still image is output to a digital photo frame,
When the date and time corresponding to the date and time information has arrived, a second still image related to the date and time information is acquired,
An image output method for outputting the second still image to the digital photo frame in order to change the first still image to the second still image. Electronic equipment,
Storing a moving image and predetermined date and time information;
Extracting a predetermined feature from the moving image and extracting a first still image based on the extracted feature;
Outputting the first still image to a digital photo frame to display the extracted first still image;
Obtaining a second still image related to the date and time information when the date and time corresponding to the date and time information has arrived;
Outputting the second still image to the digital photo frame in order to change the first still image to the second still image.

Images