JP5414842B2 - Electronic device, image display method, and content reproduction program - Google Patents

Description

  The present invention relates to an image display technique applied to an electronic device such as a personal computer.

  In recent years, image sensors such as CCD (Charge Coupled Device) and CMOS (Complementary Metal-Oxide Semiconductor) image sensors have been remarkably increased in pixel size and downsizing. It has become possible to take images. A so-called digital image such as a moving image or a still image captured by the image sensor can be stored in a card-type storage medium or HDD (hard disk drive) and managed in a compact manner.

  Recently, an image reproducing device called a digital photo frame has begun to spread. The digital photo frame has, for example, a slot for storing a card-type storage medium, and has a function of sequentially displaying a plurality of still images stored in the storage medium stored in the slot at predetermined intervals. It is used as a desktop accessory. For example, personal computers, electronic cameras, and the like generally have a function of sequentially displaying a plurality of still images at a predetermined interval like a digital photo frame.

  Japanese Patent Application Laid-Open No. 2005-228867 describes a “moving image list display that creates a summary of unique moving images that are easy for the user to compare by considering the correlation of each scene displayed in a list. "Apparatus" (see paragraph [0009]) is disclosed. Various mechanisms for selecting target digital data from a large amount of digital data as in Patent Document 1 have been proposed. Therefore, by applying these mechanisms, it is possible to efficiently select a digital image to be displayed as a digital photo frame from among a large number of digital images managed by the HDD.

JP 2008-153920 A

  By the way, a conventional digital photo frame (including a digital photo frame function in a personal computer or an electronic camera) selects a plurality of digital images selected from a large amount of digital images and stored in a card-type storage medium, The display is based on rigid and simple rules such as shooting date and time, storage position order on a storage medium, and digital image file name order. In other words, there has been no image display method with high intelligence, such as arranging the digital images to be displayed in an appropriate effective order so that the user can enjoy the digital images more happily.

  The present invention has been made in view of such circumstances, and provides an electronic device, an image display method, and a content reproduction program that can display images to be displayed in an effective order. Objective.

According to the embodiment, the electronic device includes setting means and display control means. The setting means sets an image selection rule and an image display rule based on a keyword selected from the keywords presented as options. The display control means is information for searching for a target image, and includes index information that can determine whether or not a person is captured and whether or not a character is captured in each image, and the selection rules and display rules set by the setting means. based on bets, select the images to be displayed, to display control by arranging images the selected.

1 is a diagram showing an external appearance of an electronic device according to an embodiment of the present invention. 2 is an exemplary diagram showing a system configuration of the electronic apparatus of the embodiment. FIG. 2 is an exemplary block diagram illustrating a functional configuration of a content reproduction application program which operates on the electronic apparatus of the embodiment. FIG. 6 is an exemplary diagram showing a configuration example of index information used by a content reproduction application program operating on the electronic apparatus of the embodiment. FIG. 6 is an exemplary view showing an example of a person designation screen for slide show display displayed by a content reproduction application program operating on the electronic apparatus of the embodiment. FIG. 6 is an exemplary diagram showing an example of a detailed setting screen for slide show display displayed by a content reproduction application program operating on the electronic apparatus of the embodiment. FIG. 4 is a conceptual diagram for explaining a basic principle of image selection and image display by a content reproduction application program operating on the electronic apparatus of the embodiment. FIG. 6 is an exemplary flowchart showing the procedure of a slide show display process executed by the content reproduction application program operating on the electronic apparatus of the embodiment.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a diagram illustrating an external appearance of an electronic apparatus according to an embodiment of the present invention. This electronic apparatus is realized as a notebook type personal computer 10, for example.

  As shown in FIG. 1, the computer 10 includes a computer main body 11 and a display unit 12. The display unit 12 incorporates a display device having an LCD (liquid crystal display) 17. The display unit 12 is attached to the computer main body 11 so as to be rotatable between an open position where the upper surface of the computer main body 11 is exposed and a closed position covering the upper surface of the computer main body 11.

  The computer main body 11 has a thin box-shaped casing. On the upper surface of the computer main body 11, a keyboard 13, a power button 14 for powering on / off the computer 10, an input operation panel 15, a touch pad 16, Speakers 18A, 18B, etc. are arranged. Various operation buttons are provided on the input operation panel 15.

  Further, on the right side surface of the computer main body 11, for example, a USB connector 19 for connecting a USB cable of a USB (universal serial bus) 2.0 standard or a USB device is provided. Further, an external display connection terminal (not shown) corresponding to, for example, the HDMI (high-definition multimedia interface) standard is provided on the back of the computer main body 11. This external display connection terminal is used to output a digital video signal to an external display.

  FIG. 2 is a diagram showing a system configuration of the computer 10.

  As shown in FIG. 2, the computer 10 includes a CPU (central processing unit) 101, a north bridge 102, a main memory 103, a south bridge 104, a GPU (Graphics Processing Unit) 105, a VRAM (video RAM: random access). memory) 105A, sound controller 106, BIOS-ROM (basic input / output system-read only memory) 107, LAN (local area network) controller 108, HDD 109, ODD (optical disc drive) 110, USB controller 111, wireless LAN controller 112, an embedded controller / keyboard controller (EC / KBC) 113, an EEPROM (electrically erasable programmable ROM) 114, and the like.

  The CPU 101 is a processor that controls the operation of the computer 10, and executes various application programs such as an operating system (OS) 201 and a content reproduction application program 202 that are loaded from the HDD 109 to the main memory 103. The content reproduction application program 202 is software having a function of reproducing AV (audio / video) content stored in a DVD (digital versatile disc) set in the ODD 110, for example. The content reproduction application program 202 also has a function of displaying a digital image stored in the HDD 109 like a so-called digital photo frame. The CPU 101 also executes the BIOS stored in the BIOS-ROM 107. The BIOS is a program for hardware control.

  The north bridge 102 is a bridge device that connects the local bus of the CPU 101 and the south bridge 104. The north bridge 102 also includes a memory controller that controls access to the main memory 103. The north bridge 102 also has a function of executing communication with the GPU 105 via a PCI EXPRESS standard serial bus or the like.

  The GPU 105 is a display controller that controls the LCD 17 used as a display monitor of the computer 10. A display signal generated by the GPU 105 is sent to the LCD 17. The GPU 105 can also send a digital video signal to the external display device 1 via the HDMI control circuit 3 and the HDMI terminal 2.

  The HDMI terminal 2 is the aforementioned external display connection terminal. The HDMI terminal 2 can send an uncompressed digital video signal and a digital audio signal to the external display device 1 such as a television with a single cable. The HDMI control circuit 3 is an interface for sending a digital video signal to the external display device 1 called an HDMI monitor via the HDMI terminal 2.

  The south bridge 104 controls each device on a peripheral component interconnect (PCI) bus and each device on a low pin count (LPC) bus. Further, the south bridge 104 includes an IDE (Integrated Drive Electronics) controller for controlling the HDD 109 and the ODD 110. Further, the south bridge 104 has a function of executing communication with the sound controller 106.

  The sound controller 106 is a sound source device, and outputs audio data to be reproduced to the speakers 18A and 18B or the HDMI control circuit 3.

  The LAN controller 108 is a wired communication device that executes IEEE 802.3 standard wired communication, for example, while the wireless LAN controller 112 is a wireless communication device that executes IEEE 802.11g standard wireless communication, for example. In addition, the USB controller 113 executes communication with an external device (connected via the USB connector 19) compatible with, for example, the USB 2.0 standard. For example, communication is performed when a digital image managed by a digital camera that is an external device is captured and stored in the HDD 109.

  The EC / KBC 113 is a one-chip microcomputer in which an embedded controller for power management and a keyboard controller for controlling the keyboard 13 and the touch pad 16 are integrated. The EC / KBC 114 has a function of turning on / off the computer 10 in accordance with the operation of the power button 14 by the user.

  Next, a functional configuration of the content reproduction application program 202 operating on the computer 10 having the above configuration will be described with reference to FIG. Here, a configuration example for realizing a function of displaying a digital image (still image data 401) stored in the HDD 109 like a so-called digital photo frame among the functions of the content reproduction application program 202 described above will be described. To do.

  As shown in FIG. 3, the content reproduction application program 202 includes an indexing processing unit 301 and a slide show display control unit 302.

  The indexing processing unit 301 executes various processes related to indexing for creating index information for searching for a target digital image from the still image data 401 stored in the HDD 109. For example, a face image is extracted from still image data 401 containing a person. This face image extraction is executed by, for example, a face detection process for detecting a face area from the still image data 401 and a cutout process for cutting out the detected face area from the still image data 401. The detection of the face area can be performed, for example, by analyzing the characteristics of the still image data 401 and searching for an area having a feature similar to a face image feature sample prepared in advance. The face image feature sample is feature data obtained by statistically processing the face image features of a large number of persons. Note that the still image data 401 illustrated in FIG. 3 may be an image of each of a plurality of frames constituting the moving image data.

  The result of the indexing process by the indexing processing unit 301 is stored as index information 402 in the database 109A. The database 109 </ b> A is a storage area prepared in the HDD 109 for storing the index information 402. FIG. 4 shows a configuration example of the index information 402 stored in the database 109A.

  The “image ID” is identification information uniquely assigned to each still image 401, and the “imaging date / time” is time information indicating the imaging date / time of each still image 401. When the still image data 401 is an image of each of a plurality of frames constituting the moving image data, for example, a time stamp indicating an elapsed time from the first frame in the “image ID” field + “imaging date / time” field. Information is stored.

  “Face image information” is information recorded for the number of people when the still image data 401 includes a person. As the “face image information”, first, each face image extracted from the still image data 401 by the indexing processing unit 301 is stored. In addition, the indexing processing unit 301 outputs the frontality (representing the degree to which the face was photographed from the front) and size (the size of the face image area in the original still image) of each face image extracted from the still image data 401. To do. Further, the indexing processing unit 301 classifies the plurality of face images extracted from the still image data 401, that is, classifies them according to images estimated to be the same person, and outputs the classification result as classification information. The degree of frontality, size, and classification information are further stored as “face image information”.

  In addition, the indexing processing unit 301 determines whether or not any character such as a character on a signboard is captured for each still image data 401 and outputs the determination result. This determination result is stored as “character information”. Character detection can be performed by searching for a region having a feature similar to a feature sample of each character prepared in advance.

  Further, the indexing processing unit 301 groups a plurality of still image data 401 stored in the HDD 109 based on, for example, the imaging date and time, and outputs information for identifying each group. This information is stored as “group information”. For example, when a predetermined period is exceeded between two still image data 401 whose imaging dates and times are continuous in time, grouping is performed by separating the period. Further, when the still image data 401 is an image of each of a plurality of frames constituting the moving image data, for example, a so-called scene change portion where the image characteristics change greatly before and after is detected, and each scene is divided into one section. Grouping.

  That is, according to the index information 402, for each still image data 401, whether or not a face image is included, that is, whether a person is captured, who is captured, whether a character is captured, which group Can be known. If the viewpoint is changed, using this index information 402, still image data 401 that includes the target person, still image data 401 that does not include the person, the target person, and characters are captured. The still image data 401 and the like can be quickly searched from a plurality of still image data 401 stored in the HDD 109.

  Then, the slide show display control unit 302 uses the index information 402 created by the indexing processing unit 301 to select still image data 401 that satisfies a predetermined selection condition from among a plurality of still image data 401 stored in the HDD 109. Then, display processing for sequentially displaying is performed like a so-called digital photo frame. Hereinafter, the operation principle of the slide show display control unit 302 will be described in detail. In the present embodiment, displaying a plurality of still image data 401 sequentially at a predetermined interval is referred to as a slide show. Moreover, not only a simple sequential display but also a display with a processed display such as a transition effect at the time of switching the display (transition effect) may be included.

  The slide show display control unit 302 includes a user interface unit 3021. The user interface unit 3021 is used to display a person designation screen for slide show display shown in FIG.

  As shown in FIG. 5, a face list display area a1 is provided on the person designation screen. The slide show display control unit 302 uses the index information 402 stored in the database 109A to select one face image of a person shown in at least one of the plurality of still image data 401 stored in the HDD 109 one by one. Arranged on the face list display area a1. As shown in FIG. 4, since the index information 402 includes frontality, size, and classification information, the slide show display control unit 302 performs, for example, each face image with the same classification information, for example, Among those having a size equal to or larger than the threshold value, the one having the highest degree of frontality is selected. The face image to be placed on the face list display area a1 can be switched by operating the keyboard 13, the touch pad 16, or the like.

  The face list display area a1 is used for the user to designate an arbitrary person as search key information for selecting still image data 401 to be displayed as a slide show from a plurality of still image data 401 stored in the HDD 109. It is an area. Now, it is assumed that the user wants to display a still image 401 related to the person A as a slide show from the persons A to H where the face images are arranged on the face list display area a1. In this case, the user selects the face image a11 of the person A arranged on the face list display area a1.

  In addition, a “slide show start” button a2 and a “detail setting” button a3 are provided on the person designation screen. The “slide show start” button a2 is a button for instructing to start a slide show after selecting any face image on the face list display area a1, and the “detail setting” button a3 is a face list display. This is a button for setting an expansion condition related to at least one of a selection rule for the still image 401 related to the person of the face image selected on the area a1 and a display rule for the selected still image 401. When the “detail setting” button a3 is operated, the slide show display control unit 302 displays a detailed setting screen for slide show display shown in FIG. 6 on the LCD 17 by using the user interface unit 3021.

  As shown in FIG. 6, a scene list display area b1 is provided on the detailed setting screen. On this scene list display area b1, for example, when “Travel” is selected, a large number of landscape images (without a person) are selected, and the landscape image is first displayed so that it can be seen where it was taken. If you display it before other images (without being captured in the order of shooting) or select “Party”, you will select the image so that all participants will be covered, and so that you can see which party first Set extended conditions for selection rules and display rules suitable for each scene, such as displaying images with the largest number of people in front of other images (without being captured in the shooting order). can do. Here, it is assumed that the “detailed setting” button a3 is operated on the person designation screen shown in FIG. 5 and “travel” (icon b11) is selected on the detailed setting screen.

  The detailed setting screen having the scene list display area b1 as described above is provided with a “person designation” button b2 for returning to the person designation screen shown in FIG. Therefore, on the person designation screen shown in FIG. 5, first, the “detailed setting” button a3 is operated to display the detailed setting screen, the extended condition is set, and then the “person designated” button b2 is operated. Then, it is possible to return to the person designation screen shown in FIG. 5 and perform a procedure of selecting a person.

  The detailed setting screen is also provided with a “slide show start” button b3, similar to the person designation screen shown in FIG. 5, so that the slide show can be started without returning to the person designation screen shown in FIG. Can be directed.

  Next, the slide show display control unit 302 that has received an operation on the person designation screen illustrated in FIG. 5 and the detailed setting screen illustrated in FIG. 6 selects the still image data 401 to be displayed, The basic principle of how the selected still image data 401 is displayed will be described with reference to FIG.

  7A, a plurality of still image data 401 stored in the HDD 109 are arranged in time series in order of imaging date and time and grouped based on the index information 402 stored in the database 109A. It is the 1st conceptual diagram which shows a mode that it is. As shown in FIG. 7A, here, it is assumed that a plurality of still image data 401 stored in the HDD 109 is divided into three groups, group n, group n + 1, and group n + 2. Each box with numbers such as 1, 2, 3,... In each group represents one still image data 401, and the number represents the order of imaging date and time in each group. Yes.

  In addition, a circle in the box of the still image data 401 indicated by reference sign c1 represents that the still image data 401 is an image in which a person designated on the person designation screen shown in FIG. The triangle in the box of the still image data 401 indicated by reference numeral c2 indicates that the still image data 401 does not show the person designated on the person designation screen shown in FIG. 5, but shows any other person. The rhombus in the box of the still image data 401 indicated by reference numeral c3 indicates that the still image data 401 is an image in which no person is shown.

  That is, FIG. 7A shows a group including still image data 401 in which a person designated on the person designation screen shown in FIG. 5 is shown among three groups, group n, group n + 1, and group n + 2. Represents two groups, group n and group n + 2. Therefore, the slide show display control unit 302 first narrows down the still image data 401 in the group n and group n + 2 as a display target (selection of display target group).

  The slide show display control unit 302 determines that the still image data 401 belonging to the same group as the still image data 401 in which the person designated on the person designation screen shown in FIG. It is determined that the image is related to the designated person. The slide show display control unit 302 selects the still image data 401 based on the index information 402 stored in the database 109A, for example, with a predetermined number as the upper limit, with the selected display target group as a range. At this time, for example, on the detailed setting screen shown in FIG. 6, for example, “travel”, in principle, the still image data 401 in which the person designated on the person designation screen shown in FIG. ”Is selected, it is more than just selecting an image that includes the person, such as selecting a large number of landscape images (not including a person). To do.

  7B is a second conceptual diagram illustrating a selection result of the still image data 401 by the slide show display control unit 302. FIG. As shown in FIG. 7B, in the group n, the person designated on the person designation screen shown in FIG. 5 indicated by reference numeral d1 is not shown, but any other person is shown. An image is selected, and an image indicated by reference sign d2 in which no person is shown is also selected. Similarly, in the group n + 2, the person designated on the person designation screen shown in FIG. 5 indicated by the symbol d3 is not shown, but an image showing any other person is selected. In addition, an image indicated by reference sign d4 and not showing any person is also selected.

  The images indicated by the symbols d1 to d4 are images that cannot be selected when a person is designated as search key information in the conventional image search method. In this regard, the content reproduction application program 202 is, for example, In order to display a slide show so that “where” and “who” can be understood, an effective image search with a high degree of intelligence is executed.

  7C is a third conceptual diagram illustrating the display order of the still image data 401 by the slide show display control unit 302. FIG. As shown in FIG. 7C, the slide show display control unit 302 configures an array for guiding from the present to the past in a group unit (group n + 2) in order to satisfy the request to view the latest image first. → Group n) The images in each group are arranged so that they can be enjoyed more naturally. That is, different algorithms are used for the arrangement of groups and the arrangement of images in each group.

  As described above, since it is assumed here that “travel” is selected on the detailed setting screen shown in FIG. 6, the slide show display control unit 302 can first understand “where”, for example. In addition, for the group n + 2, an image indicated by reference sign d4 in which no person is captured, that is, an image that is presumed to be a landscape image is arranged at the top, and thereafter, the other images are arranged in the order of the imaging date. Similarly, for the group n, an image that is assumed to be a landscape image indicated by reference sign d2 is placed at the top, and other images are arranged in the order of the imaging date thereafter.

  That is, the content reproduction application program 202 also executes effective image display with high intelligence, such as first arranging an image suitable for grasping a scene such as “travel” as a representative image. Note that if the expansion condition is not set on the detailed setting screen shown in FIG. 6, the images in each group may be arranged in order of the shooting date and time.

  FIG. 8 is a flowchart showing the procedure of a slide show display process executed by the content reproduction application program 202.

  First, the content reproduction application 202 uses the index information 402 stored in the database 109 </ b> A in the HDD 109 to select a facial image of a person shown in at least one of the plurality of still image data 401 stored in the HDD 109. A list is displayed as options (step A1).

  When any one of the face images displayed in the list is selected (step A2), the content reproduction application program 202 uses the index information 402 stored in the database 109A in the HDD 109 to select the selected face image. A group including still image data 401 in which a person of a face image is captured is specified (step A3). Then, still image data 401 to be displayed is selected within the specified group as a range (step A4).

  Further, the content reproduction application program 202 further determines the display order of the selected still image data 401 by using the index information 402 stored in the database 109A in the HDD 109 (step A5), and the determined display According to the order, the selected still image data 401 is sequentially displayed on the LCD 17 (step A6).

  As described above, according to the computer 10, not only the image group that can be directly detected from the designated search key information but also the image having a predetermined relationship with the search key information is selected. In addition, it is possible to realize image search and image display with a high degree of intelligence, such as arranging images so that they can be viewed more happily rather than simple rules such as shooting date order.

  Here, as a method for selecting the still image data 401 to be displayed from the still image data 401 stored in the HDD 109 using the index information 402 stored in the database 109A in the HDD 109, the still image data 401 is displayed. In the above example, the face images of the persons in the list are displayed as options. However, the usage of the index information 402 (for selecting the still image data 401 to be displayed from the still image data 401 stored in the HDD 109) is not limited to this, and various modifications are possible. is there.

  For example, a table associating face image classification information with person names may be stored in the database 109A as index information 402, and the person names may be displayed as a list. In order to manage this table, user interface means for displaying a list of face images and inputting names may be provided.

  Further, for example, instead of specifying a face image or name (for specifying a target person) as search key information, year / month / day is specified as search key information, and monthly group information is index information. 402, and select not only the still image data 401 of the imaging date and time matching the specified date but also the still image data 401 of the imaging date and time of the same month in the same year or different year as relevant. You may make it display.

  As described above, the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, you may combine a component suitably in different embodiment.

  DESCRIPTION OF SYMBOLS 1 ... External display apparatus, 2 ... HDMI terminal, 3 ... HDMI control circuit, 10 ... Personal computer, 11 ... Computer main body, 12 ... Display unit, 13 ... Keyboard, 14 ... Power button, 15 ... Input operation panel, 16 ... Touch Pad, 17 ... LCD, 18A, 18B ... Speaker, 19 ... USB connector, 20 ... Remote control unit interface, 101 ... CPU, 102 ... North bridge, 103 ... Main memory, 104 ... South bridge, 105 ... GPU, 106 ... Sound 107, BIOS-ROM, 108 ... LAN controller, 109 ... HDD, 109A ... database, 110 ... ODD, 111 ... USB controller, 112 ... Wireless LAN controller, 113 ... EC / KBC, 114 ... EEPR M, 201 ... operating system, 202 ... content playback application program, 301 ... indexing process section, 302 ... slideshow control unit, 401 ... still image data, 402 ... index information, 3021 ... user interface unit.

Claims (8)

A setting means for setting an image selection rule and an image display rule based on a keyword selected from the keywords presented as options;
An information for searching the target image, based on the index information which can determine an imaging existence of imaging the presence and character of a person in each image, and set selection rule and display rule by the setting unit, the display Display control means for selecting a target image and arranging and controlling the selected images;
An electronic device comprising:   The electronic device according to claim 1, wherein when the first keyword is selected, the setting unit sets a display rule for first displaying an image in which a person is not captured.   The electronic device according to claim 1, wherein when the second keyword is selected, the setting unit sets a display rule that first displays an image in which characters are captured.   The electronic device according to claim 1, wherein when the second keyword is selected, the setting unit sets a display rule that first displays an image in which the largest number of people are captured. The setting means, the electronic device according to claim 1, wherein if the first keyword is selected, the selection ratio of an image in which the subject's captured to set a higher selection rules. The setting means, when the second keyword is selected, the electronic device according to claim 1, wherein for setting the selection rules for selecting the image to the person being imaged are covered. An image display method in an electronic device,
Setting image selection rules and image display rules based on keywords selected from the keywords presented as options,
Is information for searching a target image, based on the imaging existence of imaging the presence and character of a person in each image and possible index information checking, and the set selection rule and display rule, and displayed Selecting an image, arranging the selected image and controlling the display;
An image display method comprising: Computer
A setting means for setting an image selection rule and an image display rule based on a keyword selected from keywords presented as options;
An information for searching the target image, based on the index information which can determine an imaging existence of imaging the presence and character of a person in each image, and set selection rule and display rule by the setting unit, image Display control means for arranging and controlling display,
Content playback program to function as.

Images