JP2011175427A - Portable telephone device with image browsing function - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a portable telephone device having a function for changing a presentation image to a user based on the situations or past browsing history of a user with respect to large amounts of accumulated camera photographic images. <P>SOLUTION: A portable telephone device having an image display function includes: an image data analysis means for analyzing the features of image data from the image data; a history accumulation means for accumulating the features of the image data output from the image data analysis means; and an image display control part for, on the basis of the situations of a user, selecting the image data suitable to the situations of the user from the features of the image data accumulated in the history accumulation means. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a mobile phone device with an image browsing function and an image browsing method.

  In recent years, the opportunity to view image data on mobile phone devices has increased greatly, and the number of files such as image data to be recorded has increased more and more as the capacity of external memory such as SD cards has increased. Yes. Furthermore, it is predicted that the use of centralized management of images on PCs and network services (servers) will increase as the mobile phone network speeds up and the wireless LAN function is installed. It is considered that a large amount of images are accumulated.

  Until now, as a method of processing a large amount of image data and browsing with a mobile phone device, for example, a method described in Patent Document 1 has been devised. This is characterized in that image data with a smaller amount of data (low resolution) is selectively displayed according to the user's situation. In addition, a method as described in Patent Document 2 has been devised.

Japanese Patent Laid-Open No. 2002-158982 JP 2007-226555 A

  However, when the user and his / her family / friends browse the stored image data using the conventional method, if there is a large amount of image data, all the image data is displayed on the limited display screen of the mobile phone. It is difficult to browse at a time.

  For this reason, for example, a display method such as “display 10 images taken recently” must be used. However, the display method is always unpleasant for the user, and in random reproduction such as a digital photo frame, if an image to be displayed is not selected, a submerged image is displayed, which is not preferable.

In summary, it can be said that the conventional image browsing methods have the following problems.
(1) When there is a large amount of accessible image data, the user cannot view images at a time on a limited screen of the mobile phone.
(2) Since the image that the user wants to see changes depending on the user's situation, it is difficult to present an appropriate image to the user.

  The present invention has been made in view of the above problems, and realizes a function for extracting an image to be presented to a user based on image characteristics, access history, user status, and the like for image data. With the goal.

  The present invention relates to an image data analysis means for analyzing features of image data from image data and a history storage for storing features of image data output from the image data analysis means in a cellular phone device having an image display function. And an image display control unit for selecting image data suitable for the user's situation from the characteristics of the image data stored in the history storage unit based on the user's situation.

  The present invention also includes a step of analyzing features of the image data from the image data, a step of storing the features of the analyzed image data, and the stored image data based on a user's situation. An image browsing method comprising a step of selecting image data suitable for a user's situation from characteristics and a step of displaying an image of the selected image data.

  The first effect is to realize browsing display more suitable for the user in consideration of the situation of the user when there is a large amount of accessible image data.

  The second effect is to realize changing the image to be displayed in accordance with the user's situation.

1 shows an embodiment of a mobile phone unit according to the present invention. The functional block diagram of the mobile telephone apparatus part of FIG. 1 is shown. An example of the history data stored in the history storage unit is shown. 7 shows a flowchart when an image viewer is executed after an e-mail function is operated in an embodiment of the present invention. The figure explaining the initial image in the case of displaying an image according to a plurality of history data is shown.

  Hereinafter, an embodiment of a mobile phone device of the present invention will be described in detail with reference to the drawings.

  FIG. 1 shows a mobile phone unit 1 according to an embodiment of the present invention, which has communication means with a mobile phone network and a camera function. FIG. 2 shows a functional block diagram of the mobile phone unit 1.

  As shown in FIG. 2, the cellular phone device unit 1 includes an image data analysis unit 21, an image display control unit 22, a history storage unit 23, a wireless processing unit 24, an image display unit 25, and a control unit 26. Hereinafter, the function of each part will be described.

  The image data analysis unit 21 analyzes the characteristics of image data such as an image taken by the camera function of the mobile phone unit 1 and stores the output result in the history storage unit 23 via the control unit 26. Here, the image data analysis unit 21 performs shooting date, shooting mode, number of continuous shots, similarity detection with other images, representative image determination, subject information (photographing scene automatic determination result, object automatic recognition result, etc.), shooting The location is analyzed and output as image features.

  The following information can be used as other features of the image.

  Shooting image information (information base recorded in EXIF information, etc.) includes shooting date, shooting time (band), shooting season, GPS position information, shot image size, setting (flash, shooting mode, white balance, exposure control) , Sharpness, contrast, sensitivity, etc.). Here, from what was changed from the default, such as white balance and exposure, or was taken while changing various settings in a continuous period of time, the user's intention of "I want to take beautiful pictures" is inferred. it can.

The shooting setting information includes the number of continuous shots and the interval between continuous shots.
In addition, the photographed image analysis information includes automatic scene determination results (landscape, night view, person, memo information, etc.), personal recognition information (registered Mr. A), automatic object recognition (pet, building, etc.), best shot detection , Detection of submerged images (blur / blur), detection of similarity, determination of representative images from continuous shot images, detection of daily / unusual shots, data recognition (characters / barcodes) in captured images, and the like.
User operation information includes folder name, tag information, user editing information, user operation information (enlarged display, attached to e-mail, blog posting, schedule link, etc.), display count, viewing time, and the like.

  Next, when the image display control unit 22 receives a display image extraction request from the image display unit 25, the image display control unit 22 is based on the situation of the user of the day (emotion expressed in the email, the characteristics of the image taken on that day, etc.). In addition, it has a function of inquiring the history storage unit 23 for an image that is considered to be more preferable for the user. Then, an appropriate number of images are selected from the extracted images, and the access destination and file name information are returned to the image display unit 25.

  In addition to the above user situations, behavior estimation based on time zone and history (ON time, OFF time), emotions (emotion understanding of the email body), number of email transmissions / number of characters (from historical history statistical data) Change information), position information (travel destination, whereabouts, etc.), schedule information (travel, events, people), etc. can be used.

  The history storage unit 23 stores the characteristics of the image data described above, the operation history of the user at the time of display, the presence or absence of processing by the user of the image data, and the history of attachment to e-mails and blogs. In response to an inquiry from the image display control unit 22 based on the history data. The accumulated history data may be data for each piece of image data, or may be data statistically processed with respect to the shooting date and shooting scene. An example of the history data is shown in FIG.

  As the history data, an operation history can be used in addition to the image data described above. The operation history can be obtained by performing statistical processing on what action has occurred after browsing a certain image. For example, from the data you searched for, zoomed in, played in a slide show, attached to an email or blog, set as standby, edited, sent by infrared, printed, last access date and time, etc. A table is created based on the location information and the analyzed image information. At that time, in order to reduce the amount of statistical data, for example, it is better to classify or score broadly, such as “actions related to communication with people” and “individual operations”.

  The wireless processing unit 24 is a part that performs communication processing with the mobile phone network.

  The image display unit 25 provides an image viewer function for the user, such as presenting an image on an LCD screen (Liquid Crystal Display).

  The control unit 26 is a part that controls the mobile phone unit 1.

  In the description of the features of the present invention, camera functions (such as shooting and storage) and image display functions are not directly related, and thus detailed configurations thereof are omitted.

  By adopting the configuration and information processing method as described above, for image data accumulated in large quantities, information such as the characteristics of images taken by the user on that day, information such as keywords / emotions in schedule / email, images A function of extracting an image to be presented to the user can be realized based on information such as past operation history for data and characteristics of image data. For example, when accessing stored image data in a mobile phone device, an image from the previous year that is close to the image taken on that day is presented, or a keyword (location name, person name, etc.) entered in the user's schedule, It is possible to change the image to be presented according to the emotion expressed in e-mails.

  In addition, when searching for an image, the burden on the CPU and memory can be further reduced by providing a plurality of search history data. For example, in the initial search stage, the number of images to be searched is set to 300, and in the second stage, a maximum of 1000 images is searched. For detailed search, all the images with priority given to the folder in which the file is stored and the previous and subsequent time stamp images Search.

  Hereinafter, an example of image display using the plurality of history data will be described more specifically.

  First, a plurality of history data are provided. For example, data A: link information for 30 representative images for each shooting month, link information for 30 representative images for each shooting date, link information for 30 representative images for each shooting time period, data B: shooting scene Link information for 30 representative images for each, link information for 30 representative images for each subject (person / thing), data C: link information for 30 representative images connected to communication (email, etc.), extraordinary It is divided into a plurality of link information for 30 representative images for each month of image capture. That is, as an initial stage of image search, first, the number of images is limited to 30 in each situation.

  Then, the initial display image is determined according to the situation of the user on the day of browsing the image. For example, the relationship between the user's situation and the initial display image is as follows: a) a lot of photographs; initial display of today's representative image of data A; b) a lot of images with cherry characteristics; It is possible to effectively display a limited number of initial images, such as initial display of representative images, c) lots of fun e-mails exchanged; it can. In addition, when the initial image is displayed, as shown on the left side of FIG. 5, only nine images a) can be displayed as thumbnails, and as shown on the right side, a) It is also possible to display three thumbnail images of each of the representative images in b) and c). Furthermore, the displayed thumbnail image can be noticed by the user by switching to another image in the same group (30 images) at regular intervals.

In this way, the user can quickly select a favorite image from the limited and optimized image group, and even if the initial display image is dissatisfied, more images are stocked. It is possible to smoothly enter the feature category of each image (ie, the second stage described above).
As described above, by subdividing the history data for each category, it is possible to narrow down the target data to be initially searched, thereby reducing the calculation amount of the CPU. Also, by monitoring user operations on the data, it is possible to delete only historical data that is not of interest to the user, thereby making it possible to reduce the overall memory capacity.

  Furthermore, history information of user operations (for example, browsing for a long time, sending mail, processing, etc.) can be used in the database. That is, history data with low user interest can be deleted, or conversely, when the user uses it, conditions close to it can be learned and utilized when generating history data and selecting a representative image.

  Further, when selecting the above image, it is possible to reduce the burden by reducing the number of similar pictures (subjects, compositions).

  Hereinafter, an example of the present invention will be described based on the embodiment described above and FIGS. 2 and 3. In this embodiment, the image viewer is executed after the mail function is operated, and a flowchart thereof is shown in FIG. However, the present invention is not limited to this embodiment.

  In FIG. 4, first, S <b> 41 indicates that the user has activated the image viewer of the mobile phone device unit 1. Thereby, the image display unit 25 inquires of the image display control unit 22 about an image to be displayed.

  Next, the process proceeds to a display image extraction step shown in S42. Here, the image display control unit 22 sets the conditions of an image that seems to better match the user's propensity, inquires the history storage unit 23, extracts the display image, and displays the result as an image display. Return to section 25. For example, it is possible to extract an image taken within 3 months to 2 years having characteristics similar to the image taken on that day, and the emotion expressed in the user's email etc. In the case of “fun”, it is possible to extract an image associated with the information or an image such as a trip or a group photo of people. In addition, when there are many hits of an image, it can also display from the thing with a high score.

  Next, the process proceeds to the step shown in S43, where the image display unit 25 displays the image extracted as described above on the viewer.

  Next, as shown in S44, the process proceeds to a step where the user actually operates. Here, the user performs some operation on the image data displayed on the image display unit 25.

  Next, in the history addition step of S45, the user's operation is additionally registered in the history storage unit 23. Here, the history storage unit 23 records the contents operated by the user and recalculates the score. As an example of the score, the user's priority (attachment) to the image is expressed by adding +100 points when the user enlarges and displays certain image data, and +300 points when attaching to the mail. Make it possible.

  Then, the series of processing ends (S46).

  Although the embodiment has been described in detail above, the configuration of the mobile phone device itself is well known to those skilled in the art and is not directly related to the present invention, and therefore the details thereof are omitted.

  As another embodiment of the present invention, the configuration is basically the same as the above, but the following examples are also possible.

  Application to not only images stored in the mobile phone device but also image data existing on a system that can be accessed from the mobile phone device such as a PC or a server on a network is conceivable. That is, the user can access and operate the image data stored in the external device via the communication network, and can perform the processing based on the operation history as described above. At this time, it is assumed that data will be shared with people other than the user (family, friends, etc.), so data on the operation history other than the user should also be accumulated and used for priority (score) evaluation. You can also.

A part or all of the above-described embodiment can be described as in the following supplementary notes, but is not limited thereto.
(Appendix 1) In a mobile phone device having an image display function, image data analysis means for analyzing features of image data from image data, and history storage for storing features of image data output from the image data analysis means And a display unit for selecting image data suitable for the user's situation from the features of the image data stored in the history storage unit based on the user's situation. Telephone device.

(Additional remark 2) The said log | history storage means further accumulate | stores one or more among the operation history of the user at the time of displaying an image, the presence or absence of the image data processing by a user, and the log | history of the attachment of an image by a user The mobile phone device as set forth in appendix 1, wherein

(Supplementary note 3) The mobile phone according to Supplementary note 1 or 2, wherein the feature of the image data is one or more of photographed image information, photographing setting information, photographed image analysis information, and user operation status. Telephone device.

(Additional remark 4) The said user's situation is one or more of the characteristic of the image image | photographed on the day, the emotion or keyword expressed in the schedule and the mail, or position information. 4. The mobile phone device according to any one of items 1 to 3.

(Supplementary note 5) The mobile phone device according to any one of supplementary notes 1 to 4, wherein the image data is stored in the mobile phone device.

(Appendix 6) The mobile phone according to any one of appendices 1 to 4, wherein the image data is stored in an external device connected to the mobile phone device via a wired, wireless, or mobile phone network. apparatus.

(Supplementary note 7) The history storage means includes at least one of an operation history by a third party other than a user in the image data stored in the external device, presence / absence of image data processing, and image attachment history. The mobile phone device according to appendix 6, wherein the mobile phone device is further stored.

(Supplementary Note 8) From the characteristics of the accumulated image data based on the process of analyzing the characteristics of the image data from the image data, the process of accumulating the characteristics of the analyzed image data, and the situation of the user An image browsing method, comprising: selecting image data suitable for a user's situation; and displaying an image of the selected image data.

(Supplementary note 9) The image browsing method according to supplementary note 8, wherein in the step of accumulating the image data, the image data is accumulated in a plurality of categories.

  The present invention can be applied to a mobile phone device with an image browsing function. In addition, the present invention can be applied to PDAs and other small communication terminals.

Claims (9)

In a mobile phone device having an image display function,
Image data analysis means for analyzing the characteristics of the image data from the image data;
History storage means for storing the characteristics of the image data output from the image data analysis means;
An image display control unit that selects image data suitable for the user's situation from the characteristics of the image data stored in the history storage unit, based on the user's situation;
A mobile phone device characterized by comprising:   The history storage means further stores at least one of a user operation history when displaying an image, presence / absence of image data processing by the user, and a history of image attachment by the user. The mobile phone device according to claim 1.   The mobile phone device according to claim 1 or 2, wherein the feature of the image data is one or more of photographed image information, photographing setting information, photographed image analysis information, and user operation status.   The situation of the user is one or more of a feature of an image photographed on the day, an emotion or a keyword expressed in a schedule or an email, or position information. The mobile phone device according to any one of the above.   5. The mobile phone device according to claim 1, wherein the image data is stored in a mobile phone device.   5. The mobile phone device according to claim 1, wherein the image data is stored in an external device connected to the mobile phone device via a wired, wireless, or mobile phone network.   The history accumulating unit further accumulates one or more of an operation history by a third party other than a user in the image data stored in the external device, presence / absence of image data processing, and image attachment history. The mobile phone device according to claim 6. Analyzing the characteristics of the image data from the image data;
Storing the characteristics of the analyzed image data;
Selecting image data suitable for the user's situation from the characteristics of the accumulated image data based on the user's situation;
Displaying an image of the selected image data;
An image browsing method.   The image browsing method according to claim 8, wherein in the step of storing the image data, the image data is stored in a plurality of categories.

Images