JP5668025B2 - Mobile terminal device - Google Patents

Description

  The present invention relates to a mobile terminal device capable of collecting life logs.

  Realized with mobile terminal devices such as smartphones and tablet PCs (personal computers) through the development of input interfaces such as touch panels and the development of large-capacity and miniaturized recording media, improved wireless communication technology, and maintenance of wireless communication facilities The functions that can be performed are greatly improved, and the penetration rate is high.

  Providing new services with a view to the situation of such portable terminal devices has also started. For example, based on the assumption that the current location of the mobile terminal device is the same as the current location of the user, a location information linkage SNS (Social Networking Service) that can share information and communicate with the current location, etc. Is provided. In addition, users such as a service that distributes advertisement information according to the user, a service that distributes coupons according to the current location information, etc., based on the browsing history of the website by the user and keywords input for searching Concierge services that provide various services in accordance with the lifestyles of people have started.

  In providing such a service, in order to determine a service to be provided, it is important to obtain a life log obtained by converting the action content of the user into digital data.

  The information grouping system described in Patent Document 1 is a system that adds a group of state information related to an individual's life and behavior, and additionally writes the state information transmitted by the captured device when the user operates the capture device. A group ID is associated with the group and added to the storage means.

JP 2011-76454 A

  The information grouping system described in Patent Document 1 attempts to record state history by grouping state information by ID. In this information grouping system, the status information transmitted by each capture target device when the capture device is operated by the ID can be related to each other as one group, but a plurality of information is regarded as one group. It is only associated with each other, and the stored life log cannot be fully utilized.

  The objective of this invention is providing the portable terminal device which can fully utilize the memorize | stored life log.

The present invention includes an acquisition unit that acquires position-related information relating to the current position of the operation-related information and own apparatus associated with the operation state of the apparatus including the standby state,
Storage means for storing the motion-related information and the position-related information;
By the user's operation, an instruction unit that instructs the acquisition unit acquires the motion related information and the location-related information,
It said instruction means detects the time when the instruction to the acquisition means, and the detected time, in association with the motion related information and the location-related information obtained by the obtaining unit by the instruction control to be stored in said storage means Means,
Log output means for outputting log information based on the motion-related information and the position-related information stored in the storage means in the order of associated times;
A mobile terminal device comprising: map display means for displaying the motion-related information and the position-related information stored in the storage means on a map based on the position-related information .

The present invention also provides a display means for displaying an image,
Voice output means for outputting voice,
The operation related information includes application information related to the activated application software, image information related to the image displayed on the display means, and audio information related to the sound output by the output means. It is characterized by including.

The present invention further includes a communication means for communicating with another device,
The position related information includes longitude and latitude information indicating the position of the own device in longitude and latitude, and connection information related to a connection state by wireless communication with another device.

The present invention further includes a number of times detection means for detecting the number of times the instruction means is operated,
The control means associates the time with emotion information representing a user's emotion according to the number of times detected by the number of times detection means, and stores the information in the storage means.

According to the present invention, the acquisition means acquires the position-related information relating to the current position of the operation-related information and own apparatus associated with the operation state of the apparatus including the standby state, the storage means, said motion related information And storing the position-related information.
The log output means outputs log information based on at least one of the motion related information and the position related information stored in the storage means in the order of the associated times. Thereby, the life log can be displayed in a so-called timeline. The map display means displays the motion related information and the position related information stored in the storage means on a map based on the position related information.

By the user's operation, the instruction means the acquisition unit, when an instruction to acquire the operation related information and the location-related information, the control means detects the time when the instruction means instructs the acquisition unit, detection and time, in association with the motion related information and the location-related information obtained by the obtaining unit by the instruction is stored in the storage means.

By a user's instruction, the operation-related information and position-related information acquired by can be stored, it can be stored obtained time be associated with. For example, and outputs side by side in order of the time obtained lifelog an operation-related information and location-related information, a specific date and time, the portable terminal device at a specific time, where, what work was running The life log can be used in a wider range, such as outputting, and the stored life log can be fully utilized.

  According to the present invention, it further comprises display means for displaying an image and sound output means for outputting sound, wherein the operation related information includes application information related to activated application software, and display means. Image information related to the image displayed on the screen and sound information related to the sound output by the output means.

  Since image information and audio information can be stored as operation-related information, these information can be output as a life log.

  According to the present invention, the information processing apparatus further includes a communication unit that communicates with another device, the position related information includes longitude and latitude information indicating the position of the own device in longitude and latitude, and a connection state by wireless communication with the other device. Connection information related to.

  As position related information, longitude and latitude information and connection information can be stored, and not only direct position information such as longitude and latitude information but also indirect position information such as connection information is stored. be able to.

Further, according to the present invention, the apparatus further includes a number detection unit that detects the number of times the instruction unit is operated, and the control unit displays the time and the user's emotion according to the number of times detected by the number detection unit. The emotion information to be expressed is associated with and stored in the storage means.
Thereby, even a user's emotion when a life log is memorize | stored can be memorize | stored.

It is an external view which shows the structure of the portable apparatus 20 which is embodiment of this invention. 2 is a block diagram showing a configuration of a mobile device 20. FIG. 1 is a schematic diagram showing a communication system including a mobile device 20. It is a figure which shows an example of the data structure of log data. It is a schematic diagram which shows the output aspect of a life log. It is a figure which shows the example of the operation screen when performing a search process, and the transition of an operation screen.

  FIG. 1 is an external view showing a configuration of a mobile device 20 according to an embodiment of the present invention, and FIG. 2 is a block diagram showing a configuration of the mobile device 20.

  The mobile device 20 is configured by a mobile terminal device such as a smartphone or a tablet PC. Hereinafter, a case where the mobile device 20 is a smartphone will be described as an example.

  The portable device 20 includes a control unit 21, a storage unit 22, a temporary storage unit 23, an operation unit 24, a display unit 25, a communication unit 26, a speaker unit 27, a vibration unit 28, a microphone unit 30, a camera unit 31, and a log transmission unit 33. And it is comprised including the time measuring part which is not shown in figure.

  The control unit 21, which is a control unit and a computer, is configured by a CPU, for example, and by executing a program stored in the storage unit 22, the operation unit 24, the display unit 25, the communication unit 26, the speaker unit 27, and the vibration unit 28. The log transmission unit 32 and a timing unit (not shown) are controlled.

  The storage unit 22 as storage means is configured by a nonvolatile large-capacity storage device, for example, a semiconductor memory such as an SSD or a flash memory card, or an HDD. The storage unit 22 stores a program executed by the control unit 21 and information necessary for the control unit 21 to execute the program. Information stored in the storage unit 22 is written and read by the control unit 21. The storage unit 22 includes a log storage unit 35.

  The temporary storage unit 23 is configured by a volatile storage device having a writing speed and a reading speed higher than those of the storage unit 22, for example, a semiconductor memory such as a DRAM, and temporarily stores information necessary for the control unit 21 to execute a program. This is a work memory to be stored. Information stored in the temporary storage unit 23 is written and read by the control unit 21.

  The operation unit 24 includes a plurality of keys for inputting information such as letters, numbers, and symbols. The operation unit 24 sends information associated with the operated key to the control unit 21. The operation unit 24 may be a mechanical key or a soft key, and preferably includes a touch panel for realizing the soft key. The operation unit 24 includes a life log key A that is a feature of the present invention. Although details will be described later, when the user presses the life log key A, the operation related information which is various information related to the operation state of the mobile device 20 at that time or the position related to the current position of the mobile device 20 At least one of the related information is stored as life log information in the log storage unit 35 of the storage unit 22 each time. The life log information is stored in association with the time when the life log key A is pressed, the operation related information, and the position related information. In the present embodiment, the life log key A is a side key provided on the side surface of the casing of the mobile device 20, for example, as shown in FIG. The life log key A only needs to generate a trigger signal for storing life log information in the log storage unit 35 of the storage unit 22 when operated by the user. When the side key that is the life log key A is pressed and a key pressing signal is output, the key pressing signal itself is input to the control unit 21 as a trigger signal.

  When the trigger signal is input to the control unit 21, the control unit 21 as the operation related information, the application that is activated if activated, the image data that is displayed if it is displayed, and the image data that is displayed if it is output. Various types of information related to the operation of the own device at the time when a trigger signal is input, such as output audio data, is acquired and stored in the log storage unit 35 of the storage unit 22. When the trigger signal is input to the control unit 21, the control unit 21 detects, as position related information, the longitude and latitude output by the GPS receiving unit 29, and other devices that are connected if wirelessly connected. Is acquired and stored in the log storage unit 35 of the storage unit 22. At this time, the control unit 21 also acquires the time when the trigger signal is input, and stores it in the log storage unit 35 of the storage unit 22 as a life log in association with the motion related information and the position related information.

  The life log key other than the side key may be a small wireless key including a mechanical key serving as the life log key A and a communication unit. When the life log key A is pressed, a key press signal indicating that the life log key A is pressed is transmitted from the communication unit, and when the communication unit 26 receives this, a trigger signal for storing life log information is generated. Generated and output to the control unit 21.

  The life log key A may be a soft key that is a combination of the touch panel and the display unit 25. When a key image of a life log key is displayed on the display unit 25 and an area of the touch panel corresponding to the key image is tapped, a coordinate signal output from the touch panel is input to the control unit 21 as a trigger signal.

  The display unit 25 which is a display unit is configured by a display device such as a liquid crystal display. The display unit 25 displays an image based on the image data generated by the control unit 21. Image data can be generated by an OS (Operating System) or by an application. An image that can be displayed on the display unit 25 based on the image data stored in the storage unit 22 and according to the specifications of the display unit 25. Data is generated.

  The communication unit 26 is connected to a network such as a mobile communication network via wireless base station communication equipment, and other devices connected to the network, such as a content distribution server, an e-mail server, an SNS server, a bulletin board It is composed of a communication device that transmits and receives various information to and from a server, log collection server, and the like. It is also possible to connect to a mobile communication network by the communication unit 26 and perform telephone communication with other telephone devices.

  The communication unit 26 transmits information received from the control unit 21 to another device instructed by the control unit 21 via the network, and transmits information received from the other device to the control unit 21 via the network. The communication system for connection with the mobile communication network is the third generation mobile phone system (abbreviation: 3G), but data is transmitted via various networks such as the Internet, LAN (Local Area Network), VPN (Virtual Private Network), etc. In order to perform communication, medium-distance and short-distance wireless communication is also possible. For example, a communication method such as Wi-Fi (registered trademark), Bluetooth (registered trademark), or Zigbee (registered trademark) may be used.

  The speaker part 27 is comprised by the audio | voice output apparatus containing a speaker, for example. The speaker unit 27 outputs sound based on the sound data generated by the control unit 21. The sound data can handle various sound data such as music data and synthesized sound data. During telephone communication, voice based on voice data received from the other party's telephone device is output. The vibration unit 28 is configured by, for example, a vibration device such as a vibrator, and notifies the user together with or instead of voice by so-called vibration.

  The GPS receiving unit 29 is composed of a GPS receiver corresponding to a user segment of the Global Positioning System, and it is only necessary to be able to measure at least latitude and longitude, and performs three-dimensional positioning including altitude, You can also get

  The microphone part 30 is comprised by the audio | voice input apparatus containing a microphone, for example. The microphone unit 30 outputs the input voice as an electrical signal to the control unit 21, and the control unit 21 converts it into voice data. The converted voice data is stored in the storage unit 22, thereby realizing a voice recorder function. Further, during telephone communication, the voice input from the microphone unit 30 can be transmitted as voice data to the other party's telephone device.

  The camera unit 31 is configured by a digital camera device that captures an image of a subject such as a CCD camera or a CMOS camera, and includes a video memory 32 that stores the generated image data. The camera unit 31 can generate both image data of still image data and moving image data. The image data stored in the video memory 32 can be stored in the storage unit 22 according to an instruction from the control unit 21.

  The log transmission unit 33 reads the life log information stored in the log storage unit 35 and causes the communication unit 26 to transmit the generated log data to the log collection server based on the read life log information.

  A time measuring unit which is time measuring means (not shown) measures time. The control unit 21 can acquire the current time by referring to the time measuring unit.

  FIG. 3 is a schematic diagram illustrating a communication system including the mobile device 20. The portable device 20 is connected to the communication network N so as to be able to perform data communication by wireless communication or wired communication as necessary, and the log collection server S is connected to the communication network N so as to be able to perform data communication by wired or wireless communication. The communication network N may be a network capable of data communication such as a mobile communication network or the Internet.

  The log data based on the life log information is read by the log transmission unit 33 by a user operation or according to a schedule such as a predetermined time and time interval, and is connected to the portable device 20 from the communication unit 26 via the communication network. Sent to the log collection server S. Further, the log data stored in the log collection server S is sent from the log collection server S to the portable device 20 in response to a request from the portable device 20 according to a user operation or according to a schedule such as a predetermined time and time interval. Sent to.

  Hereinafter, the operation related information and the position related information acquired when the life log key A is pressed by the user will be described.

  The operation-related information is acquired as image information or the like that is displayed according to the operation of the activated application mainly in a state where the application is activated.

  Examples of applications that operate on the system of the mobile device 20 include an image browsing application, an image capturing application, a web browsing application, a VOD (Video on Demand) viewing application, an electronic book application, a music playback application, and an email application. .

<Image browsing app>
The image browsing application is a so-called image viewer, and displays an image based on the image data stored in the storage unit 22 on the display unit 25. The image displayed by the image browsing application may be a still image or a moving image. For still images, for example, a list of a plurality of images can be displayed in a thumbnail display or a slide show can be displayed in which a plurality of images are switched one by one at regular intervals. When the user selects any one still image, the selected one still image is displayed on the display unit 25. Editing processing such as enlargement processing, reduction processing, and rotation processing can be performed on the selected image.

  In the case of a moving image, the first frame of each moving image data is taken out as still image data, displayed as a list as a representative image of each stored moving image, or displayed as a list of moving image titles and playback times. When the user selects any one representative image or title, a moving image of the selected representative image or title is reproduced.

  In such a state that the image browsing application is activated, when the life log key A is pressed and a trigger signal is input to the control unit 21, the application that is activated, that is, the name of the image browsing application, The identifiable information is stored as operation related information. At this time, when the trigger signal is input to the control unit 21, the image data such as the image displayed on the display unit 25, that is, the file name of the image data of the still image or moving image displayed by the user's selection is displayed. The identifiable information may be stored as operation related information. As more detailed information, when an edit process is performed on a still image at the time when a trigger signal is input to the control unit 21, for example, a scaling factor or a rotation angle may be stored as motion-related information. As more detailed information, when a moving image is played back when the trigger signal is input to the control unit 21, for example, the playback time of the moving image or the playback time when the trigger signal is input to the control unit 21 is reproduced. May be stored as motion-related information.

<Image shooting app>
The image capturing application is an application for capturing an image with a camera. The image to be taken may be a still image or a moving image. The image shooting application can set shooting conditions such as zoom magnification, shutter speed, and shooting resolution.

  In such a state that the image capturing application is activated, when the life log key A is pressed and a trigger signal is input to the control unit 21, the application that is activated, that is, the application such as the name of the image capturing application is displayed. The identifiable information is stored as operation related information. At this time, when the trigger signal is input to the control unit 21, the image data such as an image displayed on the display unit 25, that is, a file name of image data of a still image or a moving image captured by a user operation is identified. Possible information may be stored as motion related information. As more detailed information, the imaging conditions such as the set zoom magnification may be stored as operation-related information when the trigger signal is input to the control unit 21.

<Web browsing application>
The web browsing application is a web browser for displaying a web page, is connected to a web server via a mobile communication network or the Internet by data communication, downloads web page data, and displays the web page data on the display unit 25.

  In such a state that the web browsing application is activated, when the life log key A is pressed and a trigger signal is input to the control unit 21, the application that is activated, that is, the name of the web browsing application, The identifiable information is stored as operation related information. At this time, when the trigger signal is input to the control unit 21, information that can identify the web page such as a URL (Uniform Resource Locator) of the web page displayed on the display unit 25 by the web browser is used as the operation related information. You may remember.

<VOD viewing app>
The VOD viewing application is a video streaming playback player, which is connected to a content distribution server via a mobile communication network or the Internet by data communication, and plays while downloading content data.

  When the VOD viewing application is activated, first, login processing to the distribution service and authentication processing are performed, a title of reproducible video content is acquired from the content distribution server, and a title list is displayed. You may make it display the still image which shows the content of a content with a title, or reproduces a part of moving image (reproduction time is about several seconds). When the user selects content that the user wants to view, the VOD viewing application performs billing processing as necessary and starts streaming playback of moving image content.

  In such a state that the VOD viewing application is activated, when the life log key A is pressed and a trigger signal is input to the control unit 21, the application that is activated, that is, the name of the VOD viewing application is displayed. The identifiable information is stored as operation related information. At this time, when the trigger signal is input to the control unit 21, information that can identify the content such as the title and content ID of the moving image content being streamed by the player may be stored as the operation related information. As more detailed information, the playback time of the moving image or the frame number being played back when the trigger signal is input to the control unit 21 may be stored as the motion related information.

<E-book app>
The electronic book application is an electronic book viewer that displays electronic book data. The electronic book data includes text data and illustration image data, and is data obtained by digitizing newspapers, magazines, books, and the like, and displays text and illustrations in units of pages by an electronic book viewer in the same manner as a paper medium. be able to.

  When the electronic book application is activated, a list of book titles is displayed based on the electronic book data stored in the storage unit 22. A cover image and the like can be displayed together with the title of the book. When the user selects one title from the list displayed, the page of the selected book is displayed. In addition to the electronic book data stored in the storage unit 22, the electronic book application can be connected to the electronic book distribution server by data communication, and after being charged, etc., can be downloaded to obtain electronic book data.

  In such a state that the electronic book application is activated, when the life log key A is pressed and a trigger signal is input to the control unit 21, the application that is activated, that is, the application such as the name of the electronic book application is displayed. The identifiable information is stored as operation related information. At this time, when the trigger signal is input to the control unit 21, information that can identify the electronic book such as the title of the electronic book displayed in the electronic book viewer may be stored as the operation related information. As more detailed information, the page number displayed when the trigger signal is input to the control unit 21 may be stored as operation-related information.

<Music playback application>
The music playback application is a so-called music player that plays back music data and outputs it from the speaker unit 27. Music data can correspond to a plurality of formats.

  When the music playback application is activated, a list of music titles, album titles, artist names, and the like is displayed based on the music data stored in the storage unit 22. A jacket image and the like can be displayed together with the title of the music. When the user selects one title from the list displayed, the music data of the selected music is reproduced and output from the speaker unit 27. In addition to the music data stored in the storage unit 22, the music playback application can be connected to a music distribution server by data communication, and after being charged, etc., can be downloaded to obtain music data.

  When the life log key A is pressed and a trigger signal is input to the control unit 21 in such a state that the music reproduction application is activated, the application that is activated, that is, the name of the music reproduction application is displayed. The identifiable information is stored as operation related information. At this time, when the trigger signal is input to the control unit 21, information that can identify the music, such as the title of the music being played by the music player, may be stored as the operation related information. As more detailed information, the reproduction time reproduced at the time when the trigger signal is input to the control unit 21 may be stored as the operation related information.

<E-mail application>
The e-mail application is a so-called mailer for sending / receiving, creating, browsing, and the like of e-mails.

  When the e-mail application is activated, a menu screen is displayed, and various functions such as e-mail reception, e-mail transmission, e-mail creation, e-mail display, and e-mail search can be used.

  When the life log key A is pressed and a trigger signal is input to the control unit 21 in a state where such an e-mail application is activated, the application that is activated, that is, the name of the e-mail application, The identifiable information is stored as operation related information. At this time, when the trigger signal is input to the control unit 21, the source mail address, sender name, title and header information of the received mail displayed by the mailer, the destination mail address of the created outgoing mail, Information that can identify the electronic mail such as a destination name and a title may be stored as operation related information. As more detailed information, the text of the mail text or the attached file displayed when the trigger signal is input to the control unit 21 may be stored as operation related information.

  As the operation related information, not only the information related to the operation state of the mobile device 20 itself as described above but also information related to the control target device may be stored as the operation related information.

  For example, by installing an application in the mobile device 20, the mobile device 20 can function as a remote controller that can remotely control other devices. When such an application for remote control (hereinafter referred to as a remote control application) is activated, information on a device that is a control target of remote control and specific control contents are stored as operation-related information.

  When a remote control application for remotely controlling a television receiver (abbreviated as a television) is activated, when a lifelog key A is pressed and a trigger signal is input to the control unit 21, it is a target for remote control. Information that can be identified by the device to be controlled, such as the name of the television and ID information, is stored as operation-related information. At this time, when the trigger signal is input to the control unit 21, information regarding the selected channel, such as the selected channel and the name of a program broadcast on the selected channel, may be stored as operation-related information.

  When the life log key A is pressed and the trigger signal is input to the control unit 21 in a state where the remote control application for remotely controlling the lighting fixture is activated, the name or ID of the lighting fixture that is the target of remote control. Information that can be identified by the control target device, such as information, is stored as operation-related information. At this time, when the trigger signal is input to the control unit 21, information indicating the illumination state such as brightness (illuminance) and emission color may be stored as operation-related information.

  The air conditioner that is the object of remote control when the life log key A is pressed and the trigger signal is input to the mobile device control unit 21 in a state where the remote control app for remotely controlling the air conditioner is activated Information that can be identified by the device to be controlled, such as the name and ID information, is stored as operation-related information. At this time, when the trigger signal is input to the control unit 21, the operation type information such as the operation type such as the heating operation, the cooling operation, and the dry operation, and the temperature information such as the set temperature and the room temperature is displayed as the operation related information. May be stored as

  When the life log key A is pressed and the trigger signal is input to the control unit 21 in a state where the remote control application for remotely controlling the robot is activated, the name, ID information, etc. of the robot that is the target of remote control Information that can be identified by the control target device is stored as operation-related information. At this time, when the trigger signal is input to the control unit 21, the operation content of the robot such as walking, squatting, and forward rotation may be stored as the motion related information.

  The location-related information is acquired as information indicating the connection state in the case where the communication device 26 is connected to other devices in addition to the longitude and latitude indicating the current location of the mobile device 20 as well as the wireless communication by the communication unit 26. .

  The longitude and latitude representing the current position can be acquired by the GPS receiver 29. When the life log key A is pressed and a trigger signal is input to the control unit 21, the GPS receiving unit 29 acquires the longitude and latitude, and the acquired longitude and latitude are stored as position related information.

  In addition, when wirelessly connected to other devices, such as when wirelessly connected to a TV or wirelessly connected to a PC, at least wirelessly connected if the connected device can be specified. Since it is considered that the mobile device 20 exists at a position where it can be wirelessly connected to the connected device, it can be said that the information indicating the connection state with other devices is also position related information.

  As long as the communication method is communicable by the communication unit 26 of the mobile device 20, the wireless device may be connected to other devices by any wireless communication, such as Wi-Fi (registered trademark), Bluetooth (registered trademark) as described above. Alternatively, a communication method such as Zigbee (registered trademark) may be used. If another device can wirelessly communicate with the communication unit 26, the connection state can be acquired and stored as position related information. As the connection state, it may be only necessary to indicate whether or not the connection is established by wireless communication. Further, the connection between the devices connected to each other, such as UWB (Ultra Wide Band) communication, and the mobile device 20 is used as a reference. It is also possible to indicate the position of another connected device such as the direction in which the other device is present.

  When the life log key A is pressed and a trigger signal is input to the control unit 21, ID information or the like that allows the communication unit 26 to identify other devices wirelessly connected is extracted and stored as a connected device list.

  For other devices that can be wirelessly connected, if the location is registered in advance, the location of the connected device can be obtained when the device where the location is registered and the mobile device 20 are wirelessly connected. May be regarded as the location of the mobile device 20, and the location may be stored as location related information. For example, among the devices that can be wirelessly connected, the location “home” is registered for each identifiable ID information for a television, a PC, a hard disk recorder, a refrigerator, a weight scale, and the like placed at home. In addition, the location “company” is registered for ID information that can be identified for a PC placed in the company.

  When the life log key A is pressed and a trigger signal is input to the control unit 21, the communication unit 26 extracts ID information of other devices that are wirelessly connected, and is registered for the extracted ID information. The location is stored as position related information.

  In more detail, “Home-living” is the location of the TV and hard disk recorder, “Home-kitchen” is the location of the refrigerator, “Home-toilet” is the location of the scale, etc. The sub location may be registered as a sub location and subordinate to the main location, and more detailed position related information may be stored.

  When storing the motion-related information and the position-related information as described above in the storage unit 22, the current time when the motion-related information and the position-related information to be stored are acquired is associated with the acquired motion-related information and the position-related information. Store in the storage unit 22. When the life log key A is pressed and a trigger signal is input to the control unit 21, the operation related information and the position related information are acquired, and the operation related information and the current time as the life log acquisition time with reference to the time measuring unit are obtained. The information is stored in the storage unit 22 in association with the position related information.

  The acquisition time may be stored in addition to the current time stored in association with the motion related information and the position related information. When the lifelog key A is pressed and a trigger signal is input to the control unit 21, the current time is acquired. After the lifelog key A is pressed once, the lifelog key A is continuously pressed. In such a case, it is assumed that the series of life log keys A is pressed, and the storage unit 22 is associated with the operation related information and the position related information together with the current time, with the time when the life log key A is continuously pressed as the acquisition time. To remember. Whether or not the life log key A is continuously pressed is determined by whether or not the life log key A is newly pressed within a predetermined period after the life log key A is pressed last time. The predetermined period is, for example, 0.1 to 1 second. When the life log key A is newly pressed until 0.1 to 1 second elapses after the previous life log key A is pressed, It is determined that pressing of the life log key A this time is continuous. The acquisition time is the time when a series of life log keys A is pressed, and from the time when the life log key A is first pressed to the time when the life log key A is last pressed in the series of presses. It's time.

  When the acquisition time is stored in association with the motion-related information and the position-related information, it is preferable to store changes in the acquisition time of the motion-related information and the position-related information. For example, when the life log key A is continuously pressed during playback and browsing of a moving image, the playback period during the acquisition time is stored in addition to the motion related information. When the life log key A is pressed only once, even when a moving image is being played back, it is played back when the life log key A is pressed and a trigger signal is input to the control unit 21. However, when the acquisition time is stored, the frame number of all the frames reproduced during the acquisition period and the reproduction period are changed in the movement related information. Remember as.

  Regarding the operation of the life log key A, in the above, when it is pressed once, a trigger signal is generated to store the life log, and when the life log key A is continuously pressed, it is acquired as a series of life log key A presses. Remember time. When the life log key A is continuously pressed, the acquisition time may not be stored, but emotion information representing the user's emotion or evaluation may be stored. When the life log key A is continuously pressed, the number of times the life log key A is pressed is associated with the emotion or evaluation of the user in advance, and emotion information is obtained based on the number of times the life log key A is continuously pressed. Store as life log.

  For example, when the life log key A is pressed twice in succession, the user's emotion or evaluation is positive, such as fun, joyful, good, wonderful, etc. When life log key A is pressed for 3 consecutive times or when it is pressed for a long time, negative emotions such as boring, sad, bad, or disliked as user's emotion or evaluation It shall represent. As emotion information, “good” is stored when expressing positive emotion, and “no” is stored when expressing negative emotion.

  As in the case of storing the acquisition time, when the life log key A is pressed for the first time, the current time and the acquired motion related information and position related information are stored. When A is pressed, “good” is stored as emotion information. Further, when the life log key A is pressed for the third time continuously or when the life log key A is pressed for a long time, “No” is stored instead of “Good” as emotion information. After the fourth time, even if the life log key A is continuously pressed, the state where “NO” is stored as emotion information is maintained. If the second continuous life log key is not pressed, only motion related information and position related information are stored, and emotion information is not stored.

  FIG. 4 is a diagram illustrating an example of a data configuration of log data. The log data is data including motion-related information and position-related information acquired when the life log key A is pressed, “TIME” D1 indicating the current time or acquisition time, and “Place” D2 indicating the location. “ACT” D3 indicating emotion information, “App” D4 indicating the application name as information that can be identified by the application, “Contents” D5 indicating the operation content of the application, and “Connect” D6 indicating the connection state with other devices Are associated with each other and stored in the storage unit 22.

  In “TIME” D1, the time when the life log key A is pressed is described. In this example, the year, month, day, hour, minute, and second are described. In “Place” D2, a position when the life log key A is pressed is described, and in this example, longitude and latitude or a connected device is described. In “ACT” D3, emotion information indicating the emotion of the user when the life log key A is pressed is described. In this example, when the life log key A is pressed once (× 1), the motion related information and the position are simply displayed. “Check” that only stores related information is described. When pressed twice (× 2), “Good” indicating positive emotion is described, and when long pressed (Long), negative “BAD” representing the emotion of the person is described. In “App” D4, the name of the application is described. The standby state where the application is not activated is described as (standby) in this example. “Contents” D5 describes the operation content of the application, for example, the channel selected for the remote control application and the URL of the site displayed for the web browsing application. In “Connect” D6, identification information of another connected device is described, and for example, the model number of the device is described.

  Since the current time is always stored in the life log at the time of log acquisition, the stored operation-related information and position-related information are displayed on the display unit 25 along the flow of time based on the current time. . The operation-related information and the position-related information may be displayed as they are, and according to the stored information, abbreviations and keywords for character information, thumbnails for image information, etc. Display with time. By such display, the life log can be displayed in a so-called timeline.

FIG. 5 is a schematic diagram showing a life log output mode.
FIG. 5A shows a mode in which the motion-related information and the position-related information are displayed in a timeline along the flow of time. FIG. 5B shows the motion-related information and the position-related information for each month and day. FIG. 5C shows an output mode of a map display in which operation-related information and position-related information are displayed on a map based on the position-related information.

  In the timeline display, past information is displayed below the display unit 25 so that time flows along the vertical direction of the display unit 25, for example. If it does not fit on one screen, it can be displayed by scrolling up and down.

  In the timeline display, with one time as one unit, all the motion related information and position related information stored in association with one time are displayed in an area representing one unit. In the example shown in FIG. 5A, “time” 41 indicated by year, month, day, hour, minute, “location” 42 indicated by city, town, and village address, and “application name” 44 are displayed in one area 40, and a captured image (thumbnail) is displayed. 45, a VOD video frame (thumbnail) 46, a location 47, a control target device name 48, emotion information 49, and the like are displayed in an area 40 corresponding to one time represented by the year, month, day, hour and minute.

  The thumbnail 45 of the captured image and the thumbnail 46 of the VOD video frame are linked to the image data stored in the storage unit 22. When the thumbnail 45 of the captured image is selected by tapping or the like, image browsing is performed. The app starts and displays the corresponding captured image. When the thumbnail 46 of the VOD video frame is selected by tapping or the like, the VOD viewing application is activated and the corresponding video is played. In addition, when the location position 47 and the control target device name 48 are linked to each other and the location location 47 is selected by tapping or the like, the map application is started to display a map image around the location location, or the control target device When the name 48 is selected, a remote control application for remotely controlling the corresponding control target device is activated.

  As another display mode of the life log based on the same time as the timeline display, there is a calendar display shown in FIG. When the date for January is displayed on the display unit 25 and the life log associated with the date is stored with the date as one unit, the life log is displayed in the rectangular area 50 indicating the date. An icon 51 indicating that is stored is displayed. When a photographed image or the like is stored as operation related information, a thumbnail 52 of the photographed image is displayed in the date area 50.

  When the date area 50 in which the icon 51 or the thumbnail 52 is displayed is tapped or selected by tapping the icon 51 or the thumbnail 52, the corresponding area 40 in the timeline display is displayed in the center portion of the display screen. The

  Moreover, when position related information is stored, there is a map display as a mode of displaying a life log on a map based on the position related information. When a life log is stored based on position-related information with each prefecture as a unit, an icon 61 indicating that the life log is stored is displayed in the area 60 indicating the prefecture. When the region 60 of the prefecture where the icon 61 is displayed is tapped or selected by tapping the icon 61, the corresponding region 40 of the timeline display is displayed in the center portion of the display screen.

  In this way, the life log such as the operating state and position of the mobile device 20 at that time can be recorded with a simple operation by simply pressing the life log key A, and the recorded life log can be displayed later. You can look back and remember and check.

  Moreover, you may memorize | store friend acquaintance information in the portable apparatus memory | storage part 22 as a life log other than operation | movement relevant information and position relevant information.

  As for friend acquaintance information, for example, when the mobile device 20 makes a telephone communication with another mobile device, information related to a person such as a friend, acquaintance, family, or relative is previously stored in the mobile device. When associated with the owner, information about the connected portable device is not stored, but owner information such as the name of the associated owner is stored.

  When communicating with the other party's mobile device such as telephone communication, short message transmission / reception, chat communication, etc., when the lifelog key A is pressed and a trigger signal is input to the control unit 21, The mobile device on the side is specified, and owner information associated with the mobile device in advance is stored in the specified mobile device as friend acquaintance information, for example, by phone book software.

  The friend acquaintance information stored as the life log is also displayed in the same manner as the operation related information and the position related information in the timeline display, calendar display, map display and the like as shown in FIG.

  Furthermore, the portable device 20 can perform a search process on the operation related information, position related information, and friend acquaintance information of the life log recorded in the storage unit 22. As the search key for the search process, for example, date, location, person name, application name, etc. can be used.

  FIG. 6 is a diagram illustrating an example of an operation screen and a transition of the operation screen when performing a search process. For example, when a search processing application is activated, the search menu screen 70 is displayed on the display unit 25. The search menu screen 70 includes a date search button 71 for selecting a date search for searching using the date as a search key, a position search button 72 for selecting a location search for searching using the position as a search key, and searching for a person's name. A person name search button 73 for selecting a person name search to be searched as a key and an application search button 74 for application search to search using an application name as a search key are included.

  When one search button is selected from these search buttons by tapping or the like, a search process corresponding to the selected button can be executed. When the date search button 71 is selected, a date as a search key is input. In this embodiment, the date is input using the calendar display described above. In the calendar display, similar to the conventional calendar function, the displayed date can be switched, and a desired date can be displayed retroactively to the past date. The user switches the display to display a calendar display screen 75 including the date to be searched. When the date area 50 of the date to be searched is selected by tapping or the like, the control unit 21 uses the selected date as a search key to change the date as a search key from the life log stored in the storage unit 22. The associated movement-related information and position-related information are extracted, and the extracted movement-related information and position-related information are displayed on the search result display screen 76. The search key display area 77 at the top of the search result display screen 76 displays the date as the search key.

  When the position search button 72 is selected from the search buttons, a position serving as a search key is input. In this embodiment, the position is input using the above-described map display. In the map display, similar to the conventional map function, the displayed prefectures and regions can be switched, and a desired place can be displayed. The user switches the display to display the map display screen 78 including the position to be searched. When the area 60 including the position to be searched is selected by tapping or the like, the control unit 21 uses the selected position as a search key and the operation associated with the position that is the search key from the life log stored in the storage unit 22 The related information and position related information are extracted, and the extracted operation related information and position related information are displayed on the search result display screen 79. The position used as the search key is displayed as an address in the search key display area 80 at the top of the search result display screen 79.

  When the person name search button 73 is selected from the search buttons, a person name serving as a search key is input. In this embodiment, a person name is input using a phone book display. In the phone book display, similar to the conventional phone book function, the registered person names are sorted and displayed in alphabetical order and the desired person names can be displayed by switching the display. The user switches the display to display the telephone directory display screen 81 including the name of the person who wants to search. When the user name area 82 to be searched is selected by tapping or the like, the control unit 21 uses the selected person name as a search key, and the control unit 21 searches the life log stored in the storage unit 22 for operation related information associated with the name of the search key. Then, the position related information and the like are extracted, and the extracted operation related information and position related information are displayed on the search result display screen 83. In the search key display area 84 at the top of the search result display screen 83, the name of the person as the search key is displayed.

  When the application search button 74 is selected from the search buttons, an application name serving as a search key is input. In this embodiment, the application name is input using an application list display. In the application list display, a list of applications installed in the mobile device 20 can be displayed. If the application list does not fit on one screen, a desired application name can be displayed by scrolling. The user switches the display to display the display screen 85 including the application name to be searched. When the application name area 86 to be searched is selected by tapping or the like, the control unit 21 uses the selected application name as a search key and is associated with the application name that is the search key from the life log stored in the storage unit 22. The operation related information and the position related information are extracted, and the extracted operation related information and the position related information are displayed on the search result display screen 87. The search key display area 88 at the top of the search result display screen 87 displays the application name as the search key.

  In this manner, the stored life log can be searched using any search key, and desired information can be displayed or output later. The search key may be input from a soft keyboard, or the voice input from the microphone unit 30 is recognized, and the voice search using the recognized word as the search key is also possible.

  FIG. 6 shows an example in which a predetermined life log is searched from the life log stored by the search, but this is an example of calling the stored life log, and in addition to the search, the current position information of the mobile device 20 is displayed. In addition, a life log stored when the application has been in the same place in the past by an automatic operation or a user operation can be displayed. You can also look back on what you did today last year by calendaring the lifelog stored today last year based on today's date.

DESCRIPTION OF SYMBOLS 20 Portable apparatus 21 Control part 22 Storage part 23 Temporary storage part 24 Operation part 25 Display part 26 Communication part 27 Speaker part 28 Vibration part 29 GPS receiving part 30 Microphone part 31 Camera part 33 Log transmission part 35 Log storage part A Life log key

Claims (4)

An acquisition unit that acquires position-related information relating to the current position of the operation-related information and own apparatus associated with the operation state of the apparatus including the standby state,
Storage means for storing the motion-related information and the position-related information;
By the user's operation, an instruction unit that instructs the acquisition unit acquires the motion related information and the location-related information,
It said instruction means detects the time when the instruction to the acquisition means, and the detected time, in association with the motion related information and the location-related information obtained by the obtaining unit by the instruction control to be stored in said storage means Means,
Log output means for outputting log information based on the motion-related information and the position-related information stored in the storage means in the order of associated times;
A mobile terminal device comprising: map display means for displaying the motion-related information and the position-related information stored in the storage means on a map based on the position-related information . Display means for displaying an image;
Voice output means for outputting voice,
The operation related information includes application information related to the activated application software, image information related to the image displayed on the display means, and audio information related to the sound output by the output means. The mobile terminal device according to claim 1, comprising: A communication means for communicating with another device;
3. The position related information includes longitude and latitude information indicating the position of the own device in longitude and latitude, and connection information related to a connection state by wireless communication with another device. Mobile terminal device. A number of times detecting means for detecting the number of times the instruction means is operated;
The said control means links | relates the said time and emotion information showing the user's emotion according to the frequency | count detected by the said frequency | count detection means, It memorize | stores in the said memory | storage means, The said storage means is characterized by the above-mentioned. The portable terminal device as described in any one.