WO2013039025A1 - 情報管理編集機能を備えた情報処理装置 - Google Patents
情報管理編集機能を備えた情報処理装置 Download PDFInfo
- Publication number
- WO2013039025A1 WO2013039025A1 PCT/JP2012/073015 JP2012073015W WO2013039025A1 WO 2013039025 A1 WO2013039025 A1 WO 2013039025A1 JP 2012073015 W JP2012073015 W JP 2012073015W WO 2013039025 A1 WO2013039025 A1 WO 2013039025A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- information
- attribute
- unit
- user
- area
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
- G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F3/04842—Selection of displayed objects or displayed text elements
Definitions
- the present invention relates to an information processing apparatus equipped with a touch panel, and particularly to an information processing apparatus having an information management editing function.
- This application claims priority based on Japanese Patent Application No. 2011-202865 for which it applied to Japan on September 16, 2011, and uses the content here.
- This mobile terminal is equipped with a touch panel, and the user can execute an arbitrary function by touching an icon or the like displayed on the operation screen. With this touch panel, the user can easily and intuitively operate the mobile terminal.
- Patent Document 1 discloses a household account book input support system in which a user transmits a receipt image taken by a portable information terminal to a server, and the server creates household account book data based on the receipt image and transmits it to the portable information terminal. Disclosure.
- Patent Document 2 discloses an input device that allows a user to select and operate an object such as a plurality of icons simultaneously with one hand by simultaneously operating a plurality of fingers on a touch panel.
- Patent Document 3 when copying a file on a PC, a portable device, a TV receiver, or the like, moving to a folder or recording medium, or viewing by a user, a plurality of files for a plurality of files can be processed with a simple procedure.
- a screen operation method capable of executing an operation mode is disclosed.
- Patent Document 4 discloses a portable electronic device having a multi-touch input function for executing processing on a plurality of objects based on a multi-touch operation.
- Patent Document 1 discloses a household account book creation support service, which manages information acquired by a user's mobile terminal using a server. That is, a receipt image taken by the user with the mobile terminal is transmitted to the server, and the server analyzes the receipt image, extracts the information, and creates household account book data. Thereafter, the server provides the household account book data to the portable terminal. In this household account book input support system, the server extracts all recognizable character information from the characters included in the receipt image, and creates household account book data by associating information indicating date, product name, amount, etc. .
- the information management method used when creating a household account book varies from user to user. For example, there are users who want to manage all purchased items in detail, users who want to manage only food category products among purchased products, and users who want to manage only the total price for each store or receipt. In addition, there are users who want to manage all the products included in the food category out of the purchased products, and users who want to further subdivide and manage the food category such as fresh food and beverages.
- each user needs to edit the household account book data created by the server in accordance with his / her favorite management method.
- information that the user does not want to register in the household account book is automatically managed by the server.
- the user selects only the information he / she wants to manage in the household account book and inputs it to a computer using a keyboard or mouse, the user's input work becomes complicated, and an input error may occur, which is inconvenient for the user. Met.
- the present invention has been made in view of the above-described problems, and an object thereof is to provide an information processing apparatus having an information management editing function in consideration of user convenience.
- the present invention relates to an information processing apparatus equipped with a touch panel, and a display unit that displays an operation screen, an operation unit that receives a user operation on the operation screen, and information that acquires designation information from a display object area on the operation screen.
- An acquisition unit and an information management unit that manages the attribute and designation information in association with each other when a desired attribute is set in advance by the user.
- the present invention relates to an information processing apparatus equipped with a touch panel, and designates desired designation information by controlling a display unit that displays an operation screen, an operation unit that receives a user operation on the operation screen, and the display unit.
- a display control unit for displaying a specified information operation area for receiving a user operation, an input operation area for receiving an information input operation by a user, an attribute operation area for receiving a user operation for setting a desired attribute on an operation screen, and an operation unit Based on the detection result of the user operation on the operation screen, the operation content determination unit for determining the instruction content of the user operation, the desired specification information is specified by the user operation in the specification information operation area, and the user When an information input operation is performed on the input operation area, an information input unit for transferring designated information to the input operation area; Comprising.
- the present invention relates to an information processing method applied to an information processing apparatus equipped with a touch panel, accepts a user operation on an operation screen displayed on the touch panel, acquires designation information from a display object area on the operation screen, When a desired attribute is set in advance by the user, the attribute and the designation information are associated with each other and managed.
- the present invention relates to a program applied to a computer that realizes an information processing function linked to a user operation, accepts a user operation on an operation screen, acquires designation information from a display object area on the operation screen, and When a desired attribute is set by the control, the computer is controlled to manage the attribute and the designation information in association with each other.
- the present invention realizes a user operation-linked information processing program such as a household account book information acquisition application, a household account book management application, a text editing application, and a search application in an information processing apparatus such as a portable terminal, and allows simple user operations.
- a contact operation, a touch slide operation, a flick operation and a desired attribute (for example, payment item for each category of household account book data) and user-specified information (for example, payment amount for each payment item) are associated with each other.
- a desired attribute for example, payment item for each category of household account book data
- user-specified information for example, payment amount for each payment item
- management information such as household account book data can be made into a database by a simple user operation.
- FIG. 1 shows a communication network system (information processing system) to which a mobile terminal 1 according to a first embodiment of the present invention is connected.
- the information processing apparatus according to the present invention is not limited to the portable terminal 1 but may be realized by a smartphone, a portable game machine, a PDA (Personal Digital Assistant), a tablet PC (Personal Computer), or a notebook PC.
- the mobile terminal 1 includes, for example, a call function, an e-mail function, an Internet connection function (Web access function), and a digital television broadcast function (for example, a one-segment terrestrial digital television broadcast function).
- the portable terminal 1 When the portable terminal 1 is connected to the wireless communication network (mobile communication network) 2 via the nearest base station 2A and the exchange 2B, the portable terminal 1 is connected to another portable terminal via the wireless communication network 2. Calls can be made with 3. Further, when the mobile terminal 1 is connected to the Internet 4 via the wireless communication network 2, the mobile terminal 1 can access and browse the Web site. Furthermore, the mobile terminal 1 can perform streaming transmission in which multimedia contents such as moving images, still images, music, and news are downloaded from the server 5 via the Internet 4 or the wireless communication network 2 and played back.
- multimedia contents such as moving images, still images, music, and news are downloaded from the server 5 via the Internet 4 or the wireless communication network 2 and played back.
- FIG. 2 is a block diagram showing a basic configuration of the mobile terminal 1.
- the mobile terminal 1 includes a touch panel 101, a control unit 102, a storage unit 103, a camera 104, a camera control unit 105, an image processing unit 106, an image analysis unit 107, a wireless communication unit 108, an audio signal processing unit 109, and a time measuring unit 110. It has.
- a plurality of processing modes are set for the portable terminal 1.
- the mobile terminal 1 activates the camera 104 to photograph the subject and obtains the image data.
- the mobile terminal 1 acquires user designation information from an image captured by the camera 104, and writes the attribute information indicating the attribute and the designation information in the storage unit 103 in association with each other.
- the portable terminal 1 is set to the imaging mode according to the household account book information acquisition application app1, and when a desired image is captured by the camera 104, the processing mode is switched to the database mode. That is, the household account book information acquisition application app1 is an application that acquires designation information for creating a household account book for each attribute. Although a present Example demonstrates the process of the portable terminal 1 according to the household account book information acquisition application app1 downloaded from the server 5, it is not limited to this.
- the program of household account book information acquisition application app1 may be installed in the mobile terminal 1 in advance.
- the user takes a receipt with the camera 104 of the mobile terminal 1 and displays information such as a store name, a product name, and an amount of money described in the receipt on the touch panel 101.
- the household account book information acquisition application app1 for acquiring information for creating a household account book is activated on the mobile terminal 1, the touch panel 101 has a photographed receipt image and the household account book information acquisition application app1 predetermined.
- An attribute icon for designating an attribute is displayed at the same time.
- the mobile terminal 1 When the user performs a touch operation on any one of the attribute icons and performs a slide operation on a portion that displays information such as a store name, a product name, and an amount of money in the receipt image, the mobile terminal 1 displays the attribute indicated by the touched attribute icon. Then, it is written in the storage unit 103 in association with the designation information of the part that has been slid. In this way, the mobile terminal 1 associates the attribute indicated by the attribute icon touched by the user with the designation information indicating the part that has been slid.
- the portable terminal 1 can acquire attribute information and designation
- the contact operation is canceled, and then the user performs a slide operation on a desired portion to instruct designation information.
- the operation for instructing attribute information and the operation for instructing designation information are not limited to the contact operation and the slide operation, and may be performed by other user operations.
- the touch panel 101 includes an operation unit 111 and a display unit 112.
- the operation unit 111 includes a sensor that receives a user operation, and outputs a detection result of the sensor to the control unit 102. That is, the operation unit 111 detects the contact position of the user's finger on the operation screen by a sensor at regular time intervals, and outputs the detection result of the sensor to the control unit 102. This is not a limitation, and for example, the position of the user's finger or operation instruction means (stylus pen) close to the operation screen may be detected by a non-contact sensor.
- the operation unit 111 receives a user operation via an operation screen displayed on the display unit 112.
- the operation unit 111 is manufactured integrally with the display unit 112 constituting the touch panel 101, and the display screen of the display unit 112 and the operation screen of the operation unit 111 are the same.
- the display unit 112 displays an operation screen that accepts a user operation on the operation unit 111.
- the display unit 112 displays an image captured by the camera 104 in the operation screen.
- the captured image displayed on the operation screen includes display objects such as characters and diagrams. That is, the display unit 112 displays a display object instructed by the user on the operation screen.
- the control unit 102 reads various information stored in the storage unit 103 and controls the mobile terminal 1 in an integrated manner.
- the control unit 102 includes an operation content determination unit 121, a display control unit 122, a registration unit 123, an information acquisition unit 124, an information management unit 125, an application processing unit 126, and a voice control unit 127.
- the display content determination unit 121 determines the operation content received by the operation unit 111 based on the output of the operation unit 111. For example, the operation content determination unit 121 determines the movement of the user's finger on the touch panel 101 based on the contact position and contact time of the user's finger indicated by the detection result of the operation unit 111. The operation content determination unit 121 determines the operation content instructed by the movement of the user's finger based on the positional relationship between the movement of the user's finger and the image displayed on the display unit 112 when the user operation is accepted. For example, when the operation unit 111 detects the touch of the user's finger on the operation screen, the operation content determination unit 121 determines that the user operation is a contact operation.
- the operation content determination unit 121 determines that the user operation is a slide operation.
- the operation content determination unit 121 determines that the user operation is a flick operation.
- the operation content determination unit 121 may determine the operation content received by the operation unit 111 based on the time information timed by the time measuring unit 110.
- the operation content determination unit 121 It may be determined that the user operation is an area specifying operation. This area designation operation refers to an operation in which a display object area including a display object corresponding to the contact position of the user is designated by the user.
- the display object area is the character Or an area on the operation screen corresponding to an image showing numbers.
- the operation content determination unit 121 is instructed by text data of characters or figures included in the display object area. It is determined that
- the operation content determination unit 121 instructs the display control unit 122 to display the display content corresponding to the user operation on the display unit 112 based on the determination result. For example, when it is determined that the user operation is a contact operation, the operation content determination unit 121 controls the display control unit 122 so as to superimpose and display an icon image indicating the contact position of the user's finger on the operation screen. The display control unit 122 causes the display unit 112 to display a finger icon indicating the contact position of the user's finger. Further, the display control unit 122 displays a plurality of finger icons indicating contact positions of a plurality of user's fingers on the operation screen of the touch panel 101. The display control unit 122 controls the display content of the display unit 112 based on the user operation indicated by the determination result of the operation content determination unit 121.
- the registration unit 123 registers the data received from the server 5 by the wireless communication unit 108 via the Internet 4 in the storage unit 103. For example, the registration unit 123 downloads various applications to be installed on the mobile terminal 1 from the server 5 and registers them in the storage unit 103.
- the information acquisition unit 124 acquires, from the display objects displayed on the operation screen of the touch panel 101, the display object corresponding to the user-specified position on the operation screen as the specified information. Based on the determination result of the operation content determination unit 121, the information acquisition unit 124 sets an attribute of the designation information instructed by the user, and the designation information from the information stored in the storage unit 103. And a designated information acquisition unit 1242 to be acquired.
- the attribute setting unit 1241 determines the attribute indicated by the attribute icon designated by the contact operation based on the user contact position on the operation screen. .
- the attribute setting unit 1241 sets the attribute indicated by the attribute icon based on the determination result as the attribute of the designation information designated by the user. In this embodiment, the attribute setting unit 1241 sets only one attribute.
- the designation information acquisition unit 1242 determines whether or not the user has performed a slide operation on the operation screen including the display object. For example, when an image captured by the camera 104 is displayed on the operation screen of the touch panel 101, the designation information acquisition unit 1242 determines whether the user has performed a slide operation on the operation screen on which the captured image is displayed. To do. Based on the detection position of the user's slide operation on the operation screen, the designation information acquisition unit 1242 detects the display object area instructed by the slide operation. When the display object area is detected by the user's slide operation, the designation information acquisition unit 1242 acquires the designation information corresponding to the display object area.
- the information management unit 125 sets the attribute set by the attribute setting unit 1241 and the designation acquired by the designation information acquisition unit 1242. Information is managed in association with each other. Also, the information management unit 125 writes the attribute set by the attribute setting unit 1241 and the designation information acquired by the designation information acquisition unit 1242 in association with each other in the designation information database 137. Further, when the information management unit 125 first writes the designation information associated with the predetermined attribute information into the designation information database 137, in other words, the information management unit 125 performs a desired operation in a state where the designation information is not associated with the predetermined attribute information.
- the information management unit 125 associates the attribute set by the attribute setting unit 1241 with the designation information acquired by the designation information acquisition unit 1242 and stores them in an external storage medium or storage device for management. Also good.
- the application processing unit 126 reads an application stored in the application storage area 135 of the storage unit 103 based on the determination result of the operation content determination unit 121, and executes the application program. For example, when the user inputs an instruction to start the household account book information acquisition application app1 via the operation unit 111, the application processing unit 126 reads the program of the household account book information acquisition application app1 from the application storage area 135 and executes it. The application processing unit 126 executes processing in the imaging mode according to the household account book information acquisition application app1, and then executes processing in the database mode when image data is acquired.
- the application processing unit 126 When the user inputs an instruction to start the household account book management application app2 via the operation unit 111, the application processing unit 126 reads the program of the household account book management application app2 from the application storage area 135 and executes it. The application processing unit 126 adds information written in the specified information database 137 according to the household account book management application app2 and updates the user's household account book data registered in advance. The application processing unit 126 may update the household account book data stored in an external storage medium or storage device.
- the digital audio data input from the audio control unit 127 and the audio signal processing 109 is transmitted to the other portable terminal 3 via the wireless communication unit 108.
- the digital audio data is output to the audio signal processing unit 109.
- the storage unit 103 stores various information used for processing of the mobile terminal 1.
- the storage unit 103 includes a camera image storage area 131, a display object storage area 132, a program storage area 133, a temporary storage area 134, an application storage area 135, and a designation information storage area 136.
- the storage unit 103 may include a removable portable memory (recording medium) such as an SD card, an IC card, or an external hard disk. Alternatively, the storage unit 103 may be set on a predetermined external server (not shown).
- the camera image storage area 131 is a storage area for storing images taken by the camera 104.
- the camera image storage area 131 stores image data processed by the image processing unit 106.
- the display object storage area 132 stores a display object (for example, text data or chart data) extracted from the photographed image of the camera 104 by the image analysis unit 107 and position information indicating a position in the photographed image in association with each other. It is an area.
- the program storage area 133 is an area for storing a program, various applications, and the like for realizing the processing of the present embodiment in accordance with various processing performed on the mobile terminal 1 by the user.
- the temporary storage area 134 is an area for temporarily storing various information necessary for the mobile terminal 1 to operate.
- the application storage area 135 is an area for storing application programs installed in the mobile terminal 1. For example, the application storage area 135 stores a household account book information acquisition application app1 and a household account book management application app2.
- the designation information storage area 136 is an area for storing a designation information database 137 that stores the designation information acquired by the information acquisition section 124 of the control section 102 and attribute information in association with each other.
- FIG. 3 shows an example of the contents stored in the designation information database 137.
- the designation information database 137 is a table that stores additional flags, attribute information, and designation information in association with each other. The additional flag indicates that the designation information is already written in the designation information database 137 in association with the attribute information.
- the designation information is acquired by the designation information acquisition unit 1242 of the information acquisition unit 124.
- the attribute information is set by the attribute setting unit 1241 when the specified information is acquired by the specified information acquisition unit 1242. In the designation information database 137 of FIG. 3, designation information “ ⁇ 180” is written in association with the attribute “food”.
- the camera 104 includes an optical system 141, an image sensor 142, and an A / D conversion unit 143.
- the image sensor 142 generates an image signal from an optical image incident through the optical system 141.
- the A / D conversion unit 143 performs analog / digital conversion on the image signal output from the image sensor 142 to generate image data.
- a power switch (not shown) is operated, the camera 104 generates continuous image data from the optical image incident on the optical system 141 and outputs it to the image processing unit 106.
- the mobile terminal 1 displays a live view on the display unit 112 of the touch panel 101 based on continuous image data. The user can determine an image to be shot while viewing the live view.
- the camera control unit 105 controls the camera 104 to acquire desired image data based on an optical image incident on the optical system 141 of the camera 104. Control.
- the photographed image is preferably an image with higher image quality than the live view.
- the image processing unit 106 performs image processing on the image data output from the camera 104 and stores the image data after the image processing in the camera image storage area 131 of the storage unit 103.
- the image analysis unit 107 reads image data from the camera image storage area 131 and extracts display objects (text data and chart data) from the image data.
- the image analysis unit 107 recognizes and extracts characters and numbers similar to a predetermined text pattern such as characters and numbers. Further, the image analysis unit 107 recognizes and extracts a logo mark, a product image, or the like similar to a chart pattern such as a predetermined logo mark or product image.
- the image analysis unit 107 determines the image area on the captured image from which the display object is extracted as the display object area, and associates the display object area with the display object. For example, the image analysis unit 107 associates the position information of the display object area on the captured image with the display object extracted from the display object area using the coordinate values of the XY coordinates indicating the position of each pixel included in the captured image. .
- the wireless communication unit 108 transmits / receives data to / from the nearest base station 2A via an antenna during operations such as a call function, an e-mail function, and an Internet connection function.
- the audio signal processing unit 109 is connected to the microphone MC and the speaker SK.
- the audio signal processing unit 109 A / D converts the analog audio signal input from the microphone MC and outputs digital audio data to the audio control unit 127.
- the audio signal processing unit 109 D / A converts the digital audio data input from the audio control unit 127 and outputs an analog audio signal to the speaker SK.
- the timer unit 110 outputs a clock signal at regular time intervals.
- FIG. 4 shows an information reception screen G1 displayed on the display unit 112 of the touch panel 101 when the user activates the household account book information acquisition application app1.
- This information reception screen G1 is displayed on the operation screen of the touch panel 101 when a program of the household account book information acquisition application app1 is executed, for example, when the user takes a receipt with the camera 104 in the imaging mode and then shifts to the database mode. Is displayed.
- the information reception screen G1 includes a designation information operation area G11 for displaying an image captured by the user with the camera 104, and an attribute operation area G12 for displaying attribute icons Z1 to Z8 for the user to designate attributes.
- a capture data display area G13 for displaying predetermined designation information for each attribute.
- the predetermined designation information is designation information selected by the user when the attribute is selected, and is written in the designation information database 137 by the information management unit 125.
- the designated information operation area G11 is displayed at the top of the operation screen of the display unit 112, and the capture data display area G13 is displayed at the bottom of the operation screen.
- the attribute operation area G12 is positioned between the designated information operation area G11 and the capture data display area G13 and is displayed in the middle part of the operation screen.
- the photographed image of the receipt is displayed in the designated information operation area G11.
- the photographed image of this receipt includes the name of the store where the payment was made (for example, “Convenience XX”), the logo mark H1 indicating the logo image of the store, and the date of payment (for example, “20 ** Year * * Month ** day "), and the item name and amount indicating the product or category for which the payment was made.
- the amount” ⁇ 180 is displayed in the right column of the payment item” bread ", and the payment item
- the amount “ ⁇ 220” is displayed in the right column of “Butter”
- the amount “ ⁇ 350” is displayed in the right column of the payment item “Magazine”
- the amount “ ⁇ 5500” is displayed in the right column of the payment item “Telephone fee”.
- the display image in the designated information operation area G11 is a part of the photographed image of the receipt, and in fact, other payment items, the amount of money, and the amount below the line “Telephone charge ⁇ 5500” are displayed. Includes the total amount of payment.
- attribute icons Z1 to Z8 indicating attributes predetermined in the household account book acquisition application app1 are displayed. Attributes of designated information that can be taken in by the household account book information acquisition application app1 are set in advance as the attribute icons Z1 to Z8. Specifically, the attribute icon Z1 indicates the attribute “logo”, which is an operation icon for instructing to take in the logo of the store where the payment was made. The attribute icon Z2 indicates the attribute “store name”, and is an operation icon for instructing to take in the name of the store where the payment is made.
- the attribute icons Z3 to Z6 are operation icons indicating the attributes “food”, “miscellaneous goods”, “light heat”, and “communication” and instructing to capture the amount of each payment item.
- the attribute icon Z7 is an operation icon indicating the attribute “total” and instructing to capture the total amount of payment.
- the attribute icon Z8 is an operation icon indicating “end” and instructing to end the household account book information acquisition application app1. Therefore, the attribute icon Z8 does not set a specific attribute but is an operation icon that supports processing for an application.
- the capture data display area G ⁇ b> 13 is an area for displaying the designation information captured in the designation information database 137 from the captured image of the camera 104.
- the operation procedure and processing for taking in the logo mark H1 of the store where the user has paid will be described.
- the operation content determination unit 121 detects the user's touch operation and displays it.
- the control unit 122 displays the finger icon Q1 superimposed on the attribute icon Z1. That is, the finger icon Q1 is displayed at a position on the touch panel 101 where the contact operation by the user is detected.
- a state is shown in which the user maintains a contact operation with the thumb of the right hand on the attribute icon Q1.
- the attribute setting unit 1241 of the information acquisition unit 124 of the control unit 102 sets the attribute “logo” indicated by the attribute icon Z1 as the attribute of the designation information based on the determination result of the operation content determination unit 121.
- the attribute setting unit 1241 registers attribute information indicating the currently set attribute “logo” in the temporary storage area 134.
- the display control unit 122 displays a tab Tb1 indicating the currently set attribute “logo” in the upper left part of the capture data display area G13.
- the operation content determination unit 121 detects a touch slide operation, and the display control unit 122 displays a finger icon Q2 superimposed on the store logo mark H1. As shown in FIG. 5B, the display control unit 122 moves the finger icon Q2 along the finger moving direction in the touch slide operation.
- the designation information acquisition unit 1242 of the information acquisition unit 124 of the control unit 102 based on the determination result of the operation content determination unit 121, the store logo mark corresponding to the display object region specified by the user's touch slide operation H1 is acquired as the designation information.
- the display control unit 124 superimposes and displays an animation image indicating that the store logo mark H1 moves from the designated information operation area G11 to the capture data display area G13 on the information reception screen G1. This animation image will be described later with reference to FIG.
- the information management unit 125 assigns the designation information “logo mark H1” to the attribute “logo”.
- the logo mark H1 of the store where the user has performed the touch slide operation is taken in as the designation information of the attribute “logo”.
- the display control unit 122 displays an image of the logo mark H1, which is designation information captured by the user, in the capture data display area G13.
- the operation content determination unit 121 detects the touch operation
- the display control unit 122 displays the finger icon Q1 superimposed on the attribute icon Z3. That is, the finger icon Q1 is displayed at the position where the contact operation by the user is detected.
- the user touches the attribute icon Z1 with the thumb of the right hand and maintains the contact operation.
- the attribute designation unit 1241 of the information acquisition unit 124 of the control unit 102 sets the attribute “food” indicated by the attribute icon Z3 as the attribute of the designation information based on the determination result of the operation content determination unit 121.
- the attribute setting unit 1241 registers the attribute information indicating the currently set attribute “food” in the temporary storage area 134. Further, the display control unit 122 displays the tab “Tb2” of the currently set attribute “food” in the upper left part of the capture data display area G13.
- the operation content determination unit 121 detects the user's touch slide operation, and the display control unit 122 displays the finger icon Q2 superimposed on the amount “ ⁇ 180” of the payment item “bread”. Further, as illustrated in FIG. 6B, the display control unit 122 moves the finger icon Q2 along the moving direction of the user's finger in the touch slide operation.
- the designation information acquisition unit 1242 of the information acquisition unit 124 of the control unit 102 based on the determination result of the operation content determination unit 121, the amount of the payment item “bread” corresponding to the display object region specified by the user's touch slide operation “ ⁇ 180” is acquired as the designation information.
- the display control unit 124 superimposes an animation image displayed when the image of the payment item “bread” amount “ ⁇ 180” moves from the designated information operation area G11 to the capture data display area G13 on the information reception screen G1. indicate. This animation image will be described later with reference to FIG.
- the information management unit 125 associates the designation information “ ⁇ 180” with the attribute “food” and designates the designation information. Write to database 137. As a result, the amount “ ⁇ 180” of the payment item “bread” that has been touch-slid by the user is captured as designation information for the attribute “food”. In addition, the information management unit 125 sets an additional flag for the attribute “food” in the designation information database 137. The display control unit 122 displays the amount “ ⁇ 180” of the payment item “bread”, which is the designation information captured by the user, in the capture data display area G13.
- the operation content determination unit 121 Detects the user's touch slide operation, and the display control unit 122 displays the finger icon Q2 superimposed on the amount “ ⁇ 220” of the payment item “butter”. Further, as shown in FIG. 6C, the display control unit 122 moves the finger icon Q2 along the moving direction of the user's finger in the touch slide operation.
- the designation information acquisition unit 1242 of the information acquisition unit 124 of the control unit 102 determines the payment item “butter” corresponding to the display object area designated by the user's touch slide operation based on the determination result of the operation content determination unit 121.
- the amount “ ⁇ 220” is acquired as the specified information.
- the display control unit 124 superimposes an animation image displayed when the image of the payment item “butter” amount “ ⁇ 220” moves from the designated information operation area G11 to the capture data display area G13 on the information reception screen G1. indicate.
- the animation image will be described later with reference to FIG.
- the information management unit 125 When the desired attribute is already set by the user and the desired designation information is acquired from the display object area, the information management unit 125 writes the designation information in association with the attribute in the designation information database 137.
- the amount “ ⁇ 180” of the payment item “bread” has already been written in the designation information database 137 as the designation information of the attribute “food”. Therefore, in the designation information database 137, an additional flag is set for the attribute “food”. Further, the attribute information of the currently set attribute “food” is registered in the temporary storage area 134.
- the information management unit 125 When the amount “ ⁇ 220” of the payment item “butter” is written in the designated information database 137, an additional flag is already set in the attribute “food”, so the information management unit 125 writes it in correspondence with the attribute “food”. After the designation information “ ⁇ 180” is added, an addition symbol “+” is added, and the amount “ ⁇ 220” of the payment item “butter” is written. That is, the information management unit 125 writes “ ⁇ 180 + ⁇ 220” as the designation information associated with the attribute “food” in the designation information database 137. As a result, the amount “ ⁇ 220” of the payment item “butter” that has been touch-slid by the user is captured as the designation information of the attribute “food”.
- the information management unit 125 determines whether or not attribute information indicating the currently set attribute is registered in the temporary storage area 134. When it is determined that the attribute information is registered in the temporary storage area 134, the information management unit 125 writes the addition symbol “+” after the designation information “ ⁇ 180” written in association with the attribute “food”. In addition, the amount “ ⁇ 220” of the payment item “butter” may be written.
- the display control unit 122 displays “ ⁇ 180 + ⁇ 220” indicating that the amount “ ⁇ 220” of the payment item “butter” which is the designation information captured by the user is added to the amount “ ⁇ 180” of the payment item “bread”. The image is displayed in the capture data display area G13.
- the information management unit 125 adds the addition symbol “+” and then writes the new designation information following the already written designation information. Another example of processing in this case will be described below.
- the information management unit 125 adds the amount “ ⁇ 220” of the payment item “butter” to the amount “ ⁇ 180” of the payment item “bread” already written.
- the total amount “ ⁇ 400” may be calculated, and the specified information associated with the attribute “food” may be overwritten with the total amount “ ⁇ 400”.
- the display control unit 122 adds “ ⁇ 400” which is the total amount of the amount “ ⁇ 220” of the payment item “butter” and the amount “ ⁇ 180” of the payment item “bread” which is the designation information captured by the user. "Is displayed in the capture data display area G13.
- the user touches the attribute icon Z3 with the thumb of the right hand.
- the operation content determination unit 121 detects the user's contact operation, and the display control unit 122 displays the finger icon Q1 superimposed on the attribute icon Z3.
- the information management unit 125 sets an additional flag for the attribute “food” in the designation information database 137.
- attribute information indicating the currently set attribute “logo” is registered in the temporary storage area 134.
- the attribute setting unit 1241 since the attribute designated by the user's touch operation has not been changed, the attribute setting unit 1241 does not rewrite the storage contents of the designation information database 137 and the temporary storage area 134.
- the attribute information setting unit 1241 rewrites the storage contents of the designation information database 137 and the temporary storage area 134.
- the operation content determination unit 121 touches the user.
- the display control unit 122 displays the finger icon Q2 superimposed on the amount “ ⁇ 150” of the payment item “snack candy”. Further, as shown in FIG. 7B, the display control unit 122 moves the finger icon Q2 along the moving direction of the user's finger in the touch slide operation.
- the designation information acquisition unit 1242 of the information acquisition unit 124 of the control unit 102 based on the determination result of the operation content determination unit 121, the payment item “snack candy” corresponding to the display object area specified by the user's touch slide operation.
- the amount of money “ ⁇ 150” is acquired as the specified information.
- the display control unit 122 superimposes the animation image displayed when the image of the payment item “snack candy” amount “ ⁇ 150” moves from the designated information operation area G11 to the capture data display area G13 on the information reception screen G1. indicate. This animation image will be described later with reference to FIG.
- the information management unit 125 associates the designation information “ ⁇ 150” with the attribute “food” and designates the designation information. Write to database 137.
- the designation information of the attribute “food” the amount “ ⁇ 180” of the payment item “bread” and the amount “ ⁇ 220” of the payment item “butter” are already written in the designation information database 137. Therefore, in the designation information database 137, an additional flag is set for the attribute “food”. Further, attribute information indicating the currently set attribute “food” is registered in the temporary storage area 134.
- the information management unit 125 Since the additional flag is set for the attribute “food” when the amount “ ⁇ 150” of the payment item “snack candy” is written in the designated information database 137, the information management unit 125 writes it in association with the attribute “food”.
- the addition symbol “+” is added after “ ⁇ 180 + ⁇ 220” which is the designated information, and the amount “ ⁇ 150” of the payment item “snack candy” is written. That is, the information management unit 125 writes “ ⁇ 180 + ⁇ 220 + ⁇ 150” as the designation information associated with the attribute “food” in the designation information database 137.
- the amount “ ⁇ 150” of the payment item “snack confectionery” that has been touch-slid by the user is captured as designation information for the attribute “food”.
- the information management unit 125 determines whether the currently set attribute information is registered in the temporary storage area 134. When it is determined that the attribute information is registered in the temporary storage area 134, the information management unit 125 adds the designation symbol “+” following the designation information “ ⁇ 180 + ⁇ 220” written in association with the attribute “food”. , And the amount “ ⁇ 150” of the payment item “snack confectionery” may be written.
- the display control unit 122 adds the amount “ ⁇ 150” of the payment item “snack confectionery”, which is designation information newly acquired by the user, to the total amount “ ⁇ 180 + ⁇ 220” of the payment items “bread” and “butter”. The total amount “ ⁇ 180 + ⁇ 220 + ⁇ 150” is displayed in the capture data display area G13.
- FIG. 8A shows an animation image displayed when the logo mark H displayed in the designated information operation area G11 is taken in as designated information.
- the image of the designation information “logo mark H1” moves from the designation information operation area G11 to the capture data display area G13, the image of the designation information “logo mark H1” is gradually enlarged.
- an animation image that is gradually reduced is displayed.
- a plurality of “logo mark H1” images are displayed on the information reception screen G1 in time series.
- FIG. 8B shows an animation image displayed when the amount “ ⁇ 180” of the payment item “bread” displayed in the designated information operation area G11 is captured as designated information.
- the image of the designation information “ ⁇ 180” is moved from the designation information operation area G11 to the capture data display area G13, the image of the designation information “ ⁇ 180” is gradually enlarged to the maximum size.
- an animation image that is gradually reduced is displayed.
- a plurality of “ ⁇ 180” images are displayed on the information reception screen G1 in time series.
- FIG. 9 is a flowchart showing basic processing of the mobile terminal 1.
- Step ST1 First, the user operates the operation unit 111 of the touch panel 101 to start the household account book information acquisition application app1.
- the application processing unit 126 of the control unit 102 reads the program of the household account book information acquisition application app1 from the application storage area 135 of the storage unit 103 and executes it. Thereby, the imaging mode is activated.
- the camera control unit 105 activates the camera 104 in the imaging mode.
- the camera 104 shoots a display object that is a subject in accordance with a user's imaging instruction made on the touch panel 101.
- the display object is information described in the receipt.
- Image data generated by the camera 104 is subjected to image processing by the image processing unit 106, and the image processing result is stored in the camera image storage area 131.
- the control unit 102 activates the database creation mode according to the household account book information acquisition application app1.
- the display control unit 122 displays the information reception screen G1 on the display unit 112 of the touch panel 101 in the database mode.
- the display control unit 122 displays the information reception screen G1 shown in FIG. That is, the display control unit 122 displays an image of a receipt photographed in the designated information operation area G11 in the information reception screen G1.
- the receipt image displayed on the display unit 112 of the touch panel 101 is associated with text data and chart data extracted from the image data of the receipt by the image analysis unit 107.
- Step ST3 The display control unit 122 displays the attribute icons Z1 to Z8 in the attribute operation area G12 in the information reception screen G1.
- Step ST4 The display control unit 122 displays a blank display area in the capture data display area G13 in the information reception screen G1. Initially, tabs indicating attributes are not displayed in the capture data display area G13. This is because the attribute is not specified by the user at this point.
- Step ST5 As shown in FIG. 4, an information reception screen G ⁇ b> 1 is displayed on the operation screen of the touch panel 101. Based on the detection result of the operation unit 111 of the touch panel 101, the operation content determination unit 121 determines whether or not a user's contact operation indicating any of the attribute icons Z1 to Z8 has been performed.
- the display control unit 122 displays the finger icon Q1 superimposed on the information reception screen G1.
- Step ST6 If it is determined that the user's contact operation has been performed on the attribute operation area G12 based on the detection result of the operation unit 111 of the touch panel 101, the operation content determination unit 121 determines the attribute icons Z1 to Z1 based on the contact position of the user's finger. It is determined which attribute icon of Z8 is designated by the user. The operation content determination unit 121 determines that the attribute indicated by the attribute icon designated by the user has been set.
- the operation content determination unit 121 determines that the attribute “logo” is set.
- the attribute setting unit 1241 of the information acquisition unit 124 sets the attribute “logo” of the attribute icon Z1 specified by the user as the currently specified attribute.
- the attribute setting unit 1241 sets an additional flag for the attribute information indicating the currently set attribute “logo” in the designation information database 137.
- the attribute setting unit 1241 registers attribute information indicating the currently set attribute “logo” in the temporary storage area 134.
- the display control unit 124 displays a tab indicating the currently set attribute at the upper left of the capture data display area G13. In FIG.
- Step ST8 Based on the detection result of the operation unit 111 of the touch panel 101, the operation content determination unit 121 determines whether or not the user has performed a touch slide operation that specifies a display object area in the specified information operation area G11. When the operation content determination unit 121 detects a user's touch slide operation, the display control unit 122 displays the finger icon Q2 on the contact position of the user's finger so as to be superimposed on the designated information operation area G11 of the information reception screen G1.
- Step ST9 The designation information acquisition unit 1242 of the information acquisition unit 124 executes a designation information database process for acquiring the designation information designated by the user in the receipt image according to the display object area designated by the user's touch slide operation. To do. Details of the specified information database processing will be described later with reference to FIG. (Step ST10)
- the operation content determination unit 121 determines whether or not the user has released the right thumb from the attribute icon Z1 and the contact operation has been released. Until the contact operation by the user is released, the operation content determination unit 121 repeatedly determines whether or not the user has performed a touch slide operation on the designated information operation area G11. When the user releases the contact operation, the flow returns to step ST5.
- step ST11 In step ST5, when the user's touch operation indicating any of the attribute icons Z1 to Z8 is not detected, the operation content determination unit 121 determines whether or not the user's scroll operation has been detected.
- Step ST12 When the operation content determination unit 121 detects a scroll operation of the user, the display control unit 122 scrolls the display content of the information reception screen G1 on the display unit 112 of the touch panel 101 in the scroll direction as illustrated in FIG. .
- Step ST13 On the other hand, when the user's scroll operation is not detected, the operation content determination unit 121 determines whether or not to end the household account book information acquisition application app1 based on the detection result of the operation unit 111 of the touch panel 101.
- Step ST14 When the user operates the determination unit 111 to end the household account book information acquisition application app1, the flow moves to step ST14. For example, when the user performs a touch operation on the attribute icon Z8 in the attribute operation area G11, the operation content determination unit 121 determines to end the household account book information acquisition application app1. Thereafter, the application processing unit 126 refers to the designation information storage area 136 of the storage unit 103 and determines whether or not attribute information and designation information associated with each other are stored. (Step ST15) When the attribute information and the designation information associated with each other are stored in the designation information storage area 136, the application processing unit 126 activates the household account book management application app2.
- Step ST16 The application processing unit 126 reads attribute information and designation information associated with each other from the designation information storage area 136 according to the household account book management application app2.
- Step ST17 The application processing unit 126 executes the household account book management application app2 based on the attribute information and the designation information read from the designation information storage area 136.
- the application processing unit 126 adds the designation information to the associated attribute information and writes it in the household account book data.
- the application processing unit 126 may update the household account book data managed for each updated day based on the date counted by the timing unit 110.
- the image analysis unit 107 may extract the date and time information from the receipt image and write it in the temporary storage area 134 in association with the receipt image data.
- the information management unit 125 reads the date / time information associated with the receipt image data from the temporary storage area 134, and the date / time information is updated. You may make it update household account book data for every date shown.
- FIG. 10 shows processing when the operation procedure shown in FIG. 6 is performed.
- Step ST21 When the operation content determination unit 121 determines that the user's touch slide operation has been performed on the display object area in the specified information operation area G11, the specified information acquisition unit 1242 detects the display object area specified by the user. When the display object area designated by the user's touch slide operation is detected, the designation information acquisition unit 1242 acquires the display objects (text data and chart data) included in the display object area.
- the designated information acquisition unit 1242 displays the display object designated by the user's touch slide operation. Get the text data associated with the area as specified information. As shown in FIG. 6B, the display object including the amount “ ⁇ 180” of the payment item “bread” displayed in the designated information operation area G11 is touched on the attribute icon Z3 with the thumb of the right hand.
- the designation information acquisition unit 1242 receives the text data “ ⁇ 180” associated with the display object area including the amount “ ⁇ 180” of the payment item “bread” designated by the user. get.
- the information management unit 125 refers to the designation information database 137 in the designation information storage area 136 and determines whether or not an additional flag is set for the currently set attribute “food”. That is, the information management unit 125 determines whether or not the designation information associated with the currently set attribute is stored in the designation information database 137. Further, the information management unit 125 determines whether or not attribute information indicating the currently set attribute is registered in the temporary storage area 134. (Step ST23) When the additional flag is not set for the attribute “food” in the designation information database 137 of the designation information storage area 136, the information management unit 125 indicates that the designation information associated with the currently set attribute “food” is the designation information. It is determined that it is not stored in the database 137.
- the information management unit 125 sets an additional flag for the attribute “food”, and writes the designation information “ ⁇ 180” in association with the attribute information in the designation information database 137. If the attribute information indicating the currently set attribute is not registered in the temporary storage area 134, the information management unit 125 stores the specified information associated with the currently set attribute “food” in the specified information database 137. It is determined that it is not stored. Thereafter, the information management unit 125 stores the attribute information indicating the currently set attribute “food” in the temporary storage area 134 and writes the designation information “ ⁇ 180” in the designation information database 137 in association with the attribute information. .
- Step ST24 The display control unit 124 superimposes and displays an animation image displayed when the designation information “ ⁇ 180” moves from the designation information operation area G11 to the capture data display area G13 on the information reception screen G1.
- the animation image shown in FIG. 8B the image of the designation information “ ⁇ 180” is moved in time series from the designation information operation area G11 to the capture data display area G13. That is, when moving from the designated information operation area G11 to the capture data display area G13, the image of the designated information “ ⁇ 180” is displayed to be gradually enlarged and then gradually reduced.
- Step ST25 After displaying the animation image (that is, after the image of the designation information “ ⁇ 180” is moved to the capture data display area G13), the display control unit 124 displays the image of the designation information “ ⁇ 180” in the capture data display area G13. Write to. As shown in FIG. 6B, the designation information “ ⁇ 180” is displayed in the capture data display area G13.
- Step ST26 In the designation information database 137 of the designation information storage area 136, when the additional flag is set for the attribute “food”, the information management unit 125 writes the designation information captured by the user into the designation information database 137.
- step ST21 information shown in FIG. 6C is displayed on the touch panel 101 of the portable terminal 1, and finger icons Q1 and Q2 are displayed by a user operation.
- the user performs a touch operation on the attribute icon Z3 indicating the attribute “food”, and the desired display object area in the designated information operation area G11 (ie, the amount “ ⁇ 220” of the payment item “butter”) is displayed.
- the user's touch slide operation is performed on the (including area).
- the designation information acquisition unit 1242 acquires text data “ ⁇ 220” associated with a desired display object area.
- the information management unit 125 associates with the designation information “ ⁇ 180” already written in association with the currently set attribute “food”, the newly obtained designation information “ Addition designation information indicating that “ ⁇ 220” has been added is written. That is, in the designation information database 137, the addition designation information “ ⁇ 180 + ⁇ 220” is written in association with the attribute “food”.
- the display control unit 122 superimposes and displays an animation image displayed when the newly acquired designation information “ ⁇ 220” moves from the designation information operation area G11 to the capture data display area G13 on the information reception screen G1. . As shown in FIG.
- Step ST28 After displaying the animation image, the display control unit 122 displays the addition designation information indicating that the newly moved designation information “ ⁇ 220” is added to the already moved designation information “ ⁇ 180”. Display in area G13. Thereby, as shown in FIG. 6C, the addition designation information “ ⁇ 180 + ⁇ 220” is displayed in the capture data display area G13.
- the information reception screen G1 including the specified information operation area G11, the attribute operation area G12, and the capture data display area G13 is displayed on the operation screen for capturing the display object (that is, the specified information).
- the user selects one of the attribute icons Z1 to Z8 displayed in the attribute operation area G12 in order to select an attribute that is to be captured of the designation information.
- Selective touch operation when adding the designation information associated with the attribute information, the user may perform the touch slide operation on the desired display object (designation information) while touching the desired attribute icon to capture the desired designation information. .
- the user can easily update the designation information by a simple operation of designating a display object to be updated for each attribute that is an update target of the household account book data.
- the information management unit 125 creates a designation information database 137 that stores display objects (designation information) captured by the user in association with attribute information. Thereby, the application process part 126 can update household account book data for every attribute. Further, the application processing unit 126 displays attribute icons Z1 to Z7 indicating attributes to be updated in the attribute operation area G12 of the information reception screen G1 according to the household account book management application app2. That is, the attribute information item of the designation information database 137 created by the information management unit 125 matches the attribute item of the household account book data to be updated by the application processing unit 126. Thereby, the designation information (text data and chart data) acquired by the information acquisition unit 124 can be used also in the processing of the household account book management application app2.
- a mobile terminal 1A according to Embodiment 2 of the present invention will be described.
- a mobile terminal 1A according to the second embodiment has the same configuration (see FIG. 2) as the mobile terminal 1 according to the first embodiment.
- the text editing application app3 is stored in the application storage area 135 of the storage unit 103.
- the display object storage area 132 of the storage unit 103 stores document data to be processed by the text editing application app3.
- FIG. 11 shows an information reception screen G2 displayed on the touch panel 101 when the text editing application app3 is started.
- the information reception screen G2 executed by the program of the text editing application app3 is displayed on the operation screen of the touch panel 101 when the control unit 102 reads out the document data to be processed for text editing from the display object storage area 132 of the storage unit 103. Is displayed.
- the information reception screen G2 includes a designated information operation area G21 for displaying document data to be processed for text editing, an attribute operation area G22 for displaying attribute icons Z21 to Z26 for specifying attributes, and a user's operation. And a work display area G24 for displaying text input by operating the operation unit 111.
- the attribute operation area G22 is displayed at the top of the operation screen, and the work display area G24 is displayed at the bottom of the operation screen.
- the designated information operation area G21 is displayed at an intermediate position between the attribute operation area G22 and the work display area G24 on the operation screen.
- document data that is a text editing process target is displayed.
- This document data includes the date “20 ** year ** month ** day (month)”, time “9: 00-10: 00”, title “business meeting, minutes”, and the meeting.
- XXX] is displayed.
- the image displayed in the designated information operation area G21 is a part of the document data.
- attribute icons Z21 to Z26 indicating attributes predetermined in the text editing application app3 are displayed.
- attributes of the attribute icons Z21 to Z26 the type of text editing process that can be executed by the text editing application app3 is determined in advance.
- the types of text editing processing that can be executed by the text editing application app3 for example, the format of text data, character shape, font, character color, character size, and display rotation angle can be set.
- the attribute icon Z21 indicates an attribute “bold”, and is an operation icon for changing the character or the like of the designation information to bold.
- the attribute icon Z22 indicates an attribute “underline”, and is an operation icon for adding an underline to the designation information.
- the attribute icon Z23 indicates the attribute “red”, and is an operation icon for changing the color of the character or the like of the designation information to red.
- the attribute icon Z24 indicates an attribute “cut”, and is an operation icon for executing cut editing for cutting specified information.
- the attribute icon Z25 indicates an attribute “copy”, and is an operation icon for executing copy editing that temporarily holds designation information.
- the attribute icon Z26 indicates an attribute “paste”, and is an operation icon for executing paste editing for pasting designation information temporarily held by copy editing.
- the work display area G24 is for accepting user's character input for the document data displayed in the specified information operation area G21.
- the work display area G24 displays key input means such as a QWERTY keyboard or a numeric keypad, and accepts input of characters and numbers corresponding to keys touched by the user.
- key input means such as a QWERTY keyboard or a numeric keypad
- handwritten input by the user may be received.
- the information reception screen G2 is displayed on the operation screen of the touch panel 101, and when the user touches the attribute icons Z21 and Z22 with a finger, the operation content determination unit 121 performs the user's touch operation. Then, the display control unit 122 displays the finger icon Q21 superimposed on the attribute icon Z21, and displays the finger icon Q22 superimposed on the attribute icon Z22.
- the finger icons Q21 and Q22 are displayed on the touch panel 101 at the position where the user's contact operation is detected. Specifically, the user contacts the attribute icon Z21 with the index finger of the right hand and maintains the contact operation.
- the attribute setting unit 1241 of the information acquisition unit 124 of the control unit 102 determines the attribute “bold” indicated by the attribute icon Z21 and the attribute “underline” indicated by the attribute icon Z22 based on the determination result of the operation content determination unit 121. Set as an attribute of the specified information acquired by the user.
- the attribute setting unit 1241 registers the attribute information indicating the currently set attributes “bold” and “underline” in the temporary storage area 134.
- the user touches and slides the "1. XXX [important]" portion of the document data displayed in the designated information operation area G21 with the thumb of the right hand.
- the operation content determination unit 121 detects the user's touch slide operation, and the display control unit 122 displays the finger icon Q23 superimposed on the portion of “1. XXX” [important] of the document data. As illustrated in FIG. 12A, the display control unit 122 moves the finger icon Q ⁇ b> 23 along the finger moving direction in the user's touch slide operation.
- the application processing unit 126 changes the character font to bold in the “1. XXX [important]” portion of the document data, and executes text editing for underlining the character string.
- the display control unit 124 displays the “1. XXX [important]” portion after text editing (that is, a character string to which bold / underline is added as shown in FIG. 12B).
- the information management unit 125 associates the attributes “bold” and “underline” with the designation information “1. XXX” [important] and outputs them to the application processing unit 126.
- the application processing unit 126 edits the designation information “1.XXX” [important] according to the type of text editing indicated by the attributes “bold” and “underline”, and displays the edited designation information as a display control unit. It outputs to 122.
- the display control unit 122 displays the designation information “1. OOOO [[important]” displayed in the designation information operation area G12 by the application processing unit 126. [Important] ”(ie, a character string with bold / underlined characters as shown in FIG. 12B).
- Step ST31 First, the user operates the operation unit 111 of the touch panel 101 to start the text editing application app3.
- the application processing unit 126 of the control unit 102 reads the program of the text editing application app3 from the application storage area 135 of the storage unit 103 and executes it.
- the application processing unit 126 reads the document data to be edited from the display object storage area 132 of the storage unit 103 according to the text editing application app3, and displays the information reception screen G2 on the display unit 112 of the touch panel 101.
- the display control unit 122 displays the information reception screen G2 shown in FIG.
- the display control unit 122 displays the read document data in the designated information operation area G21 in the information reception screen G2.
- the document data to be edited has been selected by the user from a plurality of document data stored in the display object storage area 132.
- the display control unit 122 displays the attribute icons Z21 to Z26 in the attribute operation area G22 in the information reception screen G2.
- the display control unit 122 displays a predetermined work display area G24 on the information reception screen G2.
- Step ST34 Thereby, an information reception screen G2 as shown in FIG. 12 is displayed on the operation screen of the touch panel 101.
- the operation content determination unit 121 determines whether a user's contact operation has been performed on any of the attribute icons Z21 to Z26 based on the detection result of the operation unit 111 of the touch panel 101.
- the display control 122 displays the finger icons Q1 and Q2 superimposed on the information reception screen G2. (Step ST35) If the operation content determination unit 121 determines that the user's contact operation has been performed on the attribute operation area G22 based on the detection result of the operation unit 111 of the touch panel 101, the operation content determination unit 121 determines whether the attribute icons Z21 to Z26 It is determined which of these attribute icons is designated by the user.
- the operation content determination unit 121 determines that the attribute indicated by the attribute icon instructed by the user has been set. For example, when the user touches the attribute icon Z21 with a finger, the operation content determination unit 121 determines that the attribute “bold” is set by the user. When the user touches the attribute icon Z22 with a finger, the operation content determination unit 121 determines that the attribute “underline” is set by the user.
- the attribute setting unit 1241 of the information acquisition unit 124 sets the attribute “bold” of the attribute icon Z21 designated by the user and the attribute “underline” of the attribute icon Z22 as currently set attributes. That is, the attribute setting unit 1241 stores attribute information indicating the currently set attribute “bold” and attribute “underline” in the temporary storage area 134.
- Step ST36 Based on the detection result of the operation unit 111 of the touch panel 101, the operation content determination unit 121 determines whether or not a user's touch slide operation that designates a display object area displayed in the designation information operation area G21 has been performed.
- the display control unit 122 displays a finger icon Q23 on the contact position of the user's finger so as to be superimposed on the designated information operation area G21 of the information reception screen G2.
- the specified information acquisition unit 1242 of the information acquisition unit 124 acquires a display object specified by the user.
- the designation information acquisition unit 1242 acquires the designation information “1.XXX” [important] selected from the document data in the display object area designated by the user's touch slide operation.
- the application processing unit 126 executes text editing of the type indicated by the attribute set by the user on the display object (specified information) specified by the user. In other words, the application processing unit 126 executes text editing on the “1. XXX [important]” portion of the document data while making the character font bold and underlining the character string.
- the display control unit 124 converts the “1. XXX [important]” portion of the document data to “1. XXX [important]” after text editing (as shown in FIG. 12B). / Character string with an underscore).
- Step ST39 When the touch slide operation by the user is not performed on the designated information operation area G21, the operation content determination unit 121 determines whether or not the user releases the contact operation by moving the finger away from the attribute icons Z21 and Z22. . The operation content determination unit 121 repeats the determination of whether or not the user's touch slide operation has been performed until the contact operation by the user is released. When the contact operation by the user is released, the flow returns to step ST34.
- Step ST40 In step ST34, when the user's touch operation indicating any of the attribute icons Z21 to Z26 is not detected, the operation content determination unit 121 determines whether or not a scroll operation by the user has been detected.
- Step ST41 When the operation content determination unit 121 detects the user's scroll operation, the display control unit 122 scrolls the image in the information reception screen G2 displayed on the display unit 112 of the touch panel 101 in the direction specified by the user. As a result, the document data in the designated information operation area G21 shown in FIGS. 11 and 12 is switched, or an image outside the designated information operation area G21 is displayed.
- Step ST42 When the scroll operation by the user is not detected, the operation content determination unit 121 determines whether another user operation has been performed based on the detection result of the operation unit 111 of the touch panel 101. For example, it is determined whether or not the character information input by the user in the work display area G24 is input to the user's specified position in the specified information operation area G21.
- Step ST43 When another user operation is performed in step ST42, the control unit 102 executes processing according to the user operation. For example, when the user inputs a desired character string by a key input operation in the work display area G24, the control unit 102 inputs the input character string at a position specified by the user in the specified information operation area G21.
- Step ST44 When no other user operation is detected in step ST42, the operation content determination unit 121 determines whether to end the text editing application app3 based on the detection result of the operation unit 111 of the touch panel 101.
- Step ST45 When the user operates the operation unit 111 of the touch panel 101 to end the text editing application app3, the control unit 102 ends the application program.
- the attributes “bold” and “underline” are set by the user, and the designation information “1.XXXXX [important]” is displayed from the display object area.
- the designation information “1.XXXXX [important]” is displayed from the display object area.
- the user can execute the type of processing corresponding to the attribute by designating the attribute icon and the display object displayed on the operation screen. Therefore, the operability of the user can be improved, and the user can intuitively operate the portable terminal 1A to execute processing corresponding to the attribute for desired designation information.
- the portable terminal 1B of the third embodiment has the same configuration (see FIG. 2) as the portable terminal 1 of the first embodiment.
- the search application app4 is stored in the application storage area 135 of the storage unit 103.
- the display object storage area 132 of the storage unit 103 stores document data that is a search target of the search application app4.
- FIG. 14 shows an information reception screen G3 displayed on the touch panel 101 when the search application app4 is activated in the mobile terminal 1B.
- the control unit 102 executes the program of the search application app4 and reads the document data to be searched from the display object storage area 132 of the storage unit 103
- the information reception screen G3 is displayed on the operation screen of the touch panel 101.
- the information reception screen G3 includes a designated information operation area G31 for displaying document data to be searched, an attribute operation area G32 for displaying attribute icons Z31 to Z33 for designating predetermined attributes, and a designated information operation.
- the designated information operation area G31 is displayed at the top of the operation screen, and the search result display area G34 is displayed at the bottom of the operation screen. Further, the input operation area G33 and the attribute operation area G32 are respectively displayed at the center position of the operation screen.
- the designated information operation area G31 document data that is a search target by the user is displayed.
- This document data includes a mail document.
- a part of the document data is displayed in the designated information operation area G31.
- the attribute operation area G32 displays attribute icons Z31 to Z33 indicating attributes predetermined in the search application app4.
- attributes of the attribute icons Z31 to Z33 types of search processing that can be executed by the search application app4 are determined in advance.
- the attribute icon Z31 indicates the attribute “text”, and is an operation icon for executing processing for searching text information associated with specified information from document data to be searched.
- the attribute icon Z32 indicates an attribute “similar”, and is an operation icon for executing processing for searching for similar information associated with specified information from document data to be searched.
- the attribute icon Z33 indicates the attribute “image”, and is an operation icon for executing processing for searching image information associated with the designated information from the document data to be searched.
- the information reception screen G3 is displayed on the operation screen of the touch panel 101.
- the operation content determination unit 121 detects the user's touch operation.
- the display control unit 122 displays the finger icon Q31 superimposed on the attribute icon Z31. That is, the finger icon Q31 is displayed at a position where the user's contact operation is detected. Specifically, the user touches the attribute icon Z31 with the thumb of the right hand and maintains the contact operation.
- the attribute setting unit 1241 of the control unit 102 sets the attribute “text” indicated by the attribute icon Z31 as an attribute associated with the specified information based on the determination result of the operation content determination unit 121.
- the attribute setting unit 1241 stores attribute information indicating the currently set attribute “text” in the temporary storage area 134.
- the display control unit 122 displays a tab Tb3 indicating the currently set attribute “text” in the upper left part of the search result display area G34.
- the determination unit 121 detects the user's touch slide operation, and the display control unit 122 displays the finger icon Q32 superimposed on the character string “remote lock”. As illustrated in FIG. 15A, the display control unit 122 moves the finger icon Q ⁇ b> 32 according to the movement direction of the finger in the user's touch slide operation.
- the designation information acquisition unit 1242 of the information acquisition unit 124 of the control unit 102 based on the determination result of the operation content determination unit 121, the character string “remote lock” corresponding to the display object area specified by the user by the touch slide operation. Is acquired as specified information.
- the display control unit 122 may superimpose and display the animation image displayed when the character string “remote lock” moves from the designated information operation area G31 to the input operation area G33 on the information reception screen G3. Good.
- the animation image shown in FIG. 8 may be applied.
- the display control unit 122 displays the character string “remote lock” acquired as the designation information by the user in the input field of the input operation area G33.
- the information management unit 125 associates the designation information “remote lock” with the attribute “text”. And output to the application processing unit 126.
- the application processing unit 126 obtains text information associated with the designated information “remote lock” from the search target information stored in the storage unit 103 or the search target information stored in the external server 5. Execute. For example, the application processing unit 126 obtains, as search results, document data including the designation information “remote lock” and site information explaining the meaning of the designation information “remote lock” from the search target information.
- the display control unit 122 displays the search result obtained by the search process of the application processing unit 126 in the search result display area 34.
- the attribute “text” is set by the user and the designation information “remote lock” is acquired from the display object area
- the attribute “text” is based on the designation information “remote lock”.
- the search process (text search) corresponding to the type of search process indicated by "can be executed. That is, the mobile terminal 1B can execute the type of processing set as an attribute for the processing target (display object) acquired as the designation information.
- the user can execute the type of processing corresponding to the attribute by designating the attribute icon or display object displayed on the operation screen. Therefore, the operability of the user can be improved, and the user can intuitively operate the mobile terminal 1B to execute processing according to the attribute based on desired designation information.
- FIG. 16 is a block diagram illustrating a configuration of the mobile terminal 1C.
- the mobile terminal 1C according to the fourth embodiment includes an information input unit 128 and a determination unit instead of the information acquisition unit 124 and the information management unit 125 included in the mobile terminal 1 according to the first embodiment.
- a search application app5 is stored in the application storage area 135 of the storage unit 103.
- the display object storage area 132 of the storage unit 103 stores document data to be searched by the search application app5.
- the information input unit 128 designates a part of the designated document data as designation information. Accept as. That is, the information input unit 128 inputs the designation information as input information for executing the program of the search application app5.
- the determination unit 129 determines the form type of the designation information received by the information input unit 128 and outputs the determination result to the application processing unit 126.
- the determination unit 129 determines whether the input designation information is text data or image data.
- the application processing unit 126 reads the search application app5 stored in the application storage area 135 of the storage unit 103 based on the determination result of the operation content determination unit 121 and executes the program.
- the application processing unit 126 functions as a search processing unit.
- the application processing unit 126 performs a predetermined type of search processing based on the determination result of the determination unit 129. For example, when the determination unit 129 determines that the specified information is text data, the application processing unit 126 searches for text information associated with the specified information from the search target information. When the determination unit 129 determines that the specified information is image data, the application processing unit 126 searches the search target information for image information associated with the specified information.
- FIG. 17 shows an information reception screen G4 displayed on the touch panel 101 when the search application app5 is activated in the mobile terminal 1C.
- the program of the search application app5 is executed on the portable terminal 1C and the control unit 102 reads out the document data to be searched from the display object storage area 132 of the storage unit 103, the information reception screen G4 is displayed on the operation screen of the touch panel 101. Is done.
- the information reception screen G4 includes a designated information operation area G41 that displays document data that is a search target, an input operation area G43 that receives designation information designated by the user among the document data that is a search target, A search result display area G44 for displaying a search result of the search processing based on the designation information input to the input operation area G43 by executing the program of the search application app5.
- the designated information operation area G41 is displayed at the top of the operation screen, and the search result display area G44 is displayed at the bottom of the operation screen.
- the input operation area G43 is displayed at an intermediate position between the designated information operation area G41 and the search result display area G44.
- document data to be searched is displayed. This document data includes a mail document.
- the input operation area 43 includes an input field F1 for inputting designation information that serves as a search process search key, and a search icon F2 for executing the search process based on the designation information in the input field F1.
- the information reception screen G4 is displayed on the operation screen of the touch panel 101.
- the operation content determination unit 121 performs the user's touch operation.
- the display control unit 122 displays the finger icon Q41 so as to be superimposed on the input field F1. That is, the finger icon Q41 is displayed at a position where the user's contact operation is detected. Specifically, the user contacts the input field F1 with the thumb of the right hand and maintains the contact operation.
- the operation content determination unit 121 is operated. Detects the user's touch slide operation, and the display control unit 122 displays the finger icon Q42 superimposed on the character string “remote lock”. As shown in FIG. 18A, the display control unit 122 moves the finger icon Q42 in accordance with the finger moving direction in the user's touch slide operation. Accordingly, the information input unit 128 of the control unit 102 receives the character string “remote lock” associated with the display object area designated by the user's touch slide operation based on the determination result of the operation content determination unit 121. That is, the information input unit 128 acquires the character string “remote lock”, which is a part of the search target document data specified by the user, as the specified information.
- the display control unit 122 displays an animation image displayed when the character string “remote lock” designated by the user moves from the designated information operation area G41 to the input field F1 of the input operation area G43 on the information reception screen G4. You may make it display it superimposed.
- the animation image shown in FIG. 8 can be applied.
- the display control unit 122 displays the character string “remote lock” acquired as the designation information by the user in the input field F1 of the input operation area G43.
- the information input unit 128 outputs the designation information “remote lock” to the application processing unit 126.
- the application processing unit 126 Based on the designation information “remote lock”, the application processing unit 126 associates the search target information stored in the storage unit 103 or the search target information stored in the external server 5 with the designation information “remote lock”.
- a search process for acquiring the text information or the image information is executed.
- the application processing unit 126 acquires document data including the designation information “remote lock” and site information explaining the meaning of the designation information “remote lock” as search results from the search target information.
- the display control unit 122 displays the search result obtained by the search process of the application processing unit 126 in the search result display area G44.
- Step ST51 First, the user operates the operation unit 111 of the touch panel 101 to start the search application app5.
- the application processing unit 126 of the control unit 102 of the portable terminal 1C reads the program of the search application app5 from the application storage area 135 of the storage unit 103 and executes it.
- Step ST52 The application processing unit 126 reads the search target document data from the display object storage area 132 of the storage unit 103 according to the search application app5, and displays an information reception screen G4 as shown in FIG. 17 on the display unit 112 of the touch panel 101. That is, the display control unit 122 displays the read document data in the designated information operation area G41 in the information reception screen G4.
- the search target document data is designated by the user from a plurality of document data stored in the display object storage area 132.
- the display control unit 122 displays the input field F1 and the search icon F2 in the input operation area 43 in the information reception screen G4. Further, the display control unit 122 displays a search result display area G44 on the information reception screen G4.
- An information reception screen G 4 as shown in FIG. 17 is displayed on the operation screen of the touch panel 101.
- the operation content determination unit 121 determines whether or not the user performs a touch operation on the input field F1 based on the detection result of the operation unit 111 of the touch panel 101. When the operation content determination unit 121 detects a user's contact operation, the display control unit 122 displays the finger icon Q41 superimposed on the input field F1 of the information reception screen G4.
- Step ST55 Based on the detection result of the operation unit 111 of the touch panel 101, the operation content determination unit 121 determines whether or not the user has performed a touch slide operation on the display object area of the designated information operation area G41.
- the display control unit 122 displays a finger icon Q42 superimposed on the designated information operation area G41 of the information reception screen G4 at the contact position of the user's finger.
- the information input unit 128 acquires a display object specified by the user. That is, the information input unit 128 acquires the designation information “remote lock” from the document data in the display object area designated by the user's touch slide operation.
- Step ST57 The display control unit 122 superimposes and displays an animation image displayed when the designation information “remote lock” moves from the designation information operation area G41 to the input field F1 of the input operation area 43 on the information reception screen G4.
- the image of the designated information “remote lock” is gradually enlarged and then displayed so as to be gradually reduced.
- Step ST58 The display control unit 122 displays in the input field F1 of the input operation area G43.
- Step ST59 When the user's touch slide operation is not detected in step ST55, the operation content determination unit 121 determines whether or not the user's finger is released from the input field F1 and the contact operation is released. The operation content determination unit 121 repeats the determination of whether or not the touch slide operation has been detected until the user's contact operation is released. When the user's contact operation is released, the flow returns to step ST54.
- Step ST60 If the user's touch operation on the input field F1 is not detected in step ST54, the operation content determination unit 121 determines whether or not the user's scroll operation is detected.
- Step ST61 When the operation content determination unit 121 detects a scroll operation of the user, the display control unit 122 scrolls the image in the information reception screen G4 displayed on the display unit 112 of the touch panel 101 in the direction specified by the user. Thereby, the document data displayed in the designated information operation area G41 is switched, or an image outside the area of the designated information operation area G41 is displayed.
- Step ST62 When the user's scroll operation is not detected in step ST60, the operation content determination unit 121 determines whether or not the user has touched the search icon F2 based on the detection result of the operation unit 111 of the touch panel 101.
- Step ST63 When the contact operation of the user with respect to the search icon F ⁇ b> 2 is detected, the determination unit 129 determines the type of designation information received by the information input unit 128 and outputs the determination result to the application processing unit 126.
- the designation information “remote lock” is text data
- the form type of the designation information indicating the text data is output to the application processing unit 126.
- Step ST64 The application processing unit 126 determines the type of search processing (that is, the search method) based on the determination result of the determination unit 129. Here, based on the form type of text data, the application processing unit 126 determines to execute text search. In addition, the application processing unit 126 determines to execute a search method (for example, a search method via the Internet) that is determined in advance as a search method for text search. (Step ST65) The application processing unit 126 transmits a search processing execution request using the designation information “remote lock” as a search key to the server 5 connected via the Internet 4 via the wireless communication unit 108.
- a search method for example, a search method via the Internet
- Step ST66 the wireless communication unit 108 receives the search result from the server 5 via the Internet 4 and outputs it to the application processing unit 126.
- the application processing unit 126 instructs the display control unit 122 to display the search result in the search result display area G44. As a result, the search result is displayed in the search result display area G44.
- Step ST67 When it is detected in step ST62 that the user's contact operation with respect to the search icon F2 has not been performed within a predetermined time, the operation content determination unit 121 ends the search application app5 based on the detection result of the operation unit 111 of the touch panel 101. It is determined whether or not.
- Step ST68 When the user operates the operation unit 111 of the touch panel 101 to end the search application app5, the control unit 102 ends the execution of the application program.
- the control unit 102 when the input field F1 is designated by the user and the designation information “remote lock” is acquired from the display object area, the control unit 102 responds to the designation information “remote lock”. Perform search processing. That is, the user can execute processing of a type corresponding to a desired attribute by designating an attribute icon or a display object displayed on the operation screen. Thereby, the operability of the user can be improved, and the user can intuitively operate the mobile terminal 1C to execute processing according to the attribute based on desired designation information.
- the touch panel 101 includes the operation unit 111 and the display unit 112.
- the display unit 112 of the mobile terminal 1 may be a display not equipped with a touch sensor, and operation means such as a mouse, a keyboard, and a switch may be applied as the operation unit 111.
- the household account book information acquisition application app1 and the household account book management application app2 applied to the portable terminal 1 of the first embodiment may be a single application that is executed by the same program. Or household account book information acquisition application app1 is good also as a program installed in the portable terminal 1 previously.
- the user can visually grasp his / her contact operation by displaying a finger icon on the touch position of the user's finger on the touch panel 101.
- a finger icon indicating the contact position (or proximity position) of the user's finger on the touch panel 101 and an icon display indicating the user's touch slide operation or flick operation may be omitted.
- the processing load of the display control in the portable terminal 1 can be reduced.
- the display object area specified by the user, text data or chart data included in the display object area, hatching or translucent color is added, or the character color And visual effect processing on the operation screen such as changing the color of the chart may be performed.
- the portable terminal 1 is an example of an information processing apparatus according to the present invention, and has a computer system mounted therein. For this reason, you may memorize
- the computer system can execute the above-described operation procedure and processing by reading and executing the program from the recording medium.
- the “computer system” includes not only hardware such as a CPU, memory, and peripheral devices, but also software such as an OS (Operating System).
- OS Operating System
- a homepage providing environment or homepage display environment
- a program for realizing each step of the above-described flowchart may be recorded on a computer-readable recording medium, and a program for realizing the functions of the above-described embodiments is recorded on a computer-readable recording medium. May be.
- the estimated value of the shape of the detection target (for example, the user's finger or stylus pen) may be calculated by reading the program recorded on the recording medium into the computer system and executing it.
- the “computer-readable recording medium” refers to a flexible disk, a magneto-optical disk, a ROM, a writable nonvolatile memory such as a flash memory, a portable medium such as a CD-ROM, and a hard disk built in a computer system. And so on.
- the “computer-readable recording medium” is a volatile memory (for example, DRAM) in a computer system serving as a server or a client that can transmit a program via a network such as the Internet, a telephone line, and a communication line. Storage means capable of holding the program for a certain period of time.
- the above program may be stored in a storage device of a computer system, and then the program may be transmitted to another computer system via a transmission medium or a transmission wave propagating through the transmission medium.
- the “transmission medium” for transmitting the program refers to a medium having an information transmission function such as a network (communication network) such as the Internet, a telephone line, and a communication line.
- the above program may realize a part of the above functions.
- the above-mentioned program may be a difference file (difference program) that realizes the above-mentioned functions in combination with a program already recorded in the computer system.
- the present invention can be applied to an information processing apparatus such as a mobile terminal or a smart phone to capture a desired image, and realize analysis processing and editing / search processing of the content with high user operability.
- an information processing apparatus such as a mobile terminal or a smart phone to capture a desired image
- analysis processing and editing / search processing of the content with high user operability.
- the user does not need to learn complicated operation procedures, and can easily perform image capture, analysis processing, and editing / retrieval processing with intuitive operations. Can be applied.
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
Description
本願は、2011年9月16日に日本国に出願された特願2011-202865号に基づき優先権を主張し、その内容をここに援用する。
(ステップST1)
初めに、ユーザはタッチパネル101の操作部111を操作して家計簿情報取得アプリケーションapp1を起動する。制御部102のアプリケーション処理部126は記憶部103のアプリケーション記憶領域135から家計簿情報取得アプリケーションapp1のプログラムを読み出して実行する。これにより、撮像モードが起動する。次に、カメラ制御部105は撮像モードにおいてカメラ104を起動する。カメラ104は、タッチパネル101においてなされたユーザの撮像指示に従って、被写体である表示オブジェクトを撮影する。ここで、表示オブジェクトとはレシートに記載された情報である。カメラ104により生成された画像データは画像処理部106によって画像処理され、その画像処理結果がカメラ画像記憶領域131に格納される。
(ステップST2)
制御部102は、家計簿情報取得アプリケーションapp1に従って、データベース化モードを起動する。表示制御部122は、データベース化モードにおいて情報受付画面G1をタッチパネル101の表示部112に表示する。例えば、表示制御部122は図4に示す情報受付画面G1を表示部112に表示する。つまり、表示制御部122は情報受付画面G1のうち、指定情報操作エリアG11に撮影したレシートの画像を表示する。ここで、タッチパネル101の表示部112に表示されたレシート画像には、画像解析部107によってレシートの画像データから抽出されたテキストデータと図表データとが紐付けられている。
(ステップST3)
表示制御部122は、情報受付画面G1のうち属性操作エリアG12に属性アイコンZ1~Z8を表示する。
(ステップST4)
表示制御部122は、情報受付画面G1のうちキャプチャデータ表示エリアG13に空白の表示エリアを表示する。最初、属性を示すタブなどはキャプチャデータ表示エリアG13には表示されない。この時点では、ユーザにより属性が指定されていないからである。
(ステップST5)
図4に示すとおり、タッチパネル101の操作画面には情報受付画面G1が表示されている。操作内容判定部121は、タッチパネル101の操作部111の検出結果に基づき、属性アイコンZ1~Z8のうちいずれかを指示するユーザの接触操作がなされたか否か判定する。操作内容判定部121がユーザの接触操作を検出すると、表示制御部122は指アイコンQ1を情報受付画面G1に重畳して表示する。
(ステップST6)
タッチパネル101の操作部111の検出結果に基づき、属性操作エリアG12に対してユーザの接触操作がなされたと判定されると、操作内容判定部121はユーザの指の接触位置に基づき、属性アイコンZ1~Z8のうちのいずれの属性アイコンがユーザにより指定されたかを判定する。操作内容判定部121は、ユーザにより指定された属性アイコンが示す属性が設定されたことを判定する。例えば、ユーザが指で属性アイコンZ1に接触操作すると、操作内容判定部121は属性「ロゴ」が設定されたと判定する。情報取得部124の属性設定部1241は、ユーザにより指定された属性アイコンZ1の属性「ロゴ」を現在指定されている属性として設定する。属性設定部1241は、指定情報データベース137において、現在設定されている属性「ロゴ」を示す属性情報に対して追加フラグを設定する。或いは、属性設定部1241は現在設定されている属性「ロゴ」を示す属性情報を一時記憶領域134に登録する。
(ステップST7)
表示制御部124は、キャプチャデータ表示エリアG13の左上部に現在設定されている属性を示すタブを表示する。図5では、現在設定されている属性「ロゴ」を示すタブTb1がキャプチャデータ表示エリアG13に表示される。
(ステップST8)
操作内容判定部121は、タッチパネル101の操作部111の検出結果に基づいて、指定情報操作エリアG11においてユーザが表示オブジェクト領域を指定するタッチスライド操作を行なったか否かを判定する。操作内容判定部121がユーザのタッチスライド操作を検出すると、表示制御部122は情報受付画面G1の指定情報操作エリアG11に重畳して、ユーザの指の接触位置に指アイコンQ2を表示する。
(ステップST9)
情報取得部124の指定情報取得部1242は、ユーザのタッチスライド操作により指定された表示オブジェクト領域に応じて、レシート画像のなかでユーザにより指定された指定情報を取得する指定情報データベース化処理を実行する。この指定情報データベース化処理の詳細については、図10を参照して後述する。
(ステップST10)
ユーザが指定情報操作エリアG11に対してタッチスライド操作を行なっていない場合、操作内容判定部121はユーザが属性アイコンZ1から右手の親指を離し、その接触操作が解除されたか否かを判定する。ユーザによる接触操作が解除されるまで、操作内容判定部121はユーザが指定情報操作エリアG11に対するタッチスライド操作を行なったか否かの判定を繰り返す。ユーザが接触操作を解除すると、フローはステップST5に戻る。
ステップST5において、属性アイコンZ1~Z8のいずれかを指示するユーザの接触操作が検出されない場合、操作内容判定部121はユーザによるスクロール操作が検出されたか否かを判定する。
(ステップST12)
操作内容判定部121によりユーザのスクロール操作が検出されると、図7(a)に示すとおり、表示制御部122はタッチパネル101の表示部112における情報受付画面G1の表示内容をスクロール方向にスクロールする。
(ステップST13)
一方、ユーザのスクロール操作が検出されない場合、操作内容判定部121はタッチパネル101の操作部111の検出結果に基づいて家計簿情報取得アプリケーションapp1を終了するか否かを判定する。
(ステップST14)
ユーザが判定部111を操作して家計簿情報取得アプリケーションapp1を終了させると、フローはステップST14に移行する。例えば、ユーザが属性操作エリアG11の属性アイコンZ8に対して接触操作を行なった場合、操作内容判定部121は家計簿情報取得アプリケーションapp1を終了すると判定する。その後、アプリケーション処理部126は記憶部103の指定情報記憶領域136を参照して、相互に対応付けられた属性情報と指定情報とが格納されているか否かを判定する。
(ステップST15)
指定情報記憶領域136に相互に対応付けられた属性情報と指定情報とが格納されていた場合、アプリケーション処理部126は家計簿管理アプリケーションapp2のプログラムを起動する。
(ステップST16)
アプリケーション処理部126は、家計簿管理アプリケーションapp2に従い、指定情報記憶領域136から相互に対応付けられた属性情報と指定情報とを読み出す。
(ステップST17)
アプリケーション処理部126は、指定情報記憶領域136から読み出した属性情報と指定情報とに基づき、家計簿管理アプリケーションapp2のプログラムを実行する。アプリケーション処理部126は、指定情報を対応付けられた属性情報に追加して家計簿データに書き込む。アプリケーション処理部126は、計時部110によって計時される日付に基づき、更新される日毎に管理される家計簿データを更新してもよい。レシート画像に日時情報が含まれている場合、画像解析部107がレシート画像から日時情報を抽出して、そのレシート画像データと対応付けて一時記憶領域134に書き込むようにしてもよい。家計簿データを指定情報データベース137から読み出した指定情報で属性毎に更新する場合、情報管理部125はレシート画像データと対応付けられている日時情報を一時記憶領域134から読み出して、当該日時情報が示す日付毎に家計簿データを更新するようにしてもよい。
(ステップST21)
操作内容判定部121が指定情報操作エリアG11における表示オブジェクト領域に対してユーザのタッチスライド操作が行なわれたことを判定すると、指定情報取得部1242はユーザにより指定された表示オブジェクト領域を検出する。ユーザのタッチスライド操作により指定された表示オブジェクト領域を検出すると、指定情報取得部1242は当該表示オブジェクト領域に含まれる表示オブジェクト(テキストデータや図表データ)を取得する。例えば、指定情報操作エリアG11に表示されている文字部分が画像解析部107によって抽出されたテキストデータと紐付けられている場合、指定情報取得部1242はユーザのタッチスライド操作により指定された表示オブジェクト領域と紐付けられているテキストデータを指定情報として取得する。図6(b)に示すとおり、ユーザが右手の親指で属性アイコンZ3に接触操作しており、指定情報操作エリアG11に表示されている支払項目「食パン」の金額「¥180」を含む表示オブジェクト領域を指定するタッチスライド操作を行うと、指定情報取得部1242はユーザにより指定された支払項目「食パン」の金額「¥180」を含む表示オブジェクト領域に紐付けられたテキストデータ「¥180」を取得する。
(ステップST22)
情報管理部125は、指定情報記憶領域136の指定情報データベース137を参照して、現在設定されている属性「食品」に対して追加フラグが設定されているか否かを判定する。つまり、情報管理部125は、現在設定されている属性に対応付けられている指定情報が指定情報データベース137に格納されているか否かを判定する。また、情報管理部125は現在設定されている属性を示す属性情報が一時記憶領域134に登録されているか否かを判定する。
(ステップST23)
指定情報記憶領域136の指定情報データベース137において属性「食品」に対して追加フラグが設定されていない場合、情報管理部125は現在設定されている属性「食品」に対応付けられる指定情報が指定情報データベース137に格納されていないと判定する。その後、情報管理部125は属性「食品」に対して追加フラグを設定するとともに、その属性情報に指定情報「¥180」を対応付けて指定情報データベース137に書き込む。また、現在設定されている属性を示す属性情報が一時記憶領域134に登録されていない場合、情報管理部125は現在設定されている属性「食品」に対応付けられる指定情報が指定情報データベース137に格納されていないと判定する。その後、情報管理部125は現在設定されている属性「食品」を示す属性情報を一時記憶領域134に記憶するとともに、その属性情報に対応付けて指定情報「¥180」を指定情報データベース137に書き込む。
(ステップST24)
表示制御部124は、指定情報「¥180」が指定情報操作エリアG11からキャプチャデータ表示エリアG13に移動するときに表示されるアニメーション画像を情報受付画面G1に重畳して表示する。図8(b)に示すアニメーション画像では、指定情報「¥180」の画像が指定情報操作エリアG11からキャプチャデータ表示エリアG13に時系列に移動している。即ち、指定情報操作エリアG11からキャプチャデータ表示エリアG13まで移動する際、指定情報「¥180」の画像が徐々に拡大され、その後、徐々に縮小されるように表示される。
(ステップST25)
上記のアニメーション画像を表示した後(即ち、指定情報「¥180」の画像がキャプチャデータ表示エリアG13に移動した後)、表示制御部124は指定情報「¥180」の画像をキャプチャデータ表示エリアG13に書き込む。図6(b)に示すように、指定情報「¥180」がキャプチャデータ表示エリアG13に表示される。
指定情報記憶領域136の指定情報データベース137において、属性「食品」に対して追加フラグが設定されている場合、情報管理部125はユーザにより取り込まれた指定情報を指定情報データベース137に書き込む。ステップST21において、携帯端末1のタッチパネル101には図6(c)に示す情報が表示され、かつ、ユーザ操作により指アイコンQ1、Q2が表示されている。ここで、属性「食品」を示す属性アイコンZ3に対してユーザの接触操作がなされ、かつ、指定情報操作エリアG11における所望の表示オブジェクト領域(即ち、支払項目「バター」の金額「¥220」を含む領域)に対してユーザのタッチスライド操作がなされている。この場合、指定情報取得部1242は所望の表示オブジェクト領域に紐付けられているテキストデータ「¥220」を取得する。情報管理部125は、指定情報データベース137において、現在設定されている属性「食品」に対応付けられて既に書き込まれている指定情報「¥180」に関連して、新たに取得された指定情報「¥220」を加算したことを示す加算指定情報を書き込む。つまり、指定情報データベース137において、属性「食品」に対応付けて加算指定情報「¥180+¥220」を書き込む。
(ステップST27)
表示制御部122は、新たに取得された指定情報「¥220」が指定情報操作エリアG11からキャプチャデータ表示エリアG13に移動するときに表示されるアニメーション画像を情報受付画面G1に重畳して表示する。図8(b)に示すとおり、このアニメーション画像においても、指定情報「¥220」の画像が徐々に拡大され、その後、次第に縮小されるように表示される。
(ステップST28)
上記のアニメーション画像を表示した後、表示制御部122は新たに移動された指定情報「¥220」が既に移動された指定情報「¥180」に加算されたことを示す加算指定情報をキャプチャデータ表示エリアG13に表示する。これにより、図6(c)に示すとおり、キャプチャデータ表示エリアG13には加算指定情報「¥180+¥220」を表示される。
(ステップST31)
初めに、ユーザはタッチパネル101の操作部111を操作して、テキスト編集アプリケーションapp3を起動する。制御部102のアプリケーション処理部126は、記憶部103のアプリケーション記憶領域135からテキスト編集アプリケーションapp3のプログラムを読み出して実行する。
(ステップST32)
アプリケーション処理部126は、テキスト編集アプリケーションapp3に従って、記憶部103の表示オブジェクト記憶領域132から編集対象の文書データを読み出し、情報受付画面G2をタッチパネル101の表示部112に表示する。ここで、表示制御部122は図12に示す情報受付画面G2を表示部112に表示する。また、表示制御部122は情報受付画面G2のうち指定情報操作エリアG21に読み出した文書データを表示する。尚、編集対象の文書データは表示オブジェクト記憶領域132に保存されている複数の文書データの中からユーザによって選択されたものである。
(ステップST33)
表示制御部122は、情報受付画面G2のうち属性操作エリアG22に属性アイコンZ21~Z26を表示する。また、表示制御部122は情報受付画面G2に所定のワーク表示エリアG24を表示する。
(ステップST34)
これにより、図12に示すような情報受付画面G2がタッチパネル101の操作画面に表示される。操作内容判定部121は、タッチパネル101の操作部111の検出結果に基づき、属性アイコンZ21~Z26のいずれかに対してユーザの接触操作がなされたか否かを判定する。操作内容判定部121がユーザの接触操作を検出した場合、表示制御122は指アイコンQ1、Q2を情報受付画面G2に重畳して表示する。
(ステップST35)
操作内容判定部121は、タッチパネル101の操作部111の検出結果に基づき、属性操作エリアG22に対するユーザの接触操作がなされたと判定した場合、ユーザの指の接触位置に基づき、属性アイコンZ21~Z26のうちのいずれの属性アイコンがユーザにより指示されたか判定する。その後、操作内容判定部121はユーザにより指示された属性アイコンが示す属性が設定されたことを判定する。例えば、ユーザが指で属性アイコンZ21に接触操作すると、操作内容判定部121は属性「太字」がユーザにより設定されたと判定する。ユーザが指で属性アイコンZ22に接触操作すると、操作内容判定部121は属性「下線」がユーザにより設定されたと判定する。情報取得部124の属性設定部1241は、ユーザにより指示された属性アイコンZ21の属性「太字」と、属性アイコンZ22の属性「下線」とを現在設定されている属性として設定する。即ち、属性設定部1241は現在設定されている属性「太字」と属性「下線」とを示す属性情報を一時記憶領域134に記憶する。
操作内容判定部121は、タッチパネル101の操作部111の検出結果に基づいて、指定情報操作エリアG21に表示されている表示オブジェクト領域を指定するユーザのタッチスライド操作がなされたか否かを判定する。表示制御部122は、操作内容判定部121がユーザのタッチスライド操作を検出した場合、情報受付画面G2の指定情報操作エリアG21に重畳して、ユーザの指の接触位置に指アイコンQ23を表示する。
(ステップST37)
情報取得部124の指定情報取得部1242は、ユーザにより指定された表示オブジェクトを取得する。つまり、指定情報取得部1242はユーザのタッチスライド操作により指定された表示オブジェクト領域において、文書データの中から選択された指定情報「1.○○○○○[重要]」を取得する。
(ステップST38)
アプリケーション処理部126は、ユーザにより指定された表示オブジェクト(指定情報)に対して、ユーザにより設定された属性が示す種類のテキスト編集を実行する。つまり、アプリケーション処理部126は文書データの「1.○○○○○[重要]」部分について、文字フォントを太字にするとともに、文字列に下線を引くテキスト編集を実行する。表示制御部124は、文書データの「1.○○○○○[重要]」部分をテキスト編集後の「1.○○○○○[重要]」(図12(b)に示すとおり、太字/下線を付加した文字列)に変更する。
(ステップST39)
ユーザによるタッチスライド操作が指定情報操作エリアG21に対して行なわれない場合、操作内容判定部121はユーザが指を属性アイコンZ21、Z22から離して、その接触操作を解除したか否かを判定する。操作内容判定部121は、ユーザによる接触操作が解除されるまで、ユーザのタッチスライド操作が行なわれたか否かの判定を繰り返す。ユーザによる接触操作が解除された場合、フローはステップST34に戻る。
ステップST34において、属性アイコンZ21~Z26のうちいずれかを指示するユーザの接触操作が検出されない場合、操作内容判定部121はユーザによるスクロール操作を検出したか否かを判定する。
(ステップST41)
操作内容判定部121がユーザのスクロール操作を検出した場合、表示制御部122はタッチパネル101の表示部112に表示されている情報受付画面G2内の画像をユーザの指定方向にスクロールする。これにより、図11、図12に示した指定情報操作エリアG21内の文書データが切り替えられるか、指定情報操作エリアG21の領域外の画像が表示されるようになる。
ユーザによるスクロール操作が検出されない場合、操作内容判定部121はタッチパネル101の操作部111の検出結果に基づき、その他のユーザ操作がなされたか否かを判定する。例えば、ユーザがワーク表示エリアG24において入力した文字情報を指定情報操作エリアG21内のユーザの指定位置に入力したか否か判定される。
(ステップST43)
ステップST42においてその他のユーザ操作が行なわれた場合、制御部102は当該ユーザ操作に応じた処理を実行する。例えば、ユーザがワーク表示エリアG24においてキー入力操作により所望の文字列を入力すると、制御部102は入力文字列を指定情報操作エリアG21においてユーザにより指定された位置に入力する。
ステップST42においてその他のユーザ操作が検出されない場合、操作内容判定部121はタッチパネル101の操作部111の検出結果に基づき、テキスト編集アプリケーションapp3を終了するか否かを判定する。
(ステップST45)
ユーザがタッチパネル101の操作部111を操作して、テキスト編集アプリケーションapp3を終了した場合、制御部102はそのアプリケーションプログラムを終了する。
(ステップST51)
初めに、ユーザはタッチパネル101の操作部111を操作して、検索アプリケーションapp5を起動する。携帯端末1Cの制御部102のアプリケーション処理部126は、記憶部103のアプリケーション記憶領域135から検索アプリケーションapp5のプログラムを読み出して実行する。
(ステップST52)
アプリケーション処理部126は、検索アプリケーションapp5に従って記憶部103の表示オブジェクト記憶領域132から検索対象の文書データを読み出し、図17に示すような情報受付画面G4をタッチパネル101の表示部112に表示する。つまり、表示制御部122は情報受付画面G4のうち指定情報操作エリアG41に読み出した文書データを表示する。検索対象の文書データは、表示オブジェクト記憶領域132に保存されている複数の文書データからユーザによって指定されたものである。
(ステップST53)
表示制御部122は、情報受付画面G4のうち入力操作エリア43に入力欄F1と検索アイコンF2とを表示する。また、表示制御部122は情報受付画面G4に検索結果表示エリアG44を表示する。
(ステップST54)
図17に示すような情報受付画面G4がタッチパネル101の操作画面に表示されている。操作内容判定部121は、タッチパネル101の操作部111の検出結果に基づいて、ユーザが入力欄F1に対して接触操作したか否かを判定する。操作内容判定部121がユーザの接触操作を検出すると、表示制御部122は指アイコンQ41を情報受付画面G4の入力欄F1に重畳して表示する。
操作内容判定部121は、タッチパネル101の操作部111の検出結果に基づいて、ユーザが指定情報操作エリアG41の表示オブジェクト領域にタッチスライド操作を行なったか否かを判定する。操作内容判定部121がユーザのタッチスライド操作を検出すると、表示制御部122はユーザの指の接触位置に指アイコンQ42を情報受付画面G4の指定情報操作エリアG41に重畳して表示する。
(ステップST56)
情報入力部128は、ユーザにより指定された表示オブジェクトを取得する。つまり、情報入力部128はユーザのタッチスライド操作により指定された表示オブジェクト領域において、文書データから指定情報「遠隔ロック」を取得する。
(ステップST57)
表示制御部122は指定情報「遠隔ロック」が指定情報操作エリアG41から入力操作エリア43の入力欄F1に移動するときに表示されるアニメーション画像を情報受付画面G4に重畳して表示する。ここで、指定情報操作エリアG41から入力操作エリアG43の入力欄F1に移動する際、指定情報「遠隔ロック」の画像が徐々に拡大され、その後、次第に縮小されるように表示される。
(ステップST58)
表示制御部122は、入力操作エリアG43の入力欄F1に表示する。
(ステップST59)
ステップST55においてユーザのタッチスライド操作が検出されない場合、操作内容判定部121はユーザの指が入力欄F1から離されて、接触操作が解除されたか否かを判定する。操作内容判定部121は、ユーザの接触操作が解除されるまで、タッチスライド操作が検出されたか否かの判定を繰り返す。ユーザの接触操作が解除されると、フローはステップST54に戻る。
ステップST54において入力欄F1に対するユーザの接触操作を検出しない場合、操作内容判定部121はユーザのスクロール操作を検出したか否かを判定する。
(ステップST61)
操作内容判定部121がユーザのスクロール操作を検出すると、表示制御部122はタッチパネル101の表示部112に表示されている情報受付画面G4内の画像をユーザの指定方向にスクロールする。これにより、指定情報操作エリアG41に表示される文書データが切り替えられたり、指定情報操作エリアG41の領域外の画像が表示されたりする。
ステップST60においてユーザのスクロール操作が検出されない場合、操作内容判定部121はタッチパネル101の操作部111の検出結果に基づいて、ユーザが検索アイコンF2に接触操作したか否かを判定する。
(ステップST63)
検索アイコンF2に対するユーザの接触操作が検出されると、判定部129は情報入力部128で受け付けた指定情報の形態種別を判定し、その判定結果をアプリケーション処理部126へ出力する。ここで、指定情報「遠隔ロック」はテキストデータであるため、当該テキストデータを示す指定情報の形態種別がアプリケーション処理部126に出力される。
(ステップST64)
アプリケーション処理部126は、判定部129の判定結果に基づいて、検索処理の種類(即ち、検索方法)を決定する。ここでは、テキストデータという形態種別に基づいて、アプリケーション処理部126はテキスト検索を実行することを決定する。また、アプリケーション処理部126はテキスト検索の検索方法として予め決められている検索方法(例えば、インターネットを介した検索方法)を実行することを決定する。
(ステップST65)
アプリケーション処理部126は、無線通信部108を介して、インターネット4で接続されるサーバ5に指定情報「遠隔ロック」を検索キーとする検索処理の実行要求を送信する。
(ステップST66)
その後、無線通信部108はサーバ5からインターネット4を介して検索結果を受信して、アプリケーション処理部126へ出力する。アプリケーション処理部126は、検索結果を検索結果表示エリアG44に表示するように表示制御部122に指示する。これにより、検索結果が検索結果表示エリアG44に表示される。
ステップST62において検索アイコンF2に対するユーザの接触操作が一定時間以内に行なわれなかったことを検出すると、操作内容判定部121はタッチパネル101の操作部111の検出結果に基づいて、検索アプリケーションapp5を終了するか否かを判定する。
(ステップST68)
ユーザがタッチパネル101の操作部111を操作して、検索アプリケーションapp5を終了させると、制御部102はアプリケーションプログラムの実行を終了する。
101 タッチパネル
102 制御部
103 記憶部
104 カメラ
105 カメラ制御部
106 画像処理部
107 画像解析部
108 無線通信部
109 音声信号処理部
110 計時部
111 操作部
112 表示部
121 操作内容判定部
122 表示制御部
123 登録部
124 情報取得部
125 情報管理部
126 アプリケーション処理部
127 音声制御部
128 情報入力部
129 判定部
131 カメラ画像記憶領域
132 表示オブジェクト記憶領域
133 プログラム記憶領域
134 一時記憶領域
135 アプリケーション記憶領域
136 指定情報記憶領域
137 指定情報データベース
1241 属性設定部
1242 指定情報取得部
Claims (15)
- タッチパネルを搭載する情報処理装置であって、
操作画面を表示する表示部と、
前記操作画面におけるユーザ操作を受け付ける操作部と、
前記操作画面上の表示オブジェクト領域から指定情報を取得する情報取得部と、
予めユーザにより所望の属性が設定されていた場合、当該属性と指定情報とを対応付けて管理する情報管理部とを具備する情報処理装置。 - 前記表示部を制御して、所望の指定情報を指定するユーザ操作を受け付ける指定情報操作エリアと、所望の属性を設定するユーザ操作を受け付ける属性操作エリアとを前記操作画面に表示する表示制御部と、
前記操作部による前記操作画面上のユーザ操作の検出結果に基づいて、当該ユーザ操作の指示内容を判定する操作内容判定部とを更に具備し、
前記属性操作エリアにおけるユーザ操作により所望の属性が設定されており、かつ、前記指定情報操作エリアにおけるユーザ操作により所望の指定情報が指定された場合、前記情報管理部は前記属性操作エリアの属性と前記指定情報操作エリアの指定情報とを対応付けて管理するようにした請求項1記載の情報処理装置。 - 前記表示操作部は、前記情報取得部により取得された指定情報を転記するキャプチャデータ表示エリアを前記操作画面に表示するよう前記表示部を制御するとともに、指定情報が前記指定情報操作エリアから前記キャプチャデータ表示エリアに移動するようなアニメーション画像を前記操作画面に表示するよう前記表示部を制御するようにした請求項2記載の情報処理装置。
- ユーザにより起動されたアプリケーションプログラムを実行して属性に応じた処理を指定情報に対して実施するアプリケーション処理部を更に具備した請求項1記載の情報処理装置。
- 前記アプリケーション処理部により家計簿管理アプリケーションプログラムが実行される場合、前記操作画面の表示オブジェクト領域には属性に相当する支払項目と指定情報に相当する支払金額とが表示されており、前記情報管理部は属性として設定された支払項目と情報取得部により指定情報として取得された支払金額とを対応付けて管理するようにした請求項4記載の情報処理装置。
- 前記アプリケーション処理部により実行される処理の種類を示す属性が設定される場合、前記情報管理部は処理の種類を示す属性と情報取得部により取得された指定情報とを対応付けて管理するようにした請求項4記載の情報処理装置。
- 前記アプリケーション処理部はテキスト編集アプリケーションプログラムを実行するものであり、属性としてテキスト編集項目が設定されており、当該テキスト編集項目について前記情報取得部が指定情報として取得したテキストデータを編集するようにした請求項6記載の情報処理装置。
- 前記テキスト編集項目として、テキストデータの書式、文字形状、フォント、文字色、文字サイズ、回転角度のうち少なくとも1つを適用するようにした請求項7記載の情報処理装置。
- 前記アプリケーション処理部は、テキスト編集アプリケーションプログラムに従ってテキストデータの一部を指定情報として一時的に保存するようにした請求項7記載の情報処理装置。
- 前記情報取得部は、前記操作画面の表示オブジェクト領域に含まれる文字情報又は図形情報を指定情報として取得するようにした請求項7記載の情報処理装置。
- タッチパネルを搭載する情報処理装置であって、
操作画面を表示する表示部と、
前記操作画面におけるユーザ操作を受け付ける操作部と、
前記表示部を制御して、所望の指定情報を指定するユーザ操作を受け付ける指定情報操作エリアと、ユーザによる情報入力操作を受け付ける入力操作エリアと、所望の属性を設定するユーザ操作を受け付ける属性操作エリアとを前記操作画面に表示する表示制御部と、
前記操作部による前記操作画面上のユーザ操作の検出結果に基づいて、当該ユーザ操作の指示内容を判定する操作内容判定部と、
前記指定情報操作エリアにおけるユーザ操作により所望の指定情報が指定されており、かつ、ユーザが前記入力操作エリアに対して情報入力操作を行なった場合、指定情報を前記入力操作エリアに転記する情報入力部とを具備した情報処理装置。 - 検索アプリケーションプログラムに従って、指定情報を検索キーとして前記指定情報操作エリアに表示された検索対象情報について検索処理を実行する検索処理部を更に具備した請求項11記載の情報処理装置。
- 前記指定情報操作エリアに表示された検索対象情報の種別を判定する判定部を更に具備し、前記検索処理部は前記判定部の判定結果に応じて検索対象情報について検索処理を実行するようにした請求項12記載の情報処理装置。
- タッチパネルを搭載する情報処理装置に適用される情報処理方法であって、
前記タッチパネルに表示された操作画面に対するユーザ操作を受け付け、
前記操作画面上の表示オブジェクト領域から指定情報を取得し、
予めユーザにより所望の属性が設定されていた場合、当該属性と指定情報とを対応付けて管理するようにした情報処理方法。 - ユーザ操作に連動する情報処理機能を実現するコンピュータに適用されるプログラムであって、
操作画面に対するユーザ操作を受け付け、
前記操作画面上の表示オブジェクト領域から指定情報を取得し、
予めユーザにより所望の属性が設定されていた場合、当該属性と指定情報とを対応付けて管理するよう前記コンピュータを制御するプログラム。
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US14/344,881 US20140351709A1 (en) | 2011-09-16 | 2012-09-10 | Information processing device having information management editing function |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2011-202865 | 2011-09-16 | ||
JP2011202865 | 2011-09-16 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2013039025A1 true WO2013039025A1 (ja) | 2013-03-21 |
Family
ID=47883255
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2012/073015 WO2013039025A1 (ja) | 2011-09-16 | 2012-09-10 | 情報管理編集機能を備えた情報処理装置 |
Country Status (3)
Country | Link |
---|---|
US (1) | US20140351709A1 (ja) |
JP (1) | JPWO2013039025A1 (ja) |
WO (1) | WO2013039025A1 (ja) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2015005128A (ja) * | 2013-06-20 | 2015-01-08 | シャープ株式会社 | 情報処理装置、プログラム |
Families Citing this family (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9420251B2 (en) | 2010-02-08 | 2016-08-16 | Nikon Corporation | Imaging device and information acquisition system in which an acquired image and associated information are held on a display |
JP2014194737A (ja) * | 2012-12-11 | 2014-10-09 | Toshiba Tec Corp | 電子レシートシステム、情報処理装置およびプログラム |
KR102063915B1 (ko) * | 2013-03-14 | 2020-01-08 | 삼성전자주식회사 | 사용자 기기 및 그 동작 방법 |
WO2014208575A1 (ja) * | 2013-06-28 | 2014-12-31 | 日本電気株式会社 | 映像監視システム、映像処理装置、映像処理方法および映像処理プログラム |
KR102545768B1 (ko) | 2015-11-11 | 2023-06-21 | 삼성전자주식회사 | 메타 데이터를 처리하기 위한 장치 및 방법 |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH09146679A (ja) * | 1995-11-21 | 1997-06-06 | Nippon Telegr & Teleph Corp <Ntt> | 情報装置および情報の入力方法 |
JP2005302011A (ja) * | 2004-03-24 | 2005-10-27 | Microsoft Corp | スキャン・ドキュメントから電子フォームに取り込むための方法および装置 |
JP2007286677A (ja) * | 2006-04-12 | 2007-11-01 | Nintendo Co Ltd | 文字入力プログラムおよび文字入力装置 |
JP2010140391A (ja) * | 2008-12-15 | 2010-06-24 | Sony Corp | 画像処理装置、画像処理方法および画像処理プログラム |
-
2012
- 2012-09-10 US US14/344,881 patent/US20140351709A1/en not_active Abandoned
- 2012-09-10 JP JP2013533652A patent/JPWO2013039025A1/ja active Pending
- 2012-09-10 WO PCT/JP2012/073015 patent/WO2013039025A1/ja active Application Filing
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH09146679A (ja) * | 1995-11-21 | 1997-06-06 | Nippon Telegr & Teleph Corp <Ntt> | 情報装置および情報の入力方法 |
JP2005302011A (ja) * | 2004-03-24 | 2005-10-27 | Microsoft Corp | スキャン・ドキュメントから電子フォームに取り込むための方法および装置 |
JP2007286677A (ja) * | 2006-04-12 | 2007-11-01 | Nintendo Co Ltd | 文字入力プログラムおよび文字入力装置 |
JP2010140391A (ja) * | 2008-12-15 | 2010-06-24 | Sony Corp | 画像処理装置、画像処理方法および画像処理プログラム |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2015005128A (ja) * | 2013-06-20 | 2015-01-08 | シャープ株式会社 | 情報処理装置、プログラム |
Also Published As
Publication number | Publication date |
---|---|
US20140351709A1 (en) | 2014-11-27 |
JPWO2013039025A1 (ja) | 2015-03-26 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20220011923A1 (en) | Method of dividing screen areas and mobile terminal employing the same | |
US20200110524A1 (en) | Touch screen device, method, and graphical user interface for customizing display of content category icons | |
US9098480B2 (en) | Mobile terminal and method for controlling the mobile terminal to display bookmark icons for an e-book | |
US20180356955A1 (en) | Mobile terminal and object change support method for the same | |
WO2013039025A1 (ja) | 情報管理編集機能を備えた情報処理装置 | |
US9189500B2 (en) | Graphical flash view of documents for data navigation on a touch-screen device | |
US9628423B2 (en) | Electronic sticky note system, information processing terminal, method for processing electronic sticky note, medium storing program, and data structure of electronic sticky note | |
US9535503B2 (en) | Methods and devices for simultaneous multi-touch input | |
US8884893B2 (en) | Mobile terminal and method for controlling the same | |
KR101124083B1 (ko) | 콘텐츠 표시 장치, 텔레비전 수상기, 콘텐츠 표시 방법, 콘텐츠 표시 제어 프로그램을 기록한 컴퓨터 판독가능한 기록 매체 | |
US8060825B2 (en) | Creating digital artwork based on content file metadata | |
US8347224B2 (en) | Content viewing method, content viewing apparatus, and storage medium in which a content viewing program is stored | |
US20140160051A1 (en) | Information processing device, display control method, and recording medium | |
US20080282158A1 (en) | Glance and click user interface | |
US20230004267A1 (en) | Control Method and Apparatus | |
US20100169772A1 (en) | Tabbed content view on a touch-screen device | |
US20110161818A1 (en) | Method and apparatus for video chapter utilization in video player ui | |
JP2010521025A (ja) | 多状態統合パイユーザインターフェイス | |
KR20120131441A (ko) | 이동 단말기 및 그 제어방법 | |
CN101689192A (zh) | 基于对菜单画面的导航来限制搜索范围的方法、设备和计算机程序产品 | |
CN108604154B (zh) | 基于手势显示图形用户界面的方法及电子设备 | |
WO2023078286A1 (zh) | 一种视频显示方法、装置、电子设备和可读存储介质 | |
CN113485813A (zh) | 应用跳转的方法和装置 | |
CN112765500A (zh) | 信息搜索方法及装置 | |
WO2023093590A1 (zh) | 显示控制方法、装置、电子设备及介质 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 12831497 Country of ref document: EP Kind code of ref document: A1 |
|
WWE | Wipo information: entry into national phase |
Ref document number: 14344881 Country of ref document: US |
|
ENP | Entry into the national phase |
Ref document number: 2013533652 Country of ref document: JP Kind code of ref document: A |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 12831497 Country of ref document: EP Kind code of ref document: A1 |