US20160042496A1 - Information processing apparatus, display control method, and storage medium - Google Patents

Information processing apparatus, display control method, and storage medium Download PDF

Info

Publication number
US20160042496A1
US20160042496A1 US14/822,600 US201514822600A US2016042496A1 US 20160042496 A1 US20160042496 A1 US 20160042496A1 US 201514822600 A US201514822600 A US 201514822600A US 2016042496 A1 US2016042496 A1 US 2016042496A1
Authority
US
United States
Prior art keywords
image
enlargement
display
control unit
information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US14/822,600
Other languages
English (en)
Inventor
Hideshi Ichimi
Masayuki Yamada
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Canon Inc
Original Assignee
Canon Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Canon Inc filed Critical Canon Inc
Assigned to CANON KABUSHIKI KAISHA reassignment CANON KABUSHIKI KAISHA ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: ICHIMI, HIDESHI, YAMADA, MASAYUKI
Assigned to CANON KABUSHIKI KAISHA reassignment CANON KABUSHIKI KAISHA ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: ICHIMI, HIDESHI, YAMADA, MASAYUKI
Publication of US20160042496A1 publication Critical patent/US20160042496A1/en
Priority to US16/694,755 priority Critical patent/US11276144B2/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformation in the plane of the image
    • G06T3/40—Scaling the whole image or part thereof
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G06F3/0485—Scrolling or panning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G06F3/04883—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures for inputting data by handwriting, e.g. gesture or text
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G06F3/04886—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/048—Indexing scheme relating to G06F3/048
    • G06F2203/04806—Zoom, i.e. interaction techniques or interactors for controlling the zooming operation

Definitions

  • the present invention relates to an information processing apparatus having an enlargement function of enlarging and displaying an image presented on a display screen of a display apparatus.
  • Some information processing apparatuses including a display apparatus include an enlargement function of enlarging and displaying an image presented on the display apparatus at a desired enlargement rate.
  • the information processing apparatuses can enlarge not only content depending on application software (AP) for photographs and characters but also an entire image displayed on the display screen.
  • AP application software
  • the information processing apparatuses can display all the images displayed on the display screen including objects such as icons for users at a proper size.
  • the users can operate based on the enlarged images. Therefore, accessibility is enhanced, and even weak-eyed users can operate the apparatus without impairing its operation.
  • Japanese Patent Application Laid-Open No. 2012-252370 discusses therein such an information processing apparatus in which part of a captured display screen can be zoomed.
  • the enlargement function generally needs to enlarge an entire image while constantly updating the display screen, and is incorporated at a system level such as operating system (OS) or window system. Therefore, an entire image displayed on a display screen cannot be enlarged and displayed in a case of an information processing apparatus not employing an OS or window system having the enlargement function. Further, the enlargement function is difficult to add later in the OS or window system due to license issue.
  • OS operating system
  • an information processing apparatus is not provided in which when an AP execution screen is enlarged and displayed by the enlargement function, the AP can be operated by an operation of the enlarged image.
  • information processing apparatuses have been recently required which are capable of operating AP by operations used in the enlargement function processing such as drag operation and pinch operation.
  • the present invention is mainly directed to an information processing apparatus realizing an enlargement function without performing system-level customization.
  • An information processing apparatus includes an application unit configured to generate a first image, an enlargement control unit configured to enlarge the first image and to generate a second image including part of the enlarged first image, and a screen display unit configured to display either of the first image and the second image on a predetermined display apparatus.
  • FIG. 1 is a hardware configuration diagram of an information processing apparatus.
  • FIG. 2 is a functional block diagram of a control unit.
  • FIG. 3 is an explanatory diagram of display area switch processing.
  • FIG. 4 is an explanatory diagram of a magnifying glass frame.
  • FIG. 5 is an explanatory diagram of entirely-enlarged display.
  • FIG. 6 is a flowchart of image switch processing.
  • FIGS. 7A to 7C are flowcharts of event processing.
  • FIG. 8 is a flowchart of captured image drawing processing.
  • FIGS. 9A and 9B are flowcharts of enlarged image drawing processing.
  • FIG. 10 is a flowchart of event processing for magnifying glass according to a second exemplary embodiment.
  • FIG. 11 is a flowchart of event processing for entire enlargement according to the second exemplary embodiment.
  • FIG. 12 is a functional block diagram of the control unit according to a third exemplary embodiment.
  • FIG. 13 is a flowchart of event processing for magnifying glass according to the third exemplary embodiment.
  • FIG. 14 is a flowchart of event processing for entire enlargement according to the third exemplary embodiment.
  • FIG. 15 is a diagram illustrating an exemplary screen displayed by a general application.
  • FIG. 16 is a diagram illustrating an exemplary image in which objects are displayed according to a fourth exemplary embodiment.
  • FIG. 17 is a flowchart of event processing according to the fourth exemplary embodiment.
  • FIG. 18 is a flowchart of drawing processing for entire enlargement according to the fourth exemplary embodiment.
  • FIG. 19 is a flowchart of processing of returning an image to an original size by auto-clear according to a fifth exemplary embodiment.
  • FIG. 20 is an explanatory diagram of an enlarged image display method according to a sixth exemplary embodiment.
  • FIG. 1 is a hardware configuration diagram of an information processing apparatus according to the present exemplary embodiments.
  • An information processing apparatus 101 is a computer including a display 119 (display apparatus) such as image forming apparatus or Smartphone.
  • the information processing apparatus 101 does not have an enlargement function at a system level such as OS or window system.
  • the information processing apparatus 101 includes a control unit 110 , a touch panel 118 , and an external memory 120 in addition to the display 119 .
  • the control unit 110 includes a central processing unit (CPU) 111 , a random access memory (RAM) 112 , and a read only memory (ROM) 113 , and controls the operations of the entire information processing apparatus 101 .
  • CPU central processing unit
  • RAM random access memory
  • ROM read only memory
  • control unit 110 includes an input control unit 114 connected to the touch panel 118 , a display control unit 115 connected to the display 119 , and an external memory interface (I/F) 116 connected to the external memory 120 . Further, the control unit 110 includes a communication I/F controller 117 configured to control communication with a network 102 such as local area network (LAN).
  • the CPU 111 , the RAM 112 , the ROM 113 , the input control unit 114 , the display control unit 115 , the external memory I/F 116 , and the communication I/F controller 117 are connected to each other via a system bus B in a communicable manner.
  • the CPU 111 reads computer programs stored in the ROM 113 or the external memory 120 and executes the programs by use of the RAM 112 as a work area thereby to control the operations of the respective units in the information processing apparatus 101 .
  • the ROM 113 is a non-volatile memory and is a storage medium configured to hold various items of data required for the processing in addition to the computer programs.
  • the RAM 112 is a volatile memory and provides a temporary storage area during the processing.
  • the input control unit 114 receives a user's operation by use of an input device, and sends a control signal according to the received operation to the CPU 111 .
  • the touch panel 118 is used as an input device.
  • the input device may employ a character input device such as keyboard or a pointing device such as mouse in addition to the touch panel 118 .
  • the touch panel 118 is a pointing device configured to detect a position touched by a user's finger or stylus pen.
  • the touch panel 118 may employ various types such as resistance film type, capacitance type, surface acoustic wave type, infrared type, electromagnetic induction type, image recognition type, and optical sensor type.
  • the input control unit 114 sends a control signal according to a position detected by the touch panel 118 to the CPU 111 .
  • the CPU 111 performs processing based on the control signal.
  • the CPU 111 performs processing corresponding to a user's operation on the information processing apparatus 101 .
  • the display control unit 115 displays an image on the display 119 under control of the CPU 111 .
  • the display control unit 115 displays an image including objects associated with the processing on the display 119 , for example.
  • the display control unit 115 according to the present exemplary embodiments can display an image at an enlargement rate of 100% on the display 119 and can display an enlarged image at a predetermined enlargement rate.
  • the touch panel 118 is integrally provided on the display screen of the display 119 .
  • the touch panel 118 is configured with transmissivity which does not hinder display on the display 119 .
  • the CPU 111 identifies an object operated (touched) by the user based on the position of the object displayed on the display 119 and the touch position detected on the touch panel 118 , and performs processing associated with the object.
  • a graphical user interface in which the user intuitively operates an image displayed on the display 119 for input can be configured of the touch panel 118 and the display 119 .
  • the external memory 120 is mountable such as hard disk, flexible disk, compact disk (CD), digital versatile disc (DVD), or memory card.
  • the external memory I/F 116 reads data from the mounted external memory 120 and writes data into the external memory 120 under control of the CPU 111 .
  • the communication I/F controller 117 controls communication with other devices connected to the network 102 via the network 102 .
  • the CPU 111 can detect the following operations or states on the touch panel 118 .
  • Touch-down an indicator such as finger contacts the touch panel 118 (touching starts)
  • Touch-up an indicator contacting the touch panel 118 is separated (touching ends)
  • the CPU 111 determines which operation is performed on the touch panel 118 based on the notified information.
  • the CPU 111 determines a movement direction of a finger or stylus pen in Move per vertical component and per horizontal component, on the touch panel 118 based on a change in position coordinates. For example, it is assumed that the user performs a drag operation by touching the touch panel 118 with a finger and sliding the finger. In this case, the CPU 111 first detects Touch-down and its position coordinates, and then detects Move and its position coordinates at predetermined cycles. When a movement distance is a predetermined value or more, the CPU 111 determines that a drag operation has been performed.
  • the touch panel 118 can perform multi-touch for detecting a plurality of positions of Touch-down and Move at the same time.
  • the user may move fingers to reduce or expand a distance between two points in the direction of the line segment connecting the two points from a state in which two fingers or stylus pens contact the touch panel.
  • the operation in which the user moves two fingers close to or away from each other looks like a behavior of gripping or expanding an object by fingers, and is called “pinch operation.” Generally, in many cases, the pinch operation is performed by the thumb and the index finger of the user.
  • the CPU 111 determines that the pinch operation is performed. Further, the CPU 111 can calculate a center point coordinates of a line segment connecting two points for a pinch operation, and a distance between the two points. When a distance between two points increases, the CPU 111 determines that a pinch-out operation has been performed, and when a distance between two points decreases, the CPU 111 determines that a pinch-in operation has been performed.
  • FIG. 2 is a functional block diagram illustrating the functions for display control realized by the control unit 110 in the information processing apparatus 101 .
  • the CPU 111 reads and executes a computer program from a storage medium such as the ROM 113 or the external memory 120 so that each function is formed, but each function may be configured with hardware devices.
  • a screen control unit 201 , an enlargement control unit 301 , an application unit 401 , and a device control unit 501 are formed in the information processing apparatus 101 .
  • the information processing apparatus 101 realizes the enlargement function of enlarging an image display on the display 119 .
  • the enlargement function of the information processing apparatus 101 includes two functions, i.e., “entire enlargement function” and “magnifying glass function.”
  • the entire enlargement function is displaying an enlarged image on the entire display screen of the display 119 .
  • the magnifying glass function is a function of displaying an enlarged image superimposed on a not-enlarged image, on part of the display 119 .
  • the enlarged image is displayed in a smaller frame than the display size of the display 119 , just as caused by magnifying glass.
  • the entire enlargement function will be simply called “entire enlargement” and the magnifying glass function will be simply called “magnifying glass.”
  • the screen control unit 201 manages the AP including the enlargement function, and performs processing of transferring image information about an image to be displayed to the display 119 , processing of notifying AP of an event corresponding to a user's touch operation on the touch panel 118 and the like.
  • the screen control unit 201 includes a screen information control unit 202 , a screen information storage unit 203 , a screen display unit 204 , an input information control unit 205 , and an input information notification unit 206 .
  • the screen information control unit 202 receives an enlargement mode notification that an enlargement mode (enlargement function) is switched, from the device control unit 501 , and stores the enlargement mode notification in the screen information storage unit 203 .
  • the enlargement mode notification includes information about whether the enlargement function is valid, information about enlargement rate, and information whether entire enlargement or magnifying glass is employed.
  • the screen information control unit 202 sends the enlargement mode notification to the enlargement control unit 301 , and instructs the screen display unit 204 to switch a display area in response to an instruction from the enlargement control unit 301 .
  • the screen display unit 204 draws an image to be displayed on the display 119 , in a display area.
  • the display area includes two display areas with the same display size in which images are drawn, respectively.
  • the screen display unit 204 receives an instruction to switch a display area from the screen information control unit 202 , and performs processing of switching a display area to be displayed on the display 119 .
  • FIG. 3 is an explanatory diagram of the display area switching processing.
  • FIG. 3 illustrates the display 119 as a final image transfer destination, and a virtual display area 601 in which an image to be transferred to the display 119 is drawn.
  • the virtual display area 601 is provided in the RAM 112 , for example.
  • the virtual display area 601 is configured of a normal display area 602 and an expansion display area 603 with the same display size (with a width of 1024 pixels and a height of 600 pixels, for example).
  • the display size is determined depending on the display size of the display 119 (with a width of 1024 pixels and a height of 600 pixels, for example).
  • the display 119 displays thereon either of an image drawn in the normal display area 602 and an image drawn in the expansion display area 603 of the virtual display area 601 .
  • the screen display unit 204 draws an image with the X coordinate “0” and the Y coordinate “0” of a start coordinates 604 as a start point in the normal display area 602 .
  • An image drawn in the normal display area 602 is generated by the application unit 401 .
  • the screen display unit 204 draws an image with the X coordinate “1024” and the Y coordinate “0” of a start coordinates 605 as a start point in the expansion display area 603 .
  • An image drawn in the expansion display area 603 is generated in an enlargement display control unit 307 .
  • an AP execution image generated by executing the AP in the application unit 401 is drawn in the normal display area 602
  • an enlarged image of the execution image generated in the enlargement control unit 301 is drawn in the expansion display area 603 .
  • a copy setting screen generated by a copy application is drawn in the normal display area 602 .
  • An enlarged image in which part of the copy setting screen is enlarged is displayed in the expansion display area 603 .
  • the screen display unit 204 transfers either one of an image drawn in the normal display area 602 and an image drawn in the expansion display area 603 to the display 119 in response to a display area switching instruction from the screen information control unit 202 .
  • FIG. 3 illustrates that the enlargement function is valid and an image in the expansion display area 603 is transferred to and displayed on the display 119 .
  • the input information control unit 205 analyzes a control signal according to a user's operation on the touch panel 118 , transforms the control signal into a form of AP-processable event information, and sends the event information to the input information notification unit 206 .
  • the event information indicates an operation or state on the touch panel 118 , and includes coordinate information about a touched position.
  • the input information notification unit 206 receives the event information from the input information control unit 205 .
  • the input information notification unit 206 refers to the enlargement mode notification stored in the screen information storage unit 203 thereby to send the event information to the application unit 401 when the enlargement function is invalid or to send the event information to the enlargement control unit 301 when the enlargement function is valid. Further, the input information notification unit 206 determines that the touch panel 118 is operated when the event information is received, and instructs a timer control unit 503 described below to reset a timer.
  • the enlargement control unit 301 captures an image drawn by the application unit 401 in the normal display area 602 , and enlarges and draws the captured image in the expansion display area 603 .
  • the enlargement control unit 301 includes an enlargement information control unit 302 , an image capture unit 303 , a captured image storage unit 304 , an enlargement input control unit 305 , an enlargement input information storage unit 306 , the enlargement display control unit 307 , and an enlargement display information storage unit 308 .
  • the enlargement information control unit 302 receives an enlargement mode notification from the screen information control unit 202 , and sends the received enlargement mode notification to the enlargement display control unit 307 .
  • the enlargement information control unit 302 determines whether the enlargement function is valid based on the enlargement mode notification, and instructs the screen information control unit 202 to switch a display area depending on a determination result.
  • the image capture unit 303 captures an image drawn by the screen display unit 204 in the normal display area 602 in response to an instruction from the enlargement display control unit 307 , and stores the captured image in the captured image storage unit 304 .
  • the enlargement input control unit 305 receives the event information from the input information notification unit 206 and stores the received event information in the enlargement input information storage unit 306 , and notifies the reception of the event information to the enlargement display control unit 307 .
  • Event information stored in the enlargement input information storage unit 306 includes a type of event such as Touch-down, Touch-up or Move, and position coordinates indicating a touched position.
  • event information items of event information based on a plurality of touched positions may be stored as one piece of event information.
  • the enlargement display control unit 307 mainly performs the following three processings.
  • the enlargement display control unit 307 acquires the event information stored in the enlargement input information storage unit 306 and the information stored in the enlargement display information storage unit 308 .
  • the enlargement display information storage unit 308 stores therein enlargement display information such as enlargement rate for entire enlargement function, display position coordinate information about entire image, enlargement rate for magnifying glass, and display position coordinate information about the magnifying glass.
  • the enlargement display control unit 307 newly calculates an enlargement rate or display position coordinates by use of the acquired enlargement display information, and updates the content in the enlargement display information storage unit 308 .
  • the display position coordinates for magnifying glass and entire enlargement will be described below.
  • the enlargement display control unit 307 always monitors the enlargement display information storage unit 308 . If the enlargement function is valid, the enlargement display control unit 307 instructs the image capture unit 303 to capture an image and acquires a captured image stored in the captured image storage unit 304 . The enlargement display control unit 307 enlarges or downsizes the acquired captured image based on an enlargement rate stored in the enlargement display information storage unit 308 , and gives a drawing instruction to the screen display unit 204 according to the display position coordinate information about the magnifying glass. If the enlargement function is invalid, the enlargement display control unit 307 does not perform the screen display control.
  • the enlargement display control unit 307 receives an enlargement mode notification from the enlargement information control unit 302 , and instructs the enlargement information control unit 302 to switch a display area depending on whether the enlargement function is valid. When it is determined that the enlargement function is invalid, the enlargement display control unit 307 instructs the enlargement information control unit 302 to switch to the normal display area 602 . When it is determined that the enlargement function is valid, the enlargement display control unit 307 instructs the enlargement information control unit 302 to switch to the expansion display area 603 .
  • the application unit 401 is a group of applications configured to generate images to be displayed on the display 119 and operated on the touch panel 118 .
  • the application unit 401 includes a device setting application 402 and a general application 403 .
  • the application unit 401 sends a drawing instruction to the screen display unit 204 so that an image is drawn in the normal display area 602 .
  • the device setting application 402 is AP configured to set various setting values of the information processing apparatus 101 main body according to a use environment mainly by an administrator as a user.
  • the device setting application 402 is used for the settings such as language setting, power saving setting, network setting, and enlargement function setting.
  • the operations of the information processing apparatus 101 are determined according to the setting values.
  • whether the entire enlargement function is valid/invalid, and whether the magnifying glass function is valid/invalid are set as the enlargement function setting.
  • the entire enlargement function and the magnifying glass function are in an exclusive relationship, and only one of them can be set as valid. For example, when the entire enlargement function is set as valid, the magnifying glass function is invalid.
  • the information included in the enlargement mode notification is set according to the enlargement function setting by the device setting application 402 .
  • the general application 403 is AP used by general users and corresponding to the functions of the information processing apparatus 101 . Only one general application 403 is illustrated to simplify the description, but a plurality of general applications 403 may be present.
  • the general applications 403 may employ various APs such as copying, scanning, sending scanned data, storing (box) scanned data, and printing.
  • the device control unit 501 monitors and manages the states of the information processing apparatus 101 .
  • the device control unit 501 includes a device information control unit 502 , the timer control unit 503 , and a device information storage unit 504 .
  • the device information control unit 502 always monitors the device information storage unit 504 , and sends updated content to the screen information control unit 202 when it is detected that the setting values are updated by the device setting application 402 . For example, when the entire enlargement function or the magnifying glass function is set as valid by the device setting application 402 , the device information control unit 502 generates and sends an enlargement mode notification to the screen information control unit 202 .
  • the device information storage unit 504 stores the setting values set by the device setting application 402 , and additionally stores a transition time to the power saving mode or an auto-clear time.
  • the timer control unit 503 manages the power saving mode to which the information processing apparatus 101 transitions when it has not been operated for a certain period of time, or manages auto-clear that returns an image halfway operated by the user to a default image. Specifically, the timer control unit 503 counts up a time, and compares it with a transition time to the power saving mode or an auto-clear time stored in the device information storage unit 504 . When the transition time or the auto-clear time elapses as a result of the comparison, the timer control unit 503 makes a request for the power saving mode or auto-clear, to the device information control unit 502 .
  • the device information control unit 502 When the notification request is received from the timer control unit 503 , the device information control unit 502 notifies the screen information control unit 202 that the processing has entered the auto-clear or the power saving mode. When instructed to reset the timer from the input information notification unit 206 , the timer control unit 503 resets the own timer, and delays the processing such as power saving mode or auto-clear.
  • FIG. 4 is an explanatory diagram of a magnifying glass frame displayed on the display 119 when the magnifying glass function is set as valid.
  • An image in the expansion display area 603 is presented on display 119 .
  • a magnifying glass frame 705 is smaller than the expansion display area 603 and is displayed within the expansion display area 603 , and a display position thereof is represented by center coordinates 701 (x1, y1).
  • the magnifying glass frame 705 is movable within the expansion display area 603 according to a user's drag operation.
  • the magnifying glass frame 705 is moved to be a magnifying glass frame 706 having center coordinates 702 (x2, y2).
  • An image in the expansion display area 603 is drawn with upper left coordinates (1024, 0) in the expansion display area 603 as an origin.
  • the image is the same as an image drawn in the normal display area 602 , and is not enlarged.
  • a magnifying glass image within the magnifying glass frame 705 is drawn with upper left coordinates (0, 0) in the magnifying glass frame 705 as an origin.
  • the center coordinates 701 of the magnifying glass frame 705 is represented by the upper left coordinates (1024, 0) in the expansion display area 603 as an origin.
  • a magnifying glass image drawn in the magnifying glass frame 705 is an image drawn in the expansion display area 603 at the same coordinates as the center coordinates 701 of the magnifying glass frame 705 , which is enlarged according to an enlargement rate of the magnifying glass and part of the enlarged image is clipped.
  • Center coordinates 701 , width 707 , and height 708 of the magnifying glass frame 705 are stored as the display position coordinate information about the magnifying glass frame 705 in the enlargement display information storage unit 308 .
  • the user makes Touch-down at a movement start point 703 (xx1, yy1), makes Move, and Touch-up at a movement end point 704 (xx2, yy2) in order to move the magnifying glass frame 705 to the magnifying glass frame 706 on the display 119 .
  • the coordinates of events in a series of operations such as Touch-down, Move, and Touch-up are notified as coordinates of the virtual display area 601 including the width of the normal display area 602 from the touch panel 118 to the screen control unit 201 .
  • the center coordinates 702 of the magnifying glass frame 706 after movement is obtained so that the magnifying glass frame 706 displays therein a magnifying glass image in which an image drawn in the expansion display area 603 at the same coordinates as the center coordinates 702 is enlarged and part of the enlarged image is clipped.
  • the center coordinates 702 (x2, y2) of the magnifying glass frame 706 after movement can be obtained as follows, for example.
  • the newly-obtained display position coordinate information is stored in the enlargement display information storage unit 308 .
  • y 2 y 1+( yy 2 ⁇ yy 1)
  • FIG. 4 illustrates only the magnifying glass frame 705 at a movement start point and the magnifying glass frame 706 at a movement end point, but a plurality of movement points are actually present during a Move event between the movement start point and the movement end point.
  • the movement points during the Move event are similarly subjected to coordinate calculation and magnifying glass display.
  • the processing for coordinate calculation of the magnifying glass and display of the magnifying glass are performed by the enlargement display control unit 307 .
  • FIG. 5 is an explanatory diagram of entire enlargement display. An entirely-enlarged image is drawn in the expansion display area 603 and is displayed on the display 119 .
  • An enlarged image 801 in which an image displayed in the normal display area 602 is enlarged at a specific enlargement rate is clipped in size displayable on the display 119 and is drawn in the expansion display area 603 .
  • an image in the normal display area 602 is subjected to a pinch-out operation while being displayed on the display 119 so that the enlarged image 801 is acquired.
  • an origin 809 of the enlarged image 801 is ( ⁇ 50, ⁇ 30) relative to the origin of the original image.
  • An image in the range from the position (50, 30) relative to the origin 809 of the enlarged image 801 to a width of 1024 pixels and a height of 600 pixels of the expansion display area 603 is clipped. The clipped image is drawn in the expansion display area 603 .
  • An enlargement rate and an origin 809 of the enlarged image 801 are obtained as follows in a case where the image is enlarged by a pinch-out operation.
  • Two points on the touch panel 118 are touched in a pinch-out operation.
  • the touch position of the first point moves from a pinch start coordinates 802 at the start of the pinch operation to a pinch end coordinates 803 at the end of the pinch operation.
  • the touch position of the second point moves from a pinch start coordinates 804 at the start of the pinch operation to a pinch end coordinates 805 at the end of the pinch operation.
  • the coordinates are input as information including the width of the normal display area 602 of FIG. 3 from the touch panel 118 into the control unit 110 .
  • a distance D 2 between the pinch end coordinates 803 and the pinch end coordinates 805 is longer than a distance D 1 between the pinch start coordinates 802 and the pinch start coordinates 804 . Therefore, the control unit 110 determines that the pinch-out operation gives an instruction to enlarge the image.
  • the distances D 1 and D 2 are expressed in the following equations. Where the pinch start coordinates 802 as (xx11, yy11), the pinch start coordinates 804 as (xx21, yy21), the pinch end coordinates 803 as (xx12, yy12), and the pinch end coordinates 805 as (xx22, yy22),
  • an enlargement rate R 2 of an image by the pinch operation is expressed by the following equation.
  • the enlargement display control unit 307 enlarges the image at the new enlargement rate R 2 .
  • R 2 R 1 ⁇ D 2/ D 1
  • Coordinates (x01, y01) of the origin 809 of the enlarged image 801 are obtained using the new enlargement rate R 2 .
  • an image is enlarged with a center point coordinates of two points for the pinch operation as a start point.
  • coordinates (x1, y1) of the center point are expressed by the following equations subtracting the width of the normal display area 602 .
  • x 1 ( xx 11 ⁇ 1024+ xx 21 ⁇ 1024)/2
  • a distance between the coordinates (x00, y00) of the origin 808 of an image before enlargement and the center point is expressed by the following equations.
  • coordinates (x01, y01) of the origin 809 of the enlarged image 801 is expressed in the following equations.
  • the center point at the start of pinch is rarely the same as the center point at the end of pinch, and a slight deviation actually occurs therebetween. Therefore, the amounts of movement of the center point coordinates are calculated, and a position of the origin 809 of the enlarged image 801 is obtained in consideration of the amounts of movement.
  • the center point coordinates (x1, y1) and (x2, y2) of the center point 806 at the start of pinch and the center point 807 at the end of pinch are expressed by the following equations, respectively.
  • x 1 ( xx 11 ⁇ 1024+ xx 21 ⁇ 1024)/2
  • the amount of movement (Xd, Yd) from the center point 806 to the center point 807 is expressed by the following equations.
  • An image may be enlarged not only by a pinch operation but also by a Move operation.
  • the center point 806 is a start point and the center point 807 is an end point for a Move operation, for example.
  • the amount of movement (Xd, Yd) between the center points 806 and 807 is the amount of movement caused by the Move operation, so that coordinates of the origin 809 of the enlarged image 801 can be calculated, and an area of the image to be displayed in the expansion display area 603 can be determined.
  • the coordinates indicating the positions from a start position of an operation to an end position thereof are continuously input from the touch panel 118 into the control unit 110 during both a pinch operation and a Move operation.
  • the control unit 110 continuously makes coordinate calculations and displays an entire enlarged image during that time.
  • the enlargement display control unit 307 calculates coordinate position information and displays the entire enlarged image, and stores the enlargement display information including an enlargement rate, origin coordinates of the enlarged image, width of the enlarged image, and height of the enlarged image in an enlargement display information storage unit 308 .
  • the expansion display area 603 is provided on the right side of the normal display area 602 in the present exemplary embodiments, but may be provided below the normal display area 602 . In this case, the expansion display area 603 is provided at the coordinates (0, 600). Also as for an operation in a touch event, a value to which 600 pixels are added in the Y direction is input as coordinates indicating the operation position, from the touch panel 118 into the control unit 110 . Any other calculation equations that enables obtaining coordinates in the similar way may be employed in the present exemplary embodiments.
  • Image display processing on the display 119 performed by the information processing apparatus 101 with the above structure will be described.
  • FIG. 6 is a flowchart illustrating processing of switching an image displayed on the display 119 depending on whether the enlargement function (entire enlargement or magnifying glass) is valid.
  • step S 101 When an enlargement mode notification is received from the device information control unit 502 in step S 101 , in step S 102 , the screen information control unit 202 stores the received enlargement mode notification into the screen information storage unit 203 . In step S 103 , the screen information control unit 202 sends the received enlargement mode notification to the enlargement information control unit 302 .
  • step S 104 the enlargement information control unit 302 receives the enlargement mode notification sent from the screen information control unit 202 , and sends the received enlargement mode notification to the enlargement display control unit 307 .
  • step S 105 the enlargement display control unit 307 receives the enlargement mode notification sent from the enlargement information control unit 302 and stores the received enlargement mode notification in the enlargement display information storage unit 308 .
  • step S 106 the enlargement display control unit 307 determines whether the enlargement function has become valid by the enlargement mode notification.
  • the enlargement display control unit 307 requests the screen information control unit 202 to switch the image displayed on the display 119 to the image in the normal display area 602 via the enlargement information control unit 302 .
  • the enlargement display control unit 307 requests the screen information control unit 202 to switch the image displayed on the display 119 to the image in the expansion display area 603 via the enlargement information control unit 302 .
  • the screen information control unit 202 receives the image switch request from the enlargement information control unit 302 in step S 109 , and determines whether switch to the image in the normal display area 602 is requested in step S 110 .
  • the screen information control unit 202 switches the image displayed on the display 119 to the image in the normal display area 602 via the screen display unit 204 .
  • the screen information control unit 202 switches the image displayed on the display 119 to the image in the expansion display area 603 via the screen display unit 204 .
  • an image in the normal display area 602 and an image in the expansion display area 603 are switched to determine the image to be presented on the display 119 in response to reception of the enlargement mode notification indicating that the enlargement function is switched, from the device control unit 501 .
  • FIGS. 7A to 7C are the flowcharts illustrating event processing in response to input from the touch panel 118 by a user's operation.
  • the input information control unit 205 receives a control signal according to a user's operation on the touch panel 118 in step S 201 , and converts the received control signal into event information and inputs the event information into the input information notification unit 206 in step S 202 .
  • the input information notification unit 206 acquires an enlargement mode notification stored in the screen information storage unit 203 in step S 203 , and determines whether the enlargement function is valid according to the enlargement mode notification in step S 204 . When the enlargement function is invalid (NO in step S 204 ), in step S 205 , the input information notification unit 206 sends the event information to the application unit 401 .
  • the application unit 401 performs processing according to the event information, and terminates the processing.
  • step S 206 when event information is received indicating that a paper select button is selected on the copy setting screen of the copy application, the copy application switches the screen to the paper select screen (not illustrated).
  • the input information notification unit 206 sends the event information to the enlargement input control unit 305 .
  • step S 207 the enlargement input control unit 305 receives the event information from the input information notification unit 206 .
  • the enlargement input control unit 305 stores the received event information in the enlargement input information storage unit 306 in step S 208 , and notifies that the event information is input, to the enlargement display control unit 307 in step S 209 .
  • step S 210 the enlargement display control unit 307 acquires the event information from the enlargement input information storage unit 306 in response to the notification from the enlargement input control unit 305 .
  • step S 211 the enlargement display control unit 307 determines whether the entire enlargement function or the magnifying glass function is valid based on the enlargement mode notification stored in the enlargement display information storage unit 308 .
  • the magnifying glass function is valid (magnifying glass in step S 211 )
  • the enlargement display control unit 307 performs event processing for magnifying glass (see FIG. 7B ).
  • the entire enlargement function is valid (entire enlargement in step S 211 )
  • the enlargement display control unit 307 performs event processing for entire enlargement of image (see FIG. 7C ).
  • FIG. 7B is a flowchart illustrating the event processing for magnifying glass.
  • the enlargement display control unit 307 acquires display position coordinate information about the magnifying glass from the enlargement display information storage unit 308 .
  • the enlargement display control unit 307 determines whether the touch position acquired from the event information is within the display area of the magnifying glass frame obtained from the acquired display position coordinate information. When the touch position is not within the display area of the magnifying glass frame (NO in step S 302 ), the enlargement control unit 301 terminates the event processing for magnifying glass.
  • step S 303 the enlargement display control unit 307 determines whether the event information indicates Move.
  • step S 304 the enlargement display control unit 307 determines whether an event flag is ON.
  • the event flag is ON when Touch-down is performed on the touch panel 118 and OFF when Touch-up is performed on the touch panel 118 . That is, the event flag indicates a touched state on the touch panel 118 .
  • the event flag is set OFF in the initial state.
  • the enlargement control unit 301 terminates the event processing for magnifying glass.
  • the enlargement display control unit 307 calculates a display position coordinates of the magnifying glass frame, and updates the enlargement display information in the enlargement display information storage unit 308 according to a calculation result.
  • step S 307 determines whether the event information indicates Touch-up.
  • step S 308 the enlargement display control unit 307 sets the event flag OFF.
  • step S 309 the enlargement display control unit 307 determines whether the event information indicates Touch-down.
  • step S 310 the enlargement display control unit 307 sets the event flag ON.
  • the enlargement control unit 301 terminates the event processing for magnifying glass.
  • step S 309 When the user performs a drag operation on the magnifying glass frame, Touch-down is detected by first touching on the magnifying glass frame (YES in step S 309 ), and in step S 310 , the event flag is set ON. Subsequently, Move is detected (YES in step S 303 ) and the event flag is set ON (YES in step S 304 ), so that in steps S 305 and S 306 , a display position coordinates of the magnifying glass frame is calculated and the enlargement display information is updated. Thereafter, when the user releases the finger from the magnifying glass frame, Touch-up is detected (YES in step S 307 ), and in step S 308 , the event flag is set OFF.
  • FIG. 7C is a flowchart illustrating the event processing for entire enlargement.
  • an image displayed on the display 119 is enlarged by a user's pinch operation and then a display position is moved by a subsequent drag operation.
  • the enlargement display control unit 307 acquires display position coordinate information about the entire enlargement from the enlargement display information storage unit 308 .
  • the enlargement display control unit 307 which has acquired the display position coordinate information determines whether the event information includes only coordinate information about one point.
  • step S 403 the enlargement display control unit 307 determines that a touch operation is performed on two or more points, and calculates an entire enlargement rate using a plurality of pieces of coordinate information according to the event information.
  • step S 404 the enlargement display control unit 307 calculates a display position coordinates using of a plurality of pieces of coordinate information.
  • step S 405 the enlargement display control unit 307 updates the enlargement display information in the enlargement display information storage unit 308 based on the calculated enlargement rate and display position coordinates.
  • step S 406 the enlargement display control unit 307 determines whether the event information indicates Move.
  • the event information indicates Move (YES in step S 406 )
  • step S 407 the enlargement display control unit 307 determines whether the event flag is ON.
  • the event flag is OFF (NO in step S 407 )
  • the enlargement display control unit 307 terminates the event processing for magnifying glass.
  • step S 408 the enlargement display control unit 307 calculates a display position coordinates for the entire enlargement.
  • step S 409 the enlargement display control unit 307 updates the enlargement display information in the enlargement display information storage unit 308 based on the calculated display position coordinates.
  • step S 410 the enlargement display control unit 307 determines whether the event information indicates Touch-up. When the event information indicates Touch-up (YES in step S 410 ), in step S 411 , the enlargement display control unit 307 sets the event flag OFF. When the event information does not indicate Touch-up (NO in step S 410 ), in step S 412 , the enlargement display control unit 307 determines whether the event information indicates Touch-down. When the event information indicates Touch-down (YES in step S 412 ), in step S 413 , the enlargement display control unit 307 sets the event flag ON. When the event information does not indicate Touch-down (NO in step S 412 ), the enlargement control unit 301 terminates the event processing for entire enlargement.
  • step S 402 Since the user first performs a pinch-out operation for performing entire enlargement, event information including coordinate information about two points is detected (NO in step S 402 ). Then, an enlargement rate for the entire enlargement is calculated in step S 403 , display position coordinates are calculated in step S 404 , and the enlargement display information are updated in step S 405 . Subsequently, when the user performs a drag operation on the enlarged screen to move a display position, Touch-down is first detected (YES in step S 412 ), and in step S 413 , the event flag is set ON.
  • step S 406 Move is detected (YES in step S 406 ) and the event flag is set ON (YES in step S 407 ), so that in steps S 408 and S 409 , display position coordinates for the entire enlargement are calculated and the enlargement display information is updated.
  • step S 307 Touch-up is detected (YES in step S 307 ), and in step S 308 , the event flag is set OFF.
  • FIG. 8 is a flowchart illustrating processing of acquiring and drawing a captured image in the expansion display area 603 when the enlargement function is valid. The processing is started when the enlargement function is valid, and is performed at constant time intervals until the information processing apparatus 101 is shut down.
  • step S 501 the enlargement display control unit 307 acquires the enlargement mode notification stored in the enlargement display information storage unit 308 .
  • step S 502 the enlargement display control unit 307 determines whether the enlargement function is valid based on the enlargement mode notification. When the enlargement function is invalid (NO in step S 502 ), the enlargement display control unit 307 acquires the enlargement mode notification again. The enlargement display control unit 307 repeatedly performs the processing until the enlargement function is valid.
  • the enlargement display control unit 307 causes the image capture unit 303 to capture an image drawn in the normal display area 602 .
  • the image capture unit 303 stores the captured image in the captured image storage unit 304 .
  • step S 504 the enlargement display control unit 307 determines whether the entire enlargement function or the magnifying glass function is valid based on the enlargement mode notification.
  • the magnifying glass function is valid (YES in step S 504 )
  • step S 505 the enlargement display control unit 307 performs the drawing processing for magnifying glass.
  • step S 506 the enlargement display control unit 307 performs the drawing processing for entire enlargement.
  • FIG. 9A is a flowchart illustrating the drawing processing for magnifying glass.
  • step S 601 the enlargement display control unit 307 acquires and duplicates the captured image stored in the captured image storage unit 304 .
  • step S 602 the enlargement display control unit 307 acquires the enlargement rate and the display position coordinate information for the magnifying glass stored in the enlargement display information storage unit 308 .
  • step S 603 the enlargement display control unit 307 enlarges the duplicated captured image at the acquired enlargement rate.
  • the enlargement display control unit 307 clips the enlarged captured image in a size of the magnifying glass frame based on the display position coordinate information.
  • step S 605 the enlargement display control unit 307 combines the clipped and enlarged captured image at a position indicated by the display position coordinate information of the not-enlarged captured image.
  • step S 606 the enlargement display control unit 307 draws the combined image in the expansion display area 603 via the screen display unit 204 . Then, the screen control unit 201 switches the display to the expansion display area 603 in the processing illustrated in FIG. 6 so that an image in which the clipped part of the enlarged captured image is superimposed on the captured image is displayed on the display 119 .
  • FIG. 9B is a flowchart illustrating the drawing processing for entire enlargement.
  • step S 701 the enlargement display control unit 307 acquires a captured image stored in the captured image storage unit 304 .
  • step S 702 the enlargement display control unit 307 acquires the enlargement rate and the display position coordinate information for the entire enlargement stored in the enlargement display information storage unit 308 .
  • step S 703 the enlargement display control unit 307 enlarges the captured image according to the acquired enlargement rate.
  • step S 704 the enlargement display control unit 307 clips the enlarged captured image at the size of the display 119 based on the display position coordinate information.
  • step S 705 the enlargement display control unit 307 draws the clipped image in the expansion display area 603 via the screen display unit 204 .
  • the screen control unit 201 switches the display to the expansion display area 603 in the processing illustrated in FIG. 6 so that the clipped part of the enlarged captured image is displayed on the display 119 .
  • the information processing apparatus 101 can enlarge and display an entire image generated by an application without customizing the OS or system. Further, the information processing apparatus 101 enables magnifying glass display for enlarging and displaying part of an image generated by the application.
  • the entire enlargement function is valid, the user can enlarge and downsize the entire screen by a pinch operation, and can change a display position of the enlarged image to any position by a drag operation.
  • the magnifying glass function is valid, the user can move the magnifying glass frame to any position by a drag operation. Thereby, even weak-eyed users can operate the apparatus without impairing its operation, and accessibility is enhanced.
  • the device control unit 501 when the enlargement function setting is changed by the device setting application 402 , the device control unit 501 generates an enlargement mode notification.
  • the generated enlargement mode notification is sent to the enlargement control unit 301 via the screen control unit 201 ( FIG. 2 and FIG. 6 ).
  • the enlargement control unit 301 performs the image capture processing in the normal display area and draws an image in the expansion display area ( FIG. 8 and FIGS. 9A and 9B ), and switches the display to the expansion display area ( FIG. 6 ).
  • the processing subsequent to image capturing may be performed when a pinch-out operation is performed or when the enlargement rate exceeds a predetermined value (such as 100%) by a pinch-out operation, without performing the processing subsequent to image capturing at the time that the entire enlargement function is valid.
  • the processing subsequent to image capturing is performed assuming that the enlargement rate exceeds 100% by a pinch-out operation.
  • the enlargement control unit 301 does not perform the processing in step S 503 when the enlargement function is determined as valid in step S 502 in FIG. 8 .
  • the enlargement control unit 301 determines whether the entire enlargement function or the magnifying glass function is valid, and performs the processing subsequent to step S 503 when the magnifying glass function is valid.
  • the enlargement control unit 301 determines whether the enlargement rate exceeds 100% due to a pinch operation, and performs the processing subsequent to step S 503 when the enlargement rate is determined as over 100%.
  • the enlargement control unit 301 does not perform the processing in step S 108 when the enlargement function is determined as valid in step S 106 in FIG. 6 .
  • the enlargement control unit 301 determines whether the entire enlargement function or the magnifying glass function is valid, and performs the processing in step S 108 when the magnifying glass function is valid. When the entire enlargement function is valid, the enlargement control unit 301 determines whether the enlargement rate exceeds 100% due to a pinch operation. The enlargement control unit 301 performs the processing in step S 108 when it is determined that the enlargement rate exceeds 100%, and requests the screen control unit 201 to switch to the image in the expansion display area.
  • the enlargement rate may employ a value calculated by the enlargement control unit 301 based on the event information notified from the input information notification unit 206 in the screen control unit 201 in step S 403 in FIG. 7C .
  • the processing subsequent to image capturing is performed not when the entire enlargement function is valid but when a pinch-out operation is performed. Therefore, a processing load due to image capturing even when an operation for screen enlargement is not performed can be suppressed.
  • the first exemplary embodiment describes the mechanisms of the entire enlargement display and the magnifying glass display for a screen generated by an application.
  • the first exemplary embodiment does not consider a case in which while an image drawn in the expansion display area 603 is displayed on the display 119 , the user may operate objects such as buttons or lists on the screen. Therefore, for example, even if the user performs a touch operation of selecting an enlarged button while the entire enlargement function is valid and the entire screen is being enlarged and displayed, an application does not react thereto.
  • objects such as buttons or lists displayed on the enlarged image are operable as in a usual case.
  • the magnifying glass frame when the magnifying glass function is valid, the magnifying glass frame can be moved by a drag operation, and when the entire enlargement function is valid, a display position of an enlarged image can be moved by a drag operation. Further, it is conceivable that a touch operation for selecting a button displayed within the magnifying glass frame and a drag operation for scrolling a list included in an image enlarged and displayed by the entire enlargement function may be performed. Thus, for example, when a drag operation is performed when the entire enlargement function is valid, it is necessary to appropriately determine whether the drag operation is performed to move a display position of the enlarged image or to scroll a list included in the enlarged image.
  • the present exemplary embodiment describes an example in which an intention of the drag operation is determined depending on whether a long press operation of continuing to touch the same position on the touch panel 118 for a predetermined period of time is performed.
  • the hardware configuration and the functional block of the information processing apparatus according to the second exemplary embodiment are the same as those of the information processing apparatus according to the first exemplary embodiment (see FIG. 1 and FIG. 2 ). Differences from the first exemplary embodiment will be described below.
  • FIG. 10 is a flowchart illustrating event processing for magnifying glass. The processing in FIG. 10 is performed instead of the processing in FIG. 7B .
  • step S 800 the enlargement display control unit 307 acquires display position coordinate information about the magnifying glass from the enlargement display information storage unit 308 .
  • step S 801 the enlargement display control unit 307 determines whether the touch position acquired from the event information is within the display area of the magnifying glass frame acquired from the acquired display position coordinate information.
  • step S 802 the enlargement display control unit 307 determines whether the event information includes only coordinate information about one point.
  • the coordinate information is information about one point (YES in step S 802 )
  • step S 803 the enlargement display control unit 307 determines whether the event information indicates Move.
  • step S 804 the enlargement display control unit 307 determines whether an AP event send flag is ON. The AP event send flag becomes ON when a touch event is being sent to the application unit 401 .
  • the AP event send flag when the AP event send flag is ON, all the events received by the enlargement display control unit 307 from the touch panel 118 are sent to AP in the normal display area, and thus the enlarged image is not moved.
  • the AP event send flag when the AP event send flag is OFF, no event is sent from the touch panel to AP in the normal display area, and processing of operating an enlarged image such as moving an enlarged image, is performed.
  • the AP event send flag is set OFF in the initial state.
  • step S 805 the enlargement display control unit 307 determines whether the processing is waiting for an event after Touch-down. As described below in steps S 814 and S 815 , the processing waits for a certain period of time without immediately sending an event to AP or operating an enlarged image in order to determine whether a long press is done when first detecting Touch-down. In the processing in step S 805 , a determination is made as to whether a Move event is received during the standby time.
  • step S 806 the enlargement display control unit 307 compares the coordinates received in the Move event with the coordinates received in the Touch-down event, and determines whether the touch position is moved by a threshold value or more.
  • the enlargement display control unit 307 determines that a long press is not done, and terminates the standby state in step S 807 , and similarly sets the long press flag OFF in step S 809 .
  • the enlargement display control unit 307 sets the long press flag ON, and terminates the processing.
  • the long press flag is a flag for determining whether a user's touch operation is a long press operation of continuing to touch the same position for a predetermined period of time. When the touch position is hardly moved during the standby state after Touch-down is detected, the long press flag is ON. When the touch position is moved by a predetermined amount or more, the long press flag is OFF.
  • step S 810 the enlargement display control unit 307 calculates a display position coordinates of the magnifying glass frame.
  • step S 811 the enlargement display control unit 307 updates the enlargement display information in the enlargement display information storage unit 308 according to a calculation result.
  • the processing in steps S 810 and S 811 are the same as the processing in steps S 305 and S 306 in FIG. 7B .
  • step S 812 the enlargement display control unit 307 determines whether the event information indicates Touch-up.
  • step S 813 the enlargement display control unit 307 determines whether the AP event send flag is ON.
  • the AP event send flag is OFF (NO in step S 813 )
  • the enlargement display control unit 307 terminates the processing.
  • step S 814 the enlargement display control unit 307 determines whether the event information indicates Touch-down.
  • step S 815 the enlargement display control unit 307 waits for a predetermined period of time. When a predetermined period of time elapses, in step S 816 , the enlargement display control unit 307 determines whether the long press flag is ON. When the long press flag is OFF (NO in step S 816 ), the enlargement display control unit 307 terminates the processing. When the event information does not indicate Touch-down (NO in step S 814 ), the enlargement display control unit 307 terminates the processing.
  • the enlargement display control unit 307 determines that sending the event to the AP is to be started.
  • the enlargement display control unit 307 transforms the coordinates indicating the touch position included in the event information into coordinates in the normal display area 602 . That is, the enlargement display control unit 307 transforms coordinates in the expansion display area 603 into coordinates in the normal display area 602 in order to send all the touch events to the application unit 401 .
  • the coordinate transformation will be described with reference to FIG. 4 .
  • the center coordinates 701 (x1, y1) in FIG. 4 indicates the center position of the magnifying glass frame 705 and the movement start point 703 (xx1, yy1) indicates the touch position included in the event information.
  • the enlargement rate of the magnifying glass is R.
  • Coordinates (x, y) in the normal display area 602 after coordinate transformation is carried out is expressed by the following equations by use of the upper left coordinates (1024, 0) in the expansion display area 603 .
  • step S 818 the enlargement display control unit 307 updates the event information based on the coordinates calculated by coordinate transformation, and sends the event information to the application unit 401 via the screen control unit 201 .
  • the user touches the image enlarged and displayed within the magnifying glass frame, and thus the event information by the touch operation is transformed into the event information performed on the image in the normal display area 602 .
  • the transformed event information is notified to the AP which displays the image in the normal display area 602 .
  • the AP performs a predetermined operation based on the notified event information.
  • the user can operate the AP while viewing the image enlarged and displayed within the magnifying glass frame.
  • the enlargement display control unit 307 determines whether the event information indicates Touch-up. When the event information indicates Touch-up (YES in step S 819 ), in step S 820 , the enlargement display control unit 307 sets the AP event send flag OFF. When the event information is other than Touch-up (NO in step S 819 ), in step S 821 , the enlargement display control unit 307 sets the AP event send flag ON. The enlargement display control unit 307 sets the AP event send flag, and then terminates the event processing for magnifying glass.
  • step S 822 the enlargement display control unit 307 transforms the coordinates indicating the touch position included in the event information into coordinates in the normal display area 602 .
  • the enlargement display control unit 307 determines that a touch operation is performed on two or more points.
  • step S 822 the enlargement display control unit 307 transforms the coordinates indicating the touch position included in the event information into coordinates in the normal display area 602 .
  • coordinates (x, y) in the normal display area 602 after coordinate transformation is carried out is expressed by the following equations by use of the upper left coordinates (1024, 0) in the expansion display area 603 .
  • step S 823 the enlargement display control unit 307 updates the event information by the coordinates calculated by coordinate transformation, and sends the updated event information to the application unit 401 via the screen control unit 201 .
  • the event information by the touch operation is transformed into the event information performed on the image in the normal display area 602 .
  • the transformed event information is notified to the AP displaying the image in the normal display area 602 .
  • the AP performs a predetermined operation based on the notified event information. The user can perform an AP operation precisely as displayed, regardless outside or inside the magnifying glass frame.
  • FIG. 10 The flowchart of FIG. 10 will be described by way of specific examples. It is assumed that a send application for sending scanned data is activated in the information processing apparatus 101 . Additionally, it is assumed that an address book screen is displayed to select a data send destination in the send application. It is assumed herein that the magnifying glass function is set as valid and the magnifying glass enlarges and displays part of the address book.
  • step S 814 the user long presses a list in the address book within the magnifying glass frame and then performs a drag operation in order to scroll the list part in the address book displayed within the magnifying glass frame.
  • Touch-down is first detected in step S 814 , and in step S 815 , the processing enters a standby state.
  • step S 803 Move is detected during the standby state.
  • the AP event send flag is OFF and a finger is not moved, and thus the processing proceeds from step S 804 , step S 805 , to step S 806 , and in step S 808 , the long press flag is ON.
  • a Move event keeps being periodically detected until the standby time elapses, and thus the processing in steps S 803 to S 808 are repeatedly performed.
  • the processing proceeds to step S 816 .
  • the long press flag is ON, and thus in steps S 817 and S 818 , the coordinate transformation processing, and the processing of sending an event to the send application after coordinate transformation is carried out are performed.
  • the event information does not indicate Touch-up (NO in step S 819 ), and thus the processing proceeds to step S 821 , where the AP event send flag is ON.
  • step S 803 When the user performs a drag operation after a long press, Move is detected again in step S 803 .
  • the AP event send flag is ON (YES in step S 804 ), and thus in steps S 817 and S 818 , the coordinate transformation processing and the processing of sending the event to the send application after coordinate transformation is carried out are performed.
  • the event information subjected to coordinate transformation based on the touch position is sent to the send application and the list in the address book is scrolled based on the event information in the send application while the user is performing a drag operation after a long press.
  • the image being scrolled by the send application is captured by the enlargement control unit 301 in the processing in FIG. 8 , and is displayed on the display 119 in the processing in FIG. 9B .
  • step S 814 Touch-down is detected in step S 814 and the processing enters a standby state in step S 815 , and then in step S 803 , Move is detected.
  • the AP event send flag is OFF (NO in step S 804 ), and in step S 805 , the processing is in the standby state, and thus in step S 806 , the enlargement display control unit 307 determines whether the touch position is moved by a threshold value or more based on the coordinates received in the Touch-down event and the coordinates received in the Move event.
  • step S 807 the standby state ends, and in step S 809 , the long press flag is OFF.
  • steps S 810 and S 811 the display position coordinates of the magnifying glass frame is calculated and the enlargement display information is updated.
  • the enlargement control unit processes the event to move the magnifying glass frame.
  • the enlargement control unit 301 performs coordinate transformation in consideration of the enlargement rate and notifies the event to the AP. Therefore, the user can perform an AP operation by a drag operation such as scrolling a list displayed by AP, for example.
  • an event not processed by the enlargement control unit 301 is subjected to coordinate transformation in consideration of an enlargement rate, and is notified to AP.
  • the user can perform an AP operation such as enlarging or downsizing an image displayed by AP, for example.
  • FIG. 11 is a flowchart illustrating the event processing for entire enlargement. The processing in FIG. 11 is performed instead of the processing in FIG. 7C .
  • step S 900 the enlargement display control unit 307 acquires display position coordinate information about the entire enlargement from the enlargement display information storage unit 308 .
  • step S 901 the enlargement display control unit 307 which has acquired the display position coordinate information determines whether the event information includes only coordinate information about one point.
  • the enlargement display control unit 307 determines that a touch operation is performed on two or more points, and in step S 902 , determines whether the AP event send flag is ON. As described above, the AP event send flag becomes ON while a touch event is being sent to the application unit 401 . When the AP event send flag is OFF (NO in step S 902 ), in step S 903 , the enlargement display control unit 307 calculates an entire enlargement rate by use of a plurality of pieces of coordinate information according to the event information. In step S 904 , the enlargement display control unit 307 calculates a display position coordinates by use of the pieces of coordinate information.
  • step S 905 the enlargement display control unit 307 updates the enlargement display information in the enlargement display information storage unit 308 based on the calculated enlargement rate and display position coordinates.
  • the processing in steps S 903 to S 905 are the same as the processing in steps S 403 to S 405 in FIG. 7C .
  • step S 906 the enlargement display control unit 307 determines whether the event information indicates Move.
  • the event information indicates Move (YES in step S 906 )
  • step S 907 the enlargement display control unit 307 determines whether the AP event send flag is ON.
  • the AP event send flag is OFF (NO in step S 907 )
  • step S 908 the enlargement display control unit 307 determines whether the processing is waiting for an event after Touch-down.
  • step S 908 the enlargement display control unit 307 determines whether a Move event is received during the standby time.
  • step S 909 the enlargement display control unit 307 compares the coordinates received in the Move event with the coordinates received in the Touch-down event, and determines whether the touch position is moved by a threshold value or more.
  • the enlargement display control unit 307 determines that Touch-down is not a long press, and terminates the standby state, and in step S 911 , sets the long press flag ON.
  • step S 912 the enlargement display control unit 307 calculates a display position coordinates for entire enlargement.
  • step S 913 the enlargement display control unit 307 updates enlargement display information in the enlargement display information storage unit 308 according to a calculation result.
  • the processing in steps S 912 and S 913 are the same as the processing in steps S 408 and S 409 in FIG. 7C .
  • the enlargement display control unit 307 sets the long press flag ON, and terminates the processing.
  • the long press flag is a flag for determining whether a user's touch operation is a long press operation of keeping touching the same position for a predetermined period of time.
  • step S 915 the enlargement display control unit 307 determines whether the event information indicates Touch-up.
  • step S 916 the enlargement display control unit 307 determines whether the AP event send flag is ON.
  • step S 917 the enlargement display control unit 307 determines whether the event information indicates Touch-down.
  • step S 918 the enlargement display control unit 307 waits for a predetermined period of time.
  • step S 919 the enlargement display control unit 307 determines whether the long press flag is ON.
  • the enlargement display control unit 307 determines that sending the event to the AP is to be started, and performs the processing subsequent to step S 920 .
  • step S 919 When the long press flag is OFF (NO in step S 919 ), the enlargement display control unit 307 terminates the processing.
  • the event information does not indicate Touch-down (NO in step S 917 )
  • the enlargement display control unit 307 terminates the processing.
  • step S 920 the enlargement display control unit 307 transforms the coordinates indicating the touch position included in the event information into coordinates in the normal display area 602 . Also when the AP event send flag is ON (YES in step S 902 , YES in step S 907 , or YES in step S 916 ), the enlargement display control unit 307 transforms the coordinates indicating the touch position included in the event information into coordinates in the normal display area 602 . That is, the enlargement display control unit 307 transforms coordinates in the expansion display area 603 into coordinates in the normal display area 602 in order to send all the touch events to the application unit 401 .
  • the coordinate transformation is performed as follows. For example, where an origin of an enlarged image is (x01, y01), an enlargement rate is R, and coordinates indicating a touch position included in event information is (xx1, yy1), coordinates (x, y) in the normal display area 602 after coordinate transformation are expressed by the following equations by use of the upper left coordinates (1024, 0) in the expansion display area 603 .
  • step S 921 the enlargement display control unit 307 updates the event information by the coordinates calculated by coordinate transformation, and sends the event information to the application unit 401 via the screen control unit 201 .
  • the event information by the touch operation is transformed into the event information performed on the image in the normal display area 602 .
  • the transformed event information is notified to the AP displaying the image in the normal display area 602 , and the AP performs a predetermined operation based on the notified event information. Therefore, the user can operate the AP while viewing the enlarged image.
  • the enlargement display control unit 307 determines whether the event information indicates Touch-up. When the event information indicates Touch-up (YES in step S 922 ), in step S 923 , the enlargement display control unit 307 sets the AP event send flag OFF. When the event information indicates other than Touch-up (NO in step S 922 ), in step S 924 , the enlargement display control unit 307 sets the AP event send flag ON. The enlargement display control unit 307 sets the AP event send flag, and then terminates the event processing for entire enlargement.
  • FIG. 11 The flowchart of FIG. 11 will be described by way of specific examples. It is assumed that the send application is activated and an address book screen for selecting a data send destination is displayed in the information processing apparatus 101 . Here, the entire enlargement function is set valid and a list part in the address book is enlarged and displayed.
  • step S 901 Touch-down is first detected in step S 901 , and then in step S 918 , the processing enters a standby state. Subsequently, in step S 906 , Move is detected during the standby state. Since the AP event send flag is OFF and the user does not move the finger at this time, the processing from step S 907 , step S 908 , to step S 909 are performed, and in step S 914 , the long press flag is set ON.
  • a Move event continues to be periodically detected until the standby time elapses, and thus the processing in steps S 906 to S 914 are repeatedly performed.
  • the processing in step S 919 is performed.
  • the long press flag is ON at this time, and thus in steps S 920 and S 921 , the coordinate transformation processing and the processing of sending an event to the send application after coordinate transformation is carried out are performed.
  • the event information does not indicate Touch-up (NO in step S 922 ), and thus the processing in step S 924 is performed and the AP event send flag is set ON.
  • step S 906 Move is detected again.
  • the AP event send flag is ON at this time (YES in step S 907 ), and thus in steps S 920 and S 921 , the coordinate transformation processing and the processing of sending an event to the send application after coordinate transformation are performed.
  • the event information with transformed coordinates based on the touch position is sent to the send application, and the list part in the address book is scrolled based on the event information in the send application.
  • the image being scrolled by the send application is captured in the processing in FIG. 8 , and is displayed on the display 119 in the processing in FIG. 9B .
  • step S 917 Touch-down is detected in step S 917 and the processing enters a standby state in step S 918 , and then in step S 906 , Move is detected.
  • the AP event send flag is OFF (NO in step S 907 ) at this time, and in step S 908 , the processing is in the standby state.
  • step S 909 the enlargement display control unit 307 determines whether the touch position is moved by a threshold value or more based on the coordinates received in the Touch-down event and the coordinates received in the Move event.
  • step S 910 the touch position is moved by the threshold value or more (YES in step S 909 ), and thus in step S 910 , the standby state ends, and in step S 911 , the long press flag is set OFF.
  • steps S 912 and S 913 a display position coordinates for entire enlargement is calculated and the enlargement display information is updated.
  • the enlargement control unit 301 processes the event and moves the enlargement display position.
  • the enlargement control unit 301 does not perform the processing of moving the enlargement display position, and performs coordinate transformation in consideration of the enlargement rate and notifies the event to the AP. Therefore, the user can perform an AP operation by a drag operation such as scrolling a list displayed by AP.
  • the enlargement control unit 301 processes the event and changes the enlargement rate for entire enlargement.
  • the enlargement control unit 301 does not change the enlargement rate for entire enlargement, and performs coordinate transformation in consideration of the enlargement rate and notifies the coordinates to the AP. Therefore, the user can perform an AP operation by a drag operation such as enlarging or downsizing an image displayed by AP.
  • whether an event is sent to AP or an operation is performed on an enlarged image is switched depending on whether a long press operation is first performed.
  • the switching may be performed by other methods. For example, when the user performs Touch-down twice (double tap) at a predetermined short interval, the event may be sent to the AP.
  • a dedicated button for instructing to send an event to AP may be displayed, and when the button is pressed, a touch event performed thereafter may be sent to the AP.
  • a third exemplary embodiment will be described below.
  • the second exemplary embodiment has described the example in which when the user performs a touch operation such as Touch-down or Move while an enlarged screen is being displayed, coordinate transformation is performed in consideration of an enlargement rate or a display position and then event information is notified to a running AP.
  • whether to notify event information to AP is determined depending on whether coordinates of the event are included within a coordinate range in response to a notification of the coordinate range in which the event is processed in the AP, from the running AP.
  • the hardware configuration of the information processing apparatus according to the third exemplary embodiment is similar to that of the information processing apparatus according to the first exemplary embodiment (see FIG. 1 ). Differences from the second exemplary embodiment will be mainly described.
  • FIG. 12 is a functional block diagram of the information processing apparatus 101 according to the third exemplary embodiment.
  • the information processing apparatus 101 includes an AP event send request reception unit 309 within the enlargement control unit 301 in addition to the components according to the first exemplary embodiment illustrated in FIG. 2 . Other components are the same.
  • the AP event send request reception unit 309 receives an AP event send request from the application unit 401 , and stores the AP event send request in the enlargement display information storage unit 308 .
  • the AP event send request includes information about an area range for requesting to send an event to AP.
  • An application (the device setting application 402 or the general application 403 ) in the application unit 401 sends a drawing instruction to the screen display unit 204 . Further, an application (the device setting application 402 or the general application 403 ) in the application unit 401 makes an AP event send request to the AP event send request reception unit 309 according to a display screen.
  • the enlargement display control unit 307 acquires the AP event send request information from the enlargement display information storage unit 308 . Then, the enlargement display control unit 307 determines whether the touch position acquired from the event information is within the area range acquired from the AP event send request information. If the touch position is within the area range, the enlargement display control unit 307 performs coordinate transformation according to a determination result, and performs processing of sending the event to the application unit 401 via the screen control unit 201 . The processing will be described below in detail.
  • FIG. 13 and FIG. 14 are the flowcharts illustrating the event processing in response to input from the touch panel 118 by a user's operation. If there is an event corresponding to the input from the touch panel 118 by a user's operation, the processing in FIG. 7A is first performed. When the magnifying glass function is valid (magnifying glass in step S 211 ) in the determination processing in step S 211 , the enlargement display control unit 307 performs the event processing for magnifying glass illustrated in FIG. 13 . When the entire enlargement function is valid (entire enlargement in step S 211 ), the enlargement display control unit 307 performs the event processing for entire enlargement illustrated in FIG. 14 .
  • FIG. 13 is a flowchart illustrating the event processing for magnifying glass.
  • step S 1000 the enlargement display control unit 307 acquires display position coordinate information about the magnifying glass from the enlargement display information storage unit 308 .
  • step S 1001 the enlargement display control unit 307 determines whether the touch position acquired from the event information is within the display area of the magnifying glass frame acquired from the acquired display position coordinate information.
  • step S 1002 the enlargement display control unit 307 transforms the coordinates indicating the touch position included in the event information into coordinates in the normal display area 602 .
  • the coordinate transformation is the same as the processing in step S 817 in FIG. 10 .
  • step S 1003 the enlargement display control unit 307 acquires the AP event send request information from the enlargement display information storage unit 308 .
  • step S 1004 the enlargement display control unit 307 determines whether the coordinates transformed in step S 1002 is within the area range included in the AP event send request information.
  • step S 1006 the enlargement display control unit 307 updates the event information by the coordinates calculated by coordinate transformation, and sends the updated event information to the application unit 401 via the screen control unit 201 .
  • step S 1005 the enlargement control unit 301 performs the event processing. That is, the processing of updating a magnifying glass display position is performed in the same way as described in steps S 810 and S 811 in FIG. 10 .
  • the enlargement display control unit 307 terminates the event processing for magnifying glass.
  • step S 1007 the enlargement control unit 301 transforms the coordinates indicating the touch position included in the event information into coordinates in the normal display area 602 .
  • the coordinate transformation is the same as the processing in step S 822 in FIG. 10 .
  • step S 1006 the enlargement display control unit 307 updates the event information by the coordinates calculated by coordinate transformation, and sends the updated event information to the application unit 401 via the screen control unit 201 . After sending the event information, the enlargement display control unit 307 terminates the event processing for magnifying glass.
  • FIG. 14 is a flowchart illustrating the event processing for entire enlargement.
  • the enlargement display control unit 307 acquires display position coordinate information about the entire enlargement from the enlargement display information storage unit 308 .
  • the enlargement display control unit 307 which acquires the display position coordinate information transforms the coordinates indicating the touch position included in the event information into coordinates in the normal display area 602 .
  • the coordinate transformation is the same as the processing in step S 920 in FIG. 11 .
  • step S 1102 the enlargement display control unit 307 acquires the AP event send request information from the enlargement display information storage unit 308 .
  • step S 1103 the enlargement display control unit 307 determines whether the coordinates transformed in step S 1101 is within the area range included in the AP event send request information. When the coordinates are within the area range included in the AP event send request information (YES in step S 1103 ), in step S 1105 , the enlargement display control unit 307 updates the event information by the coordinates calculated by coordinate transformation, and sends the updated event information to the application unit 401 via the screen control unit 201 .
  • step S 1104 the enlargement control unit 301 performs the event processing. That is, the enlargement rate is changed, or the display position coordinate update processing or the drawing processing is performed as described in steps S 903 , S 904 , S 905 , S 912 , and S 913 in FIG. 11 .
  • the enlargement display control unit 307 terminates the event processing for magnifying glass.
  • FIG. 15 is a diagram illustrating an exemplary screen displayed by the general application 403 .
  • An area 900 displays therein the list part in the address book stored in the information processing apparatus 101 .
  • the general application 403 performs scroll processing on the list. Even in a case of enlargement display by the magnifying glass function, when a drag operation is performed within the area 900 , the general application 403 does not update the enlargement display position coordinates, but performs the list scroll which is the processing of the general application 403 .
  • the general application 403 may make an AP event send request including coordinate information about coordinates 901 and 902 for specifying the area 900 to the AP event send request reception unit 309 at the same time as sending an instruction to draw the screen information to the screen display unit 204 .
  • an operation performed within the area 900 is subjected to coordinate transformation and is notified to the general application 403 . Therefore, when a drag operation is performed within the area 900 , the list scroll processing is performed.
  • a drag operation is performed outside the area 900 , an enlargement display position is changed.
  • the area included in the AP event send request may be the entire normal display area. In this case, all the events are notified to the application unit 401 .
  • the information processing apparatus 101 can provide the user with operations which the AP demands by instructing from the AP to send events within a predetermined area to the AP even during the enlargement display.
  • FIG. 16 is a diagram illustrating an exemplary image displaying the objects thereon.
  • the hardware configuration and the functional block of the information processing apparatus according to the fourth exemplary embodiment are the same as those of the information processing apparatus according to the first exemplary embodiment (see FIG. 1 and FIG. 2 ).
  • an end button 1001 as an object for returning an image to its original size is provided.
  • the control unit 110 terminates the enlargement function and displays the original-sized image on the display 119 .
  • Four arrow buttons 1002 as objects for displaying hidden parts of an enlarged image are provided.
  • the arrow buttons 1002 are provided on the four sides of the display 119 .
  • the control unit 110 scrolls an image in the direction indicated by the touched arrow button 1002 .
  • a touch on the arrow buttons 1002 provides operability similar to Move.
  • the screen display unit 204 superimposes the end button 1001 and the arrow buttons 1002 on an image drawn in the expansion display area 603 and displays the superimposed image on the display 119 .
  • the arrow buttons 1002 are not displayed when a hidden image is not present in the scroll direction.
  • the control unit 110 determines whether to display or hide the arrow buttons 1002 in response to an instruction from the enlargement display control unit 307 .
  • the enlargement display control unit 307 acquires display position coordinate information, and determines the presence of a hidden image in upward, downward, right, and left directions based on origin coordinates, a width, and a height of the enlarged image and a width and a height of the expansion display area 603 , thereby determining whether to display or hide the arrow buttons 1002 .
  • FIG. 17 illustrates a variant of the flowchart of FIG. 7C , and is a flowchart illustrating the processing when the end button 1001 and the arrow buttons 1002 are touched.
  • the processing in steps S 1201 to S 1212 in FIG. 17 are the same as the processing in steps S 401 to S 412 in FIG. 7C , and thus the description thereof will be omitted.
  • step S 1214 the enlargement display control unit 307 determines whether Touch-down is performed on an arrow button 1002 based on the event information. At this time, the enlargement display control unit 307 determines the arrow button 1002 which is touched.
  • step S 1215 the enlargement display control unit 307 presumes that the image is moved in the direction indicated by the arrow button 1002 by certain coordinates, and calculates display position coordinate information.
  • step S 1216 the enlargement display control unit 307 updates the enlargement display information in the enlargement display information storage unit 308 based on the calculated display position coordinates.
  • step S 1217 the enlargement display control unit 307 determines whether Touch-down is performed on the end button 1001 based on the event information.
  • the enlargement display control unit 307 returns the enlargement rate for entire enlargement to the original factor.
  • the enlargement rate is returned to 100%.
  • step S 1219 the enlargement display control unit 307 updates the enlargement display information in the enlargement display information storage unit 308 based on the enlargement rate.
  • the enlargement display control unit 307 terminates the processing.
  • FIG. 18 is a flowchart illustrating drawing processing for entire enlargement.
  • the processing in FIG. 18 is a variant of the processing in FIG. 9B , and the processing in steps S 1301 to S 1304 in FIG. 18 is the same as the processing in steps S 701 to S 704 in FIG. 9B , and thus the description thereof will be omitted.
  • step S 1305 the enlargement display control unit 307 clipping a captured image at the size of the display 119 determines whether the enlargement rate is higher than 100%.
  • step S 1306 the enlargement display control unit 307 determines display of the end button 1001 .
  • step S 1307 the enlargement display control unit 307 determines display of the upward, downward, right, and left buttons 1002 based on the display position coordinate information about the entirely-enlarged image.
  • the enlargement display control unit 307 determines that the end button 1001 and the arrow buttons 1002 are hidden. In this way, after determining that the end button 1001 and the arrow buttons 1002 are displayed or hidden, in step S 1310 , the enlargement display control unit 307 draws an image in the expansion display area 603 .
  • the screen display unit 204 displays the image drawn in the expansion display area 603 on the display 119 .
  • the screen display unit 204 may display the image drawn in either the normal display area 602 or the expansion display area 603 on the display 119 .
  • the end button 1001 and the arrow buttons 1002 described above are displayed, so that even the information processing apparatus 101 such as Smartphone, which takes practice, can be easily operated.
  • the information processing apparatus 101 may perform the processing such as power saving mode or auto-clear for resetting the screen when the apparatus has not been operated for a predetermined period of time.
  • the hardware configuration and the functional block of the information processing apparatus according to the fifth exemplary embodiment are the same as those of the information processing apparatus according to the first exemplary embodiment (see FIG. 1 and FIG. 2 ).
  • the information processing apparatus 101 measures a user's operation interval by the timer control unit 503 .
  • the timer control unit 503 sends a reset notification requesting the power saving mode or auto-clear to the device information control unit 502 .
  • the device information control unit 502 notifies the power saving mode or auto-clear to the screen control unit 201 in response to the reset notification from the timer control unit 503 .
  • the screen control unit 201 receives the power saving mode or auto-clear notification, and returns an image enlarged and displayed by the enlargement function to its original size, for example.
  • FIG. 19 is a flowchart illustrating the processing of returning an image to its original size by the auto-clear processing. The processing in the flowchart is performed when the screen control unit 201 receives a power saving mode or auto-clear notification. The same processing is performed in other processing such as power saving mode.
  • step S 1402 the screen information control unit 202 makes an auto-clear notification to the application unit 401 .
  • step S 1403 the screen information control unit 202 determines whether the enlargement function is valid based on the enlargement mode notification stored in the screen information storage unit 203 . When the enlargement function is invalid (NO in step S 1403 ), the screen information control unit 202 terminates the processing.
  • step S 1404 the screen information control unit 202 makes an auto-clear notification to the enlargement information control unit 302 .
  • step S 1405 the enlargement information control unit 302 sends an enlargement mode notification to the enlargement display control unit 307 .
  • step S 1406 the enlargement display control unit 307 receives the enlargement mode notification, and determines whether the enlargement function is magnifying glass.
  • step S 1407 the enlargement display control unit 307 sets the magnifying glass at default coordinates.
  • the enlargement display control unit 307 sets the enlargement rate for entire enlargement at 100% and updates the enlargement display information in the enlargement display information storage unit 308 .
  • FIG. 20 is an explanatory diagram of a method for displaying an enlarged image.
  • the hardware configuration and the functional block of the information processing apparatus according to the sixth exemplary embodiment are the same as those of the information processing apparatus according to the first exemplary embodiment (see FIG. 1 and FIG. 2 ).
  • RootWindow at the highest order is in a state to be immediately output to the display, and is configured of a plurality of combined Windows with the same size.
  • RootWindow 1101 is configured of an enlargement function Window 1102 for displaying an enlarged image, a base Window 1103 , and an application Window 1104 .
  • the base Window 1103 and the application Window 1104 are combined with each other thereby to form an image similar to that in the normal display area 602 in FIG. 3 .
  • an enlarged image is displayed only in the normal display area 602 , if an image in the RootWindow 1101 is acquired to obtain a captured image, the enlarged image itself is captured.
  • the expansion display area 603 when RootWindow is acquired, only an enlarged image is not captured and an image in the normal display area 602 can be handled as it is.
  • An image in the base Window 1103 and an image in the application Window 1104 are captured to be combined with each other in order to obtain an original image only in the normal display area 602 .
  • an upper left origin 1105 of a captured image in the application Window 1104 and coordinates 1106 of a captured image in the base Window 1103 are combined with each other thereby to finally generate a captured image of the original image.
  • the captured image acquired by the combination is enlarged and displayed in the enlargement function Window 1102 , and thus the operations equivalent to those in the first exemplary embodiment become possible.
  • the information processing apparatus 101 includes various apparatuses.
  • the information processing apparatus 101 is not limited to a personal computer, PDA, and cell phone, but includes a printer, scanner, facsimile (FAX), copying machine, multifunction peripheral, camera, video camera, and other image viewer.
  • FAX facsimile
  • either of a not-enlarged first image and a second image as the enlarged first image is displayed on a display apparatus thereby to realize an enlargement function without system-level customization.
  • Embodiment(s) of the present invention can also be realized by a computer of a system or apparatus that reads out and executes computer executable instructions (e.g., one or more programs) recorded on a storage medium (which may also be referred to more fully as a ‘non-transitory computer-readable storage medium’) to perform the functions of one or more of the above-described embodiment(s) and/or that includes one or more circuits (e.g., application specific integrated circuit (ASIC)) for performing the functions of one or more of the above-described embodiment(s), and by a method performed by the computer of the system or apparatus by, for example, reading out and executing the computer executable instructions from the storage medium to perform the functions of one or more of the above-described embodiment(s) and/or controlling the one or more circuits to perform the functions of one or more of the above-described embodiment(s).
  • computer executable instructions e.g., one or more programs
  • a storage medium which may also be referred to more fully as a
  • the computer may comprise one or more processors (e.g., central processing unit (CPU), micro processing unit (MPU)) and may include a network of separate computers or separate processors to read out and execute the computer executable instructions.
  • the computer executable instructions may be provided to the computer, for example, from a network or the storage medium.
  • the storage medium may include, for example, one or more of a hard disk, a random-access memory (RAM), a read only memory (ROM), a storage of distributed computing systems, an optical disk (such as a compact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)TM), a flash memory device, a memory card, and the like.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Controls And Circuits For Display Device (AREA)
  • User Interface Of Digital Computer (AREA)
US14/822,600 2014-08-11 2015-08-10 Information processing apparatus, display control method, and storage medium Abandoned US20160042496A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US16/694,755 US11276144B2 (en) 2014-08-11 2019-11-25 Information processing apparatus, display control method, and storage medium

Applications Claiming Priority (6)

Application Number Priority Date Filing Date Title
JP2014-163723 2014-08-11
JP2014163720 2014-08-11
JP2014-163720 2014-08-11
JP2014163723 2014-08-11
JP2015118437A JP6659090B2 (ja) 2014-08-11 2015-06-11 情報処理装置、情報処理装置の制御方法、及びコンピュータプログラム
JP2015-118437 2015-06-11

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US16/694,755 Continuation US11276144B2 (en) 2014-08-11 2019-11-25 Information processing apparatus, display control method, and storage medium

Publications (1)

Publication Number Publication Date
US20160042496A1 true US20160042496A1 (en) 2016-02-11

Family

ID=55267770

Family Applications (2)

Application Number Title Priority Date Filing Date
US14/822,600 Abandoned US20160042496A1 (en) 2014-08-11 2015-08-10 Information processing apparatus, display control method, and storage medium
US16/694,755 Active 2035-09-29 US11276144B2 (en) 2014-08-11 2019-11-25 Information processing apparatus, display control method, and storage medium

Family Applications After (1)

Application Number Title Priority Date Filing Date
US16/694,755 Active 2035-09-29 US11276144B2 (en) 2014-08-11 2019-11-25 Information processing apparatus, display control method, and storage medium

Country Status (4)

Country Link
US (2) US20160042496A1 (ko)
JP (1) JP6659090B2 (ko)
KR (1) KR102113072B1 (ko)
CN (1) CN105373331A (ko)

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150286400A1 (en) * 2014-04-04 2015-10-08 Canon Kabushiki Kaisha Information processing apparatus, control method, and storage medium
US20180069973A1 (en) * 2016-09-02 2018-03-08 Kyocera Document Solutions Inc. Image forming apparatus and method of controlling image forming apparatus
US20200019284A1 (en) * 2018-07-12 2020-01-16 Canon Kabushiki Kaisha Image processing apparatus, control method for image processing apparatus, and storage medium
US20200195804A1 (en) * 2018-12-14 2020-06-18 Sharp Kabushiki Kaisha Image forming apparatus and image forming apparatus system
US11036257B2 (en) 2016-08-19 2021-06-15 Samsung Electronics Co., Ltd. Electronic device and method for controlling display
US11272237B2 (en) 2017-03-07 2022-03-08 Interdigital Madison Patent Holdings, Sas Tailored video streaming for multi-device presentations
US11494986B2 (en) * 2017-04-20 2022-11-08 Samsung Electronics Co., Ltd. System and method for two dimensional application usage in three dimensional virtual reality environment
US11503314B2 (en) 2016-07-08 2022-11-15 Interdigital Madison Patent Holdings, Sas Systems and methods for region-of-interest tone remapping
US11765406B2 (en) 2017-02-17 2023-09-19 Interdigital Madison Patent Holdings, Sas Systems and methods for selective object-of-interest zooming in streaming video
US11765150B2 (en) 2013-07-25 2023-09-19 Convida Wireless, Llc End-to-end M2M service layer sessions
US11770821B2 (en) 2016-06-15 2023-09-26 Interdigital Patent Holdings, Inc. Grant-less uplink transmission for new radio
US11871451B2 (en) 2018-09-27 2024-01-09 Interdigital Patent Holdings, Inc. Sub-band operations in unlicensed spectrums of new radio
US11877308B2 (en) 2016-11-03 2024-01-16 Interdigital Patent Holdings, Inc. Frame structure in NR

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110461208B (zh) * 2017-03-29 2022-10-11 索尼奥林巴斯医疗解决方案公司 控制装置、外部装置、医学观察系统、控制方法、显示方法和程序
JP6991754B2 (ja) * 2017-07-03 2022-01-13 株式会社ミツトヨ 端末装置およびプログラム

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6037939A (en) * 1995-09-27 2000-03-14 Sharp Kabushiki Kaisha Method for enabling interactive manipulation of data retained in computer system, and a computer system for implementing the method
US20070033543A1 (en) * 2005-08-04 2007-02-08 Microsoft Corporation Virtual magnifying glass with intuitive use enhancements
US20100287493A1 (en) * 2009-05-06 2010-11-11 Cadence Design Systems, Inc. Method and system for viewing and editing an image in a magnified view
US20150309565A1 (en) * 2014-04-28 2015-10-29 Adobe Systems Incorporated Method and apparatus for controlling display of digital content using eye movement

Family Cites Families (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH06301757A (ja) * 1993-04-14 1994-10-28 Matsushita Electric Ind Co Ltd 画像処理装置
JPH10312460A (ja) * 1997-05-12 1998-11-24 Japan Aviation Electron Ind Ltd 画像処理方法及び高精度画像処理装置
JP2002215281A (ja) * 2000-12-27 2002-07-31 Internatl Business Mach Corp <Ibm> コンピュータ装置、表示デバイス、出力デバイス、表示制御装置、コンピュータプログラム、記憶媒体、イメージの処理方法
JP2004110719A (ja) * 2002-09-20 2004-04-08 Cresys:Kk 画像表示システム及びコンピュータプログラム
JP2004258476A (ja) 2003-02-27 2004-09-16 Canon Inc 画像表示装置
JP4144555B2 (ja) 2003-06-09 2008-09-03 カシオ計算機株式会社 電子機器、表示制御方法及びプログラム
US7774712B2 (en) * 2004-12-31 2010-08-10 A Far Site Better, Llc Methods and systems for displaying an enlarged image
JP4931686B2 (ja) 2007-05-07 2012-05-16 アルパイン株式会社 ナビゲーション装置
JP5001182B2 (ja) * 2008-01-10 2012-08-15 パナソニック株式会社 表示制御装置、電子機器、表示制御方法、およびプログラム
US8468469B1 (en) * 2008-04-15 2013-06-18 Google Inc. Zooming user interface interactions
JP2011154555A (ja) * 2010-01-27 2011-08-11 Fujitsu Toshiba Mobile Communications Ltd 電子機器
JP2012033058A (ja) * 2010-07-30 2012-02-16 Sony Corp 情報処理装置、情報処理方法及び情報処理プログラム
JP2012084040A (ja) * 2010-10-14 2012-04-26 Nec Casio Mobile Communications Ltd 携帯端末装置、拡大表示方法、及びプログラム
US9058098B2 (en) 2011-02-14 2015-06-16 Sony Corporation Display control device
JP2012226182A (ja) * 2011-04-20 2012-11-15 Sony Corp 画像表示制御装置、画像表示システム、画像表示制御方法、並びにコンピューター・プログラム
JP5726706B2 (ja) 2011-10-14 2015-06-03 クラリオン株式会社 ナビゲーション装置
JP2013089175A (ja) * 2011-10-21 2013-05-13 Furuno Electric Co Ltd 画像表示装置、画像表示プログラム、及び画像表示方法
CN103019537B (zh) * 2012-11-19 2016-04-13 广东欧珀移动通信有限公司 一种图像预览方法及装置

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6037939A (en) * 1995-09-27 2000-03-14 Sharp Kabushiki Kaisha Method for enabling interactive manipulation of data retained in computer system, and a computer system for implementing the method
US20070033543A1 (en) * 2005-08-04 2007-02-08 Microsoft Corporation Virtual magnifying glass with intuitive use enhancements
US20100287493A1 (en) * 2009-05-06 2010-11-11 Cadence Design Systems, Inc. Method and system for viewing and editing an image in a magnified view
US20150309565A1 (en) * 2014-04-28 2015-10-29 Adobe Systems Incorporated Method and apparatus for controlling display of digital content using eye movement

Cited By (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11765150B2 (en) 2013-07-25 2023-09-19 Convida Wireless, Llc End-to-end M2M service layer sessions
US20150286400A1 (en) * 2014-04-04 2015-10-08 Canon Kabushiki Kaisha Information processing apparatus, control method, and storage medium
US11770821B2 (en) 2016-06-15 2023-09-26 Interdigital Patent Holdings, Inc. Grant-less uplink transmission for new radio
US11503314B2 (en) 2016-07-08 2022-11-15 Interdigital Madison Patent Holdings, Sas Systems and methods for region-of-interest tone remapping
US11949891B2 (en) 2016-07-08 2024-04-02 Interdigital Madison Patent Holdings, Sas Systems and methods for region-of-interest tone remapping
US11036257B2 (en) 2016-08-19 2021-06-15 Samsung Electronics Co., Ltd. Electronic device and method for controlling display
US20180069973A1 (en) * 2016-09-02 2018-03-08 Kyocera Document Solutions Inc. Image forming apparatus and method of controlling image forming apparatus
US10057438B2 (en) * 2016-09-02 2018-08-21 Kyocera Document Solutions Inc. Image forming apparatus and method of controlling image forming apparatus
US11877308B2 (en) 2016-11-03 2024-01-16 Interdigital Patent Holdings, Inc. Frame structure in NR
US11765406B2 (en) 2017-02-17 2023-09-19 Interdigital Madison Patent Holdings, Sas Systems and methods for selective object-of-interest zooming in streaming video
US11272237B2 (en) 2017-03-07 2022-03-08 Interdigital Madison Patent Holdings, Sas Tailored video streaming for multi-device presentations
US11494986B2 (en) * 2017-04-20 2022-11-08 Samsung Electronics Co., Ltd. System and method for two dimensional application usage in three dimensional virtual reality environment
US10996821B2 (en) * 2018-07-12 2021-05-04 Canon Kabushiki Kaisha Image processing apparatus, control method for image processing apparatus, and storage medium
US11789587B2 (en) * 2018-07-12 2023-10-17 Canon Kabushiki Kaisha Image processing apparatus, control method for image processing apparatus, and storage medium
US20200019284A1 (en) * 2018-07-12 2020-01-16 Canon Kabushiki Kaisha Image processing apparatus, control method for image processing apparatus, and storage medium
US11871451B2 (en) 2018-09-27 2024-01-09 Interdigital Patent Holdings, Inc. Sub-band operations in unlicensed spectrums of new radio
US10911627B2 (en) * 2018-12-14 2021-02-02 Sharp Kabushiki Kaisha Image forming apparatus and image forming apparatus system
US20200195804A1 (en) * 2018-12-14 2020-06-18 Sharp Kabushiki Kaisha Image forming apparatus and image forming apparatus system

Also Published As

Publication number Publication date
US11276144B2 (en) 2022-03-15
KR102113072B1 (ko) 2020-05-20
CN105373331A (zh) 2016-03-02
US20200090302A1 (en) 2020-03-19
KR20160019369A (ko) 2016-02-19
JP6659090B2 (ja) 2020-03-04
JP2016040709A (ja) 2016-03-24

Similar Documents

Publication Publication Date Title
US11276144B2 (en) Information processing apparatus, display control method, and storage medium
US11057532B2 (en) Image processing apparatus, control method for image processing apparatus, and storage medium
US9292188B2 (en) Information processing apparatus, control method thereof, and storage medium
US9310986B2 (en) Image processing apparatus, method for controlling image processing apparatus, and storage medium
US10979583B2 (en) Information processing apparatus equipped with touch panel type display unit, control method therefor, and storage medium
US10735607B2 (en) Device for generating display data, information device, and display system for displaying scroll region and operation region
JP6412778B2 (ja) 映像装置、方法、およびプログラム
US11630565B2 (en) Image processing apparatus, control method for image processing apparatus, and recording medium for displaying a screen with inverted colors
US20170153751A1 (en) Information processing apparatus, control method of information processing apparatus, and storage medium
JP7130686B2 (ja) 情報処理装置、コンピュータプログラム、情報処理装置の制御方法
JP6176284B2 (ja) 操作表示システム、操作表示装置および操作表示プログラム
US20190310750A1 (en) Electronic device, method for controlling the same, and storage medium
US20220171511A1 (en) Device, method for device, and storage medium
JP6603383B2 (ja) 映像装置、方法、およびプログラム
JP2017041016A (ja) 情報処理装置、制御方法、及びプログラム
JP6210664B2 (ja) 情報処理装置とその制御方法、及びプログラムと記憶媒体
US20150286399A1 (en) Information processing apparatus, control method, and storage medium
JP2019020932A (ja) 情報処理装置、入力制御方法、及びコンピュータプログラム
JP2017123055A (ja) 画像処理装置、プレビュー画像の表示制御方法およびコンピュータプログラム
JP2016062209A (ja) 情報処理装置、表示制御方法、及びコンピュータプログラム
JP2018124631A (ja) 情報処理装置、制御方法、及びプログラム
JP2020013614A (ja) 映像装置、方法、およびプログラム
JP2020013613A (ja) 映像装置、方法、およびプログラム
JP2016053888A (ja) 情報処理装置、表示制御方法、及びコンピュータプログラム

Legal Events

Date Code Title Description
AS Assignment

Owner name: CANON KABUSHIKI KAISHA, JAPAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:ICHIMI, HIDESHI;YAMADA, MASAYUKI;REEL/FRAME:036850/0865

Effective date: 20150729

Owner name: CANON KABUSHIKI KAISHA, JAPAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:ICHIMI, HIDESHI;YAMADA, MASAYUKI;REEL/FRAME:036850/0874

Effective date: 20150729

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: FINAL REJECTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: ADVISORY ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION