JP2019040286A - Information processing apparatus and program - Google Patents

Abstract

To reduce consumption of unnecessary resources in comparison with a case where processing is completed before an instruction is given by a user.SOLUTION: Display control means 11 displays a plurality of display elements on display means 15. Specification means 12 specifies one of the plurality of displayed display elements in accordance with a detection result of detection means which detects a line of sight of a user. Execution means 13 executes preprocessing which is processing of acquiring or generating data for use in processing to be executed by an own apparatus in response to user's selecting the specified display element, at the timing of specifying one of the plurality of display elements by the specification means 12.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an information processing apparatus and a program.

  There has been proposed a technique for executing a hot link for a display element to which a line of sight is moved according to the movement of the line of sight of a user. Japanese Patent Application Laid-Open No. 2004-133867 is an apparatus that generates an action corresponding to a display element of a page on a page according to the movement of the user's line of sight, and obtains the line-of-sight position information of the user on the page. And a predicting unit that predicts the movement destination of the line-of-sight position from the acquired line-of-sight position information, and an activation unit that activates a predetermined action corresponding to the display element of the page where the line-of-sight position is predicted to move is described. ing.

  In addition, techniques for improving performance in memory management have been proposed. For example, in Patent Document 2, as a self-adjusting memory management system that improves actual performance in memory management, information on memory usage (usage frequency, time, number of times of use in a certain period, content creation time, last access time, A system is described that prioritizes pages by relative values based on (I / O transfer costs) and prefetches and / or retains more useful pages in memory.

JP 2009-015610 A JP 2004-133934 A

By the way, when the process specified by the movement of the user's line of sight is completed to the end, unnecessary processes may be executed and resources (costs, etc.) may be generated.
An object of the present invention is to reduce useless resources as compared with a case where processing is completed before an instruction from the user is detected by detecting the user's line of sight.

  An information processing apparatus according to a first aspect of the present invention includes: a display control unit that displays a plurality of display elements on a display unit; and a detection unit that detects any of the plurality of displayed display elements. Used in the process executed by the own device when the specified display element is selected by the user at the timing when any of the plurality of display elements is specified by the specifying means and the specifying means specified from the detection result Preprocessing execution means for executing preprocessing which is processing for acquiring or generating data to be generated.

  An information processing apparatus according to a second aspect of the present invention is the information processing apparatus according to the first aspect, wherein the preprocessing acquires information relating to a status of an output destination apparatus and / or an acquisition source apparatus of data to be processed. Including a process to perform.

  An information processing apparatus according to a third aspect of the present invention is characterized in that, in the configuration according to the first or second aspect, the preprocessing includes a process of acquiring data to be processed from a server.

  An information processing apparatus according to a fourth aspect of the present invention is the configuration according to any one of the first to third aspects, wherein the display control means corresponds to the display corresponding to the execution status of the preprocessing execution means. The display mode of the elements is different.

  In the information processing apparatus according to claim 5 of the present invention, in the configuration according to any one of claims 1 to 4, the number of times that the priority of the specified display element is specified by the specifying unit is high. If the free space of the memory storing the priority specifying means and the data acquired or generated by the preprocessing execution means satisfies a predetermined condition, among the data stored in the memory, Memory control means for deleting data corresponding to a display element having a low priority from the memory is further provided.

  An information processing apparatus according to a sixth aspect of the present invention is the information processing apparatus according to any one of the first to fifth aspects, wherein the preprocessing execution unit is executing a preprocessing corresponding to the specified display element. When another display element is specified by the specifying means, the preprocessing being executed is interrupted, and execution of the preprocessing corresponding to the newly specified display element is started.

  An information processing apparatus according to a seventh aspect of the present invention is the information processing apparatus according to any one of the first to sixth aspects, wherein the type of data to be acquired or generated varies depending on a detection result of the user's line of sight. Features.

  A program according to an eighth aspect of the present invention is a display control step for displaying a plurality of display elements on a display means on a computer, and a detection means for detecting one of the displayed display elements for a user's line of sight. A process that is executed by the device when the specified display element is selected by the user at a timing specified from the detection result, and at a timing when any of the plurality of display elements is specified in the specifying step. It is a program for executing a preprocessing execution step for executing a preprocessing for acquiring or generating data to be used.

According to the first and eighth aspects of the present invention, it is possible to reduce consumption of useless resources as compared with a case where processing is completed before an instruction is given from the user.
According to the second aspect of the present invention, as compared with the case where the information about the status of the output destination device and / or the acquisition source device of the data to be processed is not acquired before the processing is instructed by the user. , User waiting time is reduced.
According to the third aspect of the present invention, the waiting time of the user is shortened compared to the case where the data that is the object of the processing before the processing is instructed by the user is not acquired from the server.
According to the invention which concerns on Claim 4, a user tends to grasp | ascertain the execution condition of the pre-processing about a some display element.
According to the fifth aspect of the present invention, as the number of times specified from the detection result of the user's line of sight increases, the waiting time of the user is shortened as compared with the case where the priority of the display element is not increased.
According to the invention which concerns on Claim 6, compared with the case where the pre-processing corresponding to the newly specified display element is not performed preferentially, a user's waiting time is shortened.
According to the invention which concerns on Claim 7, compared with the case where the pre-processing corresponding to the menu item of a deep hierarchy is not performed, the waiting time of a user is shortened, so that the detection time of a user's eyes | visual_axis is long.

Diagram showing the configuration of the print system The figure which shows the function structure of information processing apparatus The figure which illustrates the hardware constitutions of information processing apparatus The figure which illustrates the appearance of an information processor The flowchart which shows the flow of the process which information processing apparatus performs Figure illustrating the operation screen Figure illustrating the contents of the management table The figure which illustrates the case where an operation screen is operated by the user

[1] Configuration FIG. 1 is a block diagram showing a configuration of a print system 1 according to the present invention. The print system 1 includes an information processing apparatus 10 and a server 20. The information processing apparatus 10 performs image processing such as image forming processing for forming an image on a medium and image reading processing for reading an image formed on the medium. The information processing apparatus 10 is connected to the server 20 via the communication line 2. The communication line 2 includes, for example, at least one of the Internet, a mobile communication network, a telephone line, and a LAN (local area network). The server 20 provides various services such as a storage service.

  FIG. 2 is a block diagram illustrating an example of a functional configuration of the print system 1. In the figure, the information processing apparatus 10 includes a display control unit 11, a specifying unit 12, a preprocessing execution unit 13, a priority specifying unit 14, and a memory control unit 15. The display control unit 11 displays a plurality of display elements on the display unit 16. In this embodiment, the display element is a UI component displayed on the display means 16. When any of the display elements displayed on the display unit 16 is selected (designated) by the user, the information processing apparatus 10 executes a process associated with the selected display element. The display elements are, for example, icons, buttons, and character strings in which images and characters are combined. The specifying unit 12 specifies one of the plurality of display elements displayed on the display unit 16 from the detection result of the detecting unit 17 that detects the user's line of sight.

  The pre-processing execution means 13 acquires data used in the process executed by the own device when any of the plurality of display elements is specified by the specifying means 12 and the specified display element is selected by the user. Alternatively, a preprocess that is a process to be generated is executed. Data acquired or generated by the preprocessing execution means 13 is stored in a predetermined storage area of the memory 18.

  The priority specifying unit 14 specifies the priority of the display element specified by the specifying unit 12 from the detection result of the user's line of sight. In this embodiment, the priority specifying unit 14 increases the priority of the display element as the number of times specified by the specifying unit 12 increases. The memory control unit 15 selects the priority among the data stored in the memory 18 when the free space of the memory 18 in which the data acquired or generated by the preprocessing execution unit 13 is stored satisfies a predetermined condition. The data corresponding to the low display element is deleted from the memory 18.

  FIG. 3 is a diagram illustrating a hardware configuration of the information processing apparatus 10. In the figure, the memory 151 stores various data. The processor 152 performs data processing according to the program stored in the memory 151. The communication IF 153 is an interface that performs data communication with an external device via a network. The UI unit 154 includes, for example, a touch screen and keys. The image forming unit 155 forms an image on a medium such as paper using an electrophotographic method. The image reading unit 156 optically reads an image of a document (medium). The image processing unit 157 includes, for example, a DSP (Digital Signal Processor) and a GPU (Graphics Processing Unit), and executes various types of image processing. The paper feeding unit 158 supplies a medium such as paper to the image forming unit 155.

  The sensor 159 is a sensor that detects the user's line of sight. The sensor 159 includes an infrared light receiving camera and an infrared LED light emitting unit. The infrared light receiving camera detects a user's pupil whose position changes depending on the line of sight, and receives reflected light from the surface of the eyeball by the infrared LED light receiving unit. Thus, the direction of the user's line of sight is detected. As a method for detecting the line of sight, a known method such as a method described in Japanese Patent Application Laid-Open No. 2014-160412 is used.

  FIG. 4 is a diagram illustrating the appearance of the information processing apparatus 10. The sensor 159 uses the display panel 1541 of the UI unit 154 as the line-of-sight detection range, and detects the user's line of sight directed to this range.

  When the processor 152 executes the program stored in the memory 151 of FIG. 3, the functions shown in FIG. 2 are implemented. The processor 152 executing the program is an example of the display control unit 11, the specifying unit 12, the preprocessing execution unit 13, the priority specifying unit 14, and the memory control unit 15. The memory 151 is an example of the memory 18.

[2] Operation Next, the operation of this embodiment will be described.

  FIG. 5 is a flowchart showing the flow of processing performed by the information processing apparatus 10. The process illustrated in FIG. 5 is triggered by a successful login process using the UI unit 154 by the user. In step S101, the processor 152 displays an operation screen of the information processing apparatus 10 on the UI unit 154 (display control step). This operation screen includes a plurality of icons (examples of display elements). When any of the icons displayed on the UI unit 154 is selected by the user on this operation screen, a process associated with the selected icon (hereinafter referred to as “designation process”) is executed.

  FIG. 6 is a diagram illustrating an operation screen G1 displayed on the UI unit 154. Icons I1, I2,..., I6 are displayed on the operation screen G1. The icon I1 is an icon representing a copy process. When the icon I1 is selected by the user, the information processing apparatus 10 executes the copy process. The icon I2 is an icon representing a scan process. When the icon I2 is selected by the user, the information processing apparatus 10 executes the scan process. The icon I3 is an icon representing the box saving process. When the icon I3 is selected by the user, the scanned image data is stored in a predetermined storage area of the server 20.

  Returning to the description of FIG. In step S102, the processor 152 specifies an icon in which the user's line of sight is continuously directed for a predetermined time or more according to the information output from the sensor 159 (specific step). When the icon is specified, that is, when it is determined that the user's line of sight is continuously directed to any of the plurality of icons displayed on the UI unit 154 for a predetermined time or longer (step S102; Y). The processor 152 proceeds to the process of step S103. On the other hand, when the icon is not specified, that is, when it is determined that the user's line of sight is not continuously directed to one icon for a predetermined time or more (step S102; N), the processor 152 displays the icon. The process of step S102 is repeatedly executed until it is specified.

  In step S103, the processor 152 determines whether the preprocessing corresponding to the identified icon has been executed. In this embodiment, the preprocessing indicates processing for acquiring or generating data (hereinafter referred to as “preparation data”) used in the designation processing represented by the icon selected by the user. The preprocessing includes, for example, processing for acquiring data to be processed (hereinafter referred to as “processing target data”) from an acquisition source device (for example, the server 20). As another example, the preprocessing includes, for example, information (hereinafter referred to as “status information”) regarding the status of the output destination device (for example, the server 20) and / or the acquisition source device (for example, the server 20) of the processing target data. Including the process of acquiring.

  The state information acquired by the processor 152 is, for example, information indicating the output destination of the processing target data and / or the power ON / OFF of the acquisition source device (for example, the server 20). As another example, the status information is, for example, information indicating power ON / OFF and usage status of the UI unit 154, the image forming unit 155, and the image reading unit 156. The status information is information indicating, for example, a finisher that performs post-processing such as processing to be bound by a stapler, power ON / OFF of a coin kit, and usage status. The status information is information indicating, for example, an open / close state of a paper storage unit that stores paper, the number of sheets, and a paper size.

  As another example, the status information includes, for example, information indicating the remaining amount of toner and the manufacturer in the image forming unit 155, and information indicating the opening / closing status of the platen cover of the image reading unit 156 and the presence / absence of a placed document. It is. The status information is information indicating whether or not a USB cable or an Ethernet (registered trademark) cable is connected.

  The preparation data generated by the processor 152 may be drawing data necessary for drawing on the UI screen. The drawing data is data generated by the processor 152 from data representing a screen displayed on the UI unit 154 stored in the memory 151. Further, the preprocessing may be a login process to the server 20. In this case, the information processing apparatus 10 transmits a login request including a user ID and the like to the server 20 and acquires a response indicating the login result transmitted from the server 20 as state information.

  When it is determined in step S103 of FIG. 5 that the preprocessing has been completed, that is, the preparation data corresponding to the specified icon has been stored in the memory 151 (step S103; Y), the processor 152 The process proceeds to S108. On the other hand, when it is determined that the preparation data is not yet stored in the memory 151 (step S103; N), the processor 152 proceeds to the process of step S104.

  In step S104, the processor 152 determines whether or not there is a free space in the predetermined storage area of the memory 151 (whether the free capacity of the memory 151 satisfies a predetermined condition). In this embodiment, the processor 152 determines that there is no free space when the free space in the predetermined storage area of the memory 151 is equal to or less than a predetermined threshold. When it is determined that there is a vacancy (step S104; Y), the processor 152 proceeds to the process of step S105. On the other hand, when it is determined that there is no free memory (step S104; N), the processor 152 proceeds to the process of step S106.

  In step S105, the processor 152 executes pre-processing of the designation processing represented by the icon specified in step S102 (pre-processing execution step). For example, in the screen illustrated in FIG. 6, when the user's line of sight remains on the icon I2 for a predetermined time or longer, the processor 152 obtains and / or generates data used in the scan process (process data is used in the scan process). Process). Specifically, the processor 152 acquires state information such as an output destination device of processing target data. The acquired state information is, for example, information indicating whether the image reading unit 156 is turned on or off or in use (platen cover open / close state, whether or not a document is left behind, lamp state, glass surface contamination, etc.). .

  At this time, for example, when the paper storage unit of the information processing apparatus 10 is not closed or the platen cover is open, information indicating that (NG) (hereinafter referred to as “NG information”) is acquired. . When the acquired information is NG information, the processor 152 waits until information indicating OK is acquired.

  Further, for example, in the screen illustrated in FIG. 6, when the user's line of sight stays on the icon I1 for a predetermined time or longer, the processor 152 acquires pre-processing of copying processing (acquiring and / or acquiring data used in copying processing) Or a process to be generated). Specifically, the processor 152, for example, information indicating whether the image forming unit 155 is turned on / off or used, information indicating the state of the paper storage unit (out of paper, etc.), toner status (toner remaining amount) Etc.) and information indicating the state of the image reading unit 156 (open / closed state of the platen cover, whether or not the document is left behind, lamp state, dirt on the glass surface, etc.).

  The preparation data acquired or generated by the preprocessing in step S105 in FIG. 5 is stored in a predetermined storage area of the memory 151. Further, the processor 152 associates the preparation data stored in the memory 151 with the priority of the preparation data and registers them in the management table. The management table is stored in a predetermined storage area of the memory 151.

  FIG. 7 is a diagram illustrating the contents of the management table. In the management table, items of “processing type”, “priority order”, and “used memory” are associated with each other. Among these items, the “processing type” item stores identification information for identifying preparation data stored in the memory 151. This identification information is information for identifying preparation data, that is, an icon (or a designation process corresponding to the icon) displayed on the UI unit 154. In the “priority order” item, information indicating the priority order is stored. Information indicating the amount of memory used is stored in the “used memory” item. The priority of the preparation data newly registered in the management table may be the lowest order, and a priority parameter may be provided for each preparation data.

  Returning to the description of FIG. In step S106, the processor 152 performs processing for recalculating the priority order. Specifically, for example, the processor 152 performs control to lower the priority of the preparation data that has not been used for a predetermined period.

  In step S107, the processor 152 performs processing for deleting the preparation data having the lowest priority from the memory 151 and also deleting it from the management table. That is, in this embodiment, when the free space of the memory 151 is equal to or less than a predetermined threshold, the preparation data corresponding to the icon having the lowest priority among the data stored in the memory is deleted from the memory 151. Is done. When the process of step S107 is completed, the processor 152 proceeds to the process of step S105.

  In step S108, the processor 152 performs processing for increasing the priority order of the identified icon. In other words, in this embodiment, as shown in steps S103 and S108, an icon having a higher specified number of times has a higher priority.

  6 is executed in the state in which the operation screen illustrated in FIG. 6 is displayed on the UI unit 154, the preparation corresponding to the icon in which the user's line of sight stays for a predetermined time or more is executed. Data is stored in the memory 151. The stored preparation data is lowered in priority when it is not used for a predetermined period of time, or is deleted in order from the lowest priority when there is no free space. In this embodiment, when the user is switched (when the login process is performed by a different user), the preparation data is temporarily deleted from the memory 151.

  In the operation screen illustrated in FIG. 6, the processor 152 changes the display mode of the icons I 1, I 2,..., I 6 depending on the execution status of the preprocessing (preparing / preparing complete, etc.). In this embodiment, the display mode is an appearance of an icon (display element), and indicates, for example, the color and size of the display element, a combination of included images, and the like. Specifically, for example, the processor 152 changes the color and size of the icon displayed on the UI unit 154 depending on whether preparation is in progress or preparation is completed. For example, when the preparation is completed, the processor 152 may display an image indicating that in the vicinity of the icon (within a predetermined range) or superimposed on the icon.

  In the operation screen illustrated in FIG. 6, the processor 152 changes the display mode of the icon depending on whether or not the state information acquired in the preprocessing corresponding to the displayed icon is information indicating NG. . For example, for an icon whose acquired state information is information indicating NG, the processor 152 changes an icon color or size, or displays an image indicating that some error occurs in the vicinity of the icon (predetermined Within the range) or superimposed on the icon.

  When the user selects (designates) one of the displayed icons in a state where the operation screen illustrated in FIG. 6 is displayed on the UI unit 154, the processor 152 causes the user to select The process (designated process) represented by the selected icon is executed. At this time, when the preparation data corresponding to the selected icon is stored in the memory 151, the processor 152 executes a designation process using the stored preparation data.

  FIG. 8 is a diagram illustrating a case where the operation screen is operated by the user. In the figure, when the user performs an operation of selecting the icon I5 representing the facsimile function, the processor 152 starts execution of facsimile processing. In this facsimile process, the processor 152 uses the preparation data associated with the icon I5 and stored in the memory 151. As described above, in this embodiment, since various types of data (preparation data) relating to the facsimile process are pre-read (or pre-read), the activation time of the facsimile process is shortened.

  In this embodiment, an icon to be selected by the user is specified by detecting a line of sight, and data (preparation data) used in processing represented by the specified icon is prefetched in the background. In addition, when the process represented by the identified icon includes a process for requesting information acquisition, information acquisition is also executed at the timing when the line of sight is detected. At this time, the processor 152 does not execute all processing (designation processing) corresponding to the specified icon, but performs preprocessing necessary for execution of the designation processing. Thereby, consumption of useless resources (cost, toner, paper, etc.) is suppressed.

  In this embodiment, when the acquired state information is information indicating NG, the processor 152 waits until information indicating OK is acquired. At this time, if an icon is selected by the user during standby, an NG message can be displayed immediately, and the UI response is improved.

[3] Modifications The above-described embodiments are merely examples of the present invention, and may be modified as follows. Moreover, you may implement combining embodiment mentioned above and each modification shown below as needed.

(1) In the above-described embodiment, the processor 152 identifies an icon in which the user's line of sight has remained for a predetermined time or more among the plurality of icons displayed on the UI unit 154. The method for specifying the icon is not limited to that shown in the above-described embodiment. For example, the processor 152 may identify an icon to which the user's line of sight is directed regardless of whether the time during which the line of sight remained is long or short. The processor 152 may specify any of the displayed icons from the detection result of the user's line of sight.

(2) In the above-described embodiment, the case where the icons I1, I2,..., I6 including character strings are displayed on the operation screen G1 illustrated in FIG. The display elements displayed on the UI unit 154 are not limited to those shown in the above-described embodiment. For example, an icon in which an image and characters are combined may be used as a display element.

(3) In the above-described embodiment, the control is performed such that the higher the priority is given to the icon that is identified more frequently. The priority setting method is not limited to that shown in the above embodiment. For example, the processor 152 may increase the priority as the length of the stop time of the user's line of sight increases. Further, for example, the processor 152 may increase the priority as the timing when the user's line of sight is detected is earlier. The processor 152 may specify the priority of the specified icon from the detection result of the user's line of sight.

(4) In the above-described embodiment, when another icon is specified during execution of the preprocessing corresponding to the specified icon, the processor 152 interrupts the preprocessing being executed and newly specifies the icon. Execution of the processing corresponding to may be started. For example, when the scan process icon is identified during the pre-process of the copy process, the processor 152 interrupts the pre-process of the copy process and starts the pre-process of the scan process.

(5) In the above-described embodiment, the processor 152 may change the type of preparation data to be acquired or generated depending on the detection result of the user's line of sight. The type of preparation data indicates items of preparation data acquired by the processor 152. For example, the type of setting item (“color / monochrome”, “double-sided / single-sided”), “toner remaining amount”, and the like. Indicates an item. For example, the processor 152 may perform control to increase the type of data to be acquired as the icon has a longer user gaze detection time. For example, when the user's line of sight with respect to the icon representing the copy process is detected, if the time for which the line of sight is detected is short, the processor 152 provides information on power ON / OFF of the image forming unit 155 and the image reading unit 156. If the time is long, the control may be performed such that state information regarding a finer state (for example, the remaining amount of color toner) is acquired.

(6) In the above-described embodiment, the program executed by the processor 152 of the information processing apparatus 10 may be downloaded via a communication line such as the Internet. These programs are provided in a state where they are recorded on a computer-readable recording medium such as a magnetic recording medium (magnetic tape, magnetic disk, etc.), an optical recording medium (optical disk, etc.), a magneto-optical recording medium, or a semiconductor memory. May be.

DESCRIPTION OF SYMBOLS 1 ... Print system, 2 ... Communication line, 10 ... Information processing apparatus, 11 ... Display control means, 12 ... Specification means, 13 ... Pre-processing execution means, 14 ... Priority specification means, 15 ... Memory control means, 16 ... Display Means, 17 ... detecting means, 18 ... memory, 20 ... server, 151 ... memory, 152 ... processor, 153 ... communication IF, 154 ... UI part, 155 ... image forming part, 156 ... image reading part, 157 ... image processing part 158... Paper feeding unit, 159.

Claims (8)

Display control means for displaying a plurality of display elements on the display means;
A specifying means for specifying any of the plurality of displayed display elements from a detection result of a detection means for detecting a user's line of sight;
A process of acquiring or generating data used in a process executed by the own apparatus when the specified display element is selected by a user at a timing when any of the plurality of display elements is specified by the specifying unit. An information processing apparatus comprising: preprocessing execution means for executing a certain preprocessing. The information processing apparatus according to claim 1, wherein the preprocessing includes a process of acquiring information related to a state of an output destination apparatus and / or an acquisition source apparatus of data to be processed. The information processing apparatus according to claim 1, wherein the preprocessing includes processing for acquiring data to be processed from a server. The information processing apparatus according to any one of claims 1 to 3, wherein the display control unit changes a display mode of the corresponding display element according to an execution status of the preprocessing execution unit. Priority specifying means for increasing the priority of the specified display element as the number of times specified by the specifying means increases;
When the free space of the memory storing the data acquired or generated by the preprocessing execution unit satisfies a predetermined condition, it corresponds to a display element with a low priority among the data stored in the memory. The information processing apparatus according to claim 1, further comprising: a memory control unit that deletes data from the memory. The preprocessing execution means interrupts the preprocessing being executed and newly specified when another display element is specified by the specifying means during execution of the preprocessing corresponding to the specified display element. The information processing apparatus according to any one of claims 1 to 5, wherein execution of preprocessing corresponding to a display element is started. The information processing apparatus according to claim 1, wherein the preprocessing execution unit changes a type of data to be acquired or generated depending on a detection result of the user's line of sight. On the computer,
A display control step for displaying a plurality of display elements on the display means;
A specific step of identifying any of the plurality of displayed display elements from a detection result of a detection unit that detects a user's line of sight;
Pre-processing for acquiring or generating data used in processing executed by the own device when the specified display element is selected by the user at the timing when any of the plurality of display elements is specified in the specifying step A program for executing a pre-processing execution step for executing.

Images