JP6763260B2 - Information processing equipment, image forming equipment and programs - Google Patents

Description

The present invention relates to an information processing device, an image forming device, and a program.

The display unit of the information processing device may display a list of functions related to the own device as an image. A method in which the user selects a desired function from this list is generally used. At this time, it has been proposed that the position of the line of sight indicating which part the user is looking at is detected and a predetermined process is performed according to the position of the line of sight.

According to Patent Document 1, the image reproduction device includes a line-of-sight detection unit that detects a person's line of sight with respect to an image displayed on a display, and a line-of-sight detection unit that detects a person's line of sight with respect to the image displayed on the display. It is described that the viewing mode determining unit for determining the viewing mode and the image quality adjusting unit for adjusting the image quality of the display image of the display according to the viewing mode determined by the viewing mode determining unit are provided.
Further, in Patent Document 2, the exercise assist device includes a seat portion on which the user sits, a drive portion that swings the seat portion, and an elevating portion that adjusts the height of the seat portion by raising and lowering the seat portion. It is equipped with a footrest on which the user rests his / her feet, a handle for the user to put his / her hand on and support the upper body, and a display panel attached to the handle. The display panel displays information such as exercise time. It is described that the display unit is provided and the display unit is configured to be adjusted to a position that matches the line of sight of the user.

Japanese Unexamined Patent Publication No. 2011-137874 JP-A-2007-167292

However, when detecting the position of the line of sight of the user, an error may occur in the detection of the position due to the characteristics of the user. In particular, as a characteristic of the user, the direction of the line of sight is likely to change depending on the height of the user, and there is a problem that it is difficult to accurately detect the position of the line of sight. In order to solve this, it is necessary to perform calibration for each user, but in this case, there arises a problem that convenience is reduced.
An object of the present invention is to provide an information processing device or the like capable of accurately detecting the position of a line of sight without performing calibration for each user.

The invention according to claim 1 is a feature information acquisition means for acquiring feature information which is information on a user's feature that affects the detection of the position of the line of sight, and an object to be viewed by the user according to the acquired feature information. The feature information is an information processing device that is information on the height of the user , including a line-of-sight detecting means for detecting the position of the line of sight on the object.
The invention according to claim 2 is a feature information acquisition means for acquiring feature information which is information on the characteristics of the user that affects the detection of the position of the line of sight, and an object to be viewed by the user according to the acquired feature information. The line-of-sight detecting means includes a line-of-sight detecting means for detecting the position of the line of sight in the object, and the line-of- sight detecting means has adjustment information for adjusting the position of the line of sight corresponding to the feature information acquired by the feature information acquiring means. When performing, the position of the line of sight is detected using the adjustment information, and when the adjustment information does not exist, the adjustment information is acquired from the relationship between the feature information and the adjustment information prepared in advance, and the acquired adjustment is performed. a information processing apparatus characterized by detecting the position of the line of sight by using the information.
The invention according to claim 3 further includes a user authentication means for authenticating a user, and a storage means for storing the user authentication information used by the user authentication means in association with the adjustment information. The information processing apparatus according to claim 2, which is a feature.
The invention according to claim 4 is the information processing device according to claim 3 , wherein the storage means further stores the user authentication information in association with the feature information.
The invention according to claim 5 is a feature information acquisition means for acquiring feature information, which is information on the characteristics of the user that affects the detection of the position of the line of sight, and an object to be viewed by the user according to the acquired feature information. made and a visual axis detecting means for detecting the position of the visual line in the object, the visual axis detecting means, is lower than the height information of the height of the user is predetermined, the information to reduce the sensitivity to movement of the line of sight report It is a processing device.
The invention according to claim 6 comprises an image forming unit that forms an image on a recording material, a display unit that displays information about the own device as an image, and the own device according to the position of the user's line of sight on the display unit. The control unit includes a control unit for controlling, and the control unit includes a feature information acquisition means for acquiring feature information which is information on the characteristics of a user that affects the detection of the position of the line of sight, and the control unit according to the acquired feature information. The feature information is an image forming apparatus that includes information on the height of the user , including a line-of-sight detecting means for detecting the position of the line of sight on the display unit.
The invention according to claim 7 is a feature information acquisition function for acquiring feature information, which is information on a user's feature that affects the detection of the position of the line of sight, and a user according to the acquired feature information. A line-of-sight detection function that detects the position of the line-of-sight on an object to be viewed is realized, and the feature information is a program that is information on the height of the user .

According to the first aspect of the present invention, it is possible to provide an information processing device capable of accurately detecting the position of the line of sight without performing calibration for each user. In addition, adjustment information can be created according to the characteristics of the user that particularly affect the detection of the position of the line of sight.
According to the invention of claim 2, the adjustment information for the user who has already performed the calibration is used, and the adjustment information is provided from the feature information for the user who has not yet performed the calibration. Can be obtained.
According to the invention of claim 3 , adjustment information can be acquired based on the user's authentication information.
According to the invention of claim 4 , it is not necessary to acquire the feature information again for the user who has acquired the feature information once.
According to the invention of claim 5 , it becomes easier to deal with a short user who is likely to be influenced by the position detection.
According to the invention of claim 6 , it is possible to provide an image forming apparatus that makes it easy for a user to select a desired function from a list of functions. In addition, adjustment information can be created according to the characteristics of the user that particularly affect the detection of the position of the line of sight.
According to the invention of claim 7, a function capable of accurately detecting the position of the line of sight without performing calibration for each user can be realized by a computer. In addition, adjustment information can be created according to the characteristics of the user that particularly affect the detection of the position of the line of sight.

It is an external view of the image forming apparatus which concerns on this embodiment. It is a figure which shows the internal structure of the image forming apparatus which concerns on this embodiment. It is a figure which showed an example of the image displayed in the user interface. (A) to (d) are diagrams showing a method of detecting the position of the user's line of sight from the image of the user's eyes taken by the camera. It is a figure which showed the influence which the height of a user has on the detection of the position of a line of sight. It is a block diagram which showed the functional configuration example of a control device. It is a flowchart explaining the operation of a control device.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings.

<Explanation of the overall configuration of the image forming apparatus 1>
FIG. 1 is an external view of the image forming apparatus 1 according to the present embodiment. FIG. 2 is a diagram showing an internal structure of the image forming apparatus 1 according to the present embodiment.
The image forming apparatus 1 includes an image reading apparatus 100 that reads an image of a document, and an image recording apparatus 200 that records an image on a recording material (hereinafter, may be collectively referred to as “paper”). There is. Further, the image forming apparatus 1 includes a user interface (UI) 300 that accepts operation input from the user and displays various information to the user. The image forming apparatus 1 includes a camera 400 for detecting the position of the user's line of sight. Further, the image forming apparatus 1 includes a control device 500 that controls the operation of the entire image forming apparatus 1.

The image reading device 100 is arranged above the image forming device 1, the image recording device 200 is arranged below the image reading device 100, and the control device 500 is built in. The user interface 300 is arranged on the front side of the upper part of the image forming apparatus 1, that is, on the front side of the image reading unit 110 described later of the image reading device 100. The camera 400 is arranged at the lower left end of the user interface 300.

First, the image reading device 100 will be described.
The image reading device 100 includes an image reading unit 110 that reads an image of a document, and a document conveying unit 120 that conveys the document to the image reading unit 110. The document transport unit 120 is arranged above the image reading device 100, and the image reading unit 110 is arranged below the image reading device 100.
The document transporting unit 120 has a document accommodating unit 121 for accommodating documents and a document discharging unit 122 for ejecting documents transported from the document accommodating unit 121, and the document is transmitted from the document accommodating unit 121 to the document discharging unit 122. To carry. The document transport unit 120 is also called an automatic document feeder (ADF: Auto Document Feeder).

The image reading unit 110 includes a platen glass 111, a light irradiation unit 112 that irradiates the surface to be read (image surface) of the document with light, and the light irradiation unit 112 that irradiates the surface to be read of the document with light L. It includes a light guide unit 113 that guides the light L reflected by the surface to be read, and an imaging lens 114 that forms an optical image of the light L guided by the light guide unit 113. Further, the image reading unit 110 is composed of a photoelectric conversion element such as a CCD (Charge Coupled Device) image sensor that photoelectrically converts the light L imaged by the imaging lens 114, and detects an imaged optical image. The unit 115 includes an image processing unit 116 that is electrically connected to the detection unit 115 and transmits an electric signal as image information obtained by the detection unit 115.
The image reading unit 110 reads an image of the original document conveyed by the document conveying unit 120 and an image of the original document placed on the platen glass 111.

Next, the image recording device 200 will be described.
The image recording device 200 includes an image forming unit 20 that forms an image on the paper P, a paper supply unit 60 that supplies the paper P to the image forming unit 20, and a paper P on which an image is formed by the image forming unit 20. A paper ejection unit 70 for ejecting an image, and an inverted conveying unit 80 for inverting the front and back of the paper P on which an image is formed on one side of the image forming unit 20 and conveying the paper P toward the image forming unit 20 again. ing.

The image forming unit 20 has four image forming units 21Y, 21M, 21C, and 21K, which are arranged in parallel at regular intervals, yellow (Y), magenta (M), cyan (C), and black (K). It has. Each image forming unit 21Y, 21M, 21C, 21K is a static electricity formed by a photoconductor drum 22, a charger 23 that uniformly charges the surface of the photoconductor drum 22, and laser irradiation by an optical system unit 50 described later. It is provided with a developer 24 that develops and visualizes an electro-latent image with a predetermined color component toner. Further, the image forming unit 20 is provided with toner cartridges 29Y, 29M, 29C and 29K for supplying toner of each color to the developing units 24 of the image forming units 21Y, 21M, 21C and 21K.

The image forming unit 20 includes an optical system unit 50 that irradiates the photoconductor drum 22 of the image forming units 21Y, 21M, 21C, 21K with a laser beam below the image forming units 21Y, 21M, 21C, 21K. There is. The optical system unit 50 includes a semiconductor laser and a modulator (not shown), a polygon mirror (not shown) that deflects and scans the laser beam emitted from the semiconductor laser, and a glass window (not shown) that passes the laser beam. , A frame (not shown) for sealing each component is provided.

Further, the image forming unit 20 includes an intermediate transfer unit 30 for multiplex transfer of toner images of each color formed on the photoconductor drums 22 of the image forming units 21Y, 21M, 21C, and 21K onto the intermediate transfer belt 31, and an intermediate transfer unit. It is provided with a secondary transfer unit 40 that transfers a toner image formed on the paper P by superimposing the toner image on the paper P, and a fixing device 45 that heats and pressurizes the toner image formed on the paper P to fix the toner image. ..

The intermediate transfer unit 30 includes an intermediate transfer belt 31, a drive roller 32 for driving the intermediate transfer belt 31, and a tension roller 33 for applying a constant tension to the intermediate transfer belt 31. Further, the intermediate transfer unit 30 is a plurality of intermediate transfer units 30 for transferring the toner image formed on the photoconductor drum 22 facing each other with the intermediate transfer belt 31 interposed therebetween (this implementation). In this form, the primary transfer roller 34) and the backup roller 35 facing the secondary transfer roller 41 described later via the intermediate transfer belt 31 are provided.

The intermediate transfer belt 31 is attached to a plurality of rotating members such as a drive roller 32, a tension roller 33, a plurality of primary transfer rollers 34, a backup roller 35, and a driven roller 36. Then, the intermediate transfer belt 31 is circulated and driven at a predetermined speed in the arrow direction by a drive roller 32 that is rotationally driven by a drive motor (not shown). As the intermediate transfer belt 31, for example, one molded from rubber or resin is used.
Further, the intermediate transfer unit 30 includes a cleaning device 37 for removing residual toner and the like existing on the intermediate transfer belt 31. The cleaning device 37 removes residual toner, paper dust, and the like from the surface of the intermediate transfer belt 31 after the toner image transfer step is completed.

The secondary transfer unit 40 includes a secondary transfer roller 41 that is provided at the secondary transfer position and presses the backup roller 35 via the intermediate transfer belt 31 to secondary transfer the image onto the paper P. The secondary transfer position where the toner image transferred to the intermediate transfer belt 31 is transferred to the paper P by the secondary transfer roller 41 and the backup roller 35 facing the secondary transfer roller 41 via the intermediate transfer belt 31 is determined. It is composed.
The fixing device 45 fixes the image (toner image) on the paper P secondarily transferred by the intermediate transfer unit 30 to the paper P by the heat fixing roller 46 and the pressure roller 47 using heat and pressure.

The paper supply unit 60 includes a paper storage unit 61 that stores the paper on which an image is formed, a delivery roll 62 that feeds out the paper P stored in each of the paper storage units 61, and the paper P that is fed by the delivery roll 62. It is provided with a transport path 63 for transporting the paper P, and transport rolls 64, 65, 66 for transporting the paper P arranged along the transport path 63 and fed by the delivery roll 62 to the secondary transfer position. Further, the paper supply unit 60 further includes a manual feed tray 67 capable of manually supplying the paper P. The manual feed tray 67 is foldable, can be folded when not in use, and can be opened to load paper P when in use. In FIGS. 1 and 2, the bypass tray 67 is shown in a folded state.

The paper ejection unit 70 is provided above the image forming unit 20, and has a first loading tray 71 for loading the paper P on which the image is formed by the image forming unit 20, and the first loading tray 71 and an image. A second loading tray 72 for loading the paper P on which the image is formed by the image forming unit 20 is provided between the reading device 100 and the reading device 100.
The paper ejection unit 70 is provided on the downstream side in the transport direction from the fixing device 45, and is provided on the transport roll 75 for transporting the paper P on which the toner image is fixed and on the downstream side in the transport direction of the transport roll 75. It is provided with a switching gate 76 for switching the conveying direction of the paper P. Further, the paper ejection unit 70 transfers the paper P transported to the downstream side in the transport direction of the switching gate 76 to one side (right side in FIG. 2) of the transport direction switched by the switching gate 76 to the first loading tray 71. A first discharge roll 77 for discharging is provided. Further, the paper ejection unit 70 includes a transport roll 78 for transporting the paper P to be transported to the downstream side of the switching gate 76 in the transport direction to the other side (upper side in FIG. 2) of the transport direction switched by the switching gate 76. It includes a second discharge roll 79 for discharging the paper P conveyed by the transfer roll 78 to the second loading tray 72.

The reversing transport unit 80 transports the inverted paper P to the side of the fixing device 45 by rotating the transport roll 78 in the direction opposite to the direction in which the paper P is discharged to the second loading tray 72. The reverse transport path 81 is provided. A plurality of transport rolls 82 are provided in the reverse transport path 81 along the reverse transport path 81. The paper P conveyed by these transfer rolls 82 is again fed to the secondary transfer position by the transfer roll 82.

Further, the image recording device 200 includes a device body frame 11 that directly or indirectly supports the image forming unit 20, the paper supply unit 60, the paper ejection unit 70, the reverse transfer unit 80, and the control device 500, and the device body. It includes a device housing 12 that is attached to the frame 11 to form the outer surface of the image forming device 1.
The apparatus main body frame 11 is provided with a switching gate 76, a first discharge roll 77, a transfer roll 78, a second discharge roll 79, and the like on the one end side in the lateral direction of the image forming apparatus 1, and is vertically and vertically provided. It includes a reader support unit 13 that extends in the direction and supports the image reader 100. The reading device support unit 13 supports the image reading device 100 in cooperation with a portion on the back side of the device main body frame 11.

Further, the image recording device 200 is provided as a part of the device housing 12 on the front side of the image forming unit 20, and also includes a front cover 15 that is openably and closably attached to the device main body frame 11.
By opening the front cover 15, the user can replace the intermediate transfer unit 30 and the toner cartridges 29Y, 29M, 29C, and 29K of the image forming unit 20 with new ones.

The user interface 300 is, for example, a touch panel. By using the user interface 300 as a touch panel, various information such as image forming conditions of the image forming apparatus 1 is displayed on the touch panel. Then, the user touches the touch panel to perform an input operation such as an image forming condition. That is, the user interface 300 functions as a display unit that displays information about the image forming apparatus 1 that is its own device as an image, and also as an operation unit that accepts a selection operation in which the user selects a target function from the displayed images. Function. The user interface 300 is not limited to the touch panel, and the function as a display unit and the function as an operation unit may be separated. That is, the user interface 300 may be configured to separately include a display unit including a liquid crystal display and the like and an operation unit including physical keys and the like.

The camera 400 captures the eyes of the user operating the user interface 300.
The camera 400 includes, for example, an optical system that converges the image of the user's eyes and an image sensor that detects the image converged by the optical system. The optical system is composed of a single lens or a combination of a plurality of lenses. The image sensor is configured by arranging image pickup elements such as CCD (Charge Coupled Device) and CMOS (Complementary Metal Oxide Semiconductor). The image sensor photoelectrically converts the detected image in pixel units and outputs it as an image signal.

<Explanation of operation of image forming apparatus 1>
The image forming apparatus 1 configured as described above operates as follows.
For example, the user can copy the original using the image forming apparatus 1. That is, the original can be copied by forming an image on the paper P by the image recording device 200 based on the image data of the original read by the image reading device 100. Further, the user can print by transmitting a print job to the image forming apparatus 1 from a personal computer (PC) or the like (not shown) connected to the communication line. That is, a print job can be received via the communication line, and the image can be printed by forming the image on the paper P by the image recording device 200 based on the image data included in the print job. In addition, the user can send and receive facsimiles. That is, the image data of the original document read by the image reading device 100 can be transmitted via the communication line. Alternatively, the user can save the image data of the original. That is, the image data of the original can be saved inside the image forming apparatus 1 or in a PC connected to the communication line.

Here, the operation of the image forming apparatus 1 will be described in detail by taking the case where the user copies or prints the original as an example.
The image of the original read by the image reader 100 and the image data received from the PC or the like are subjected to predetermined image processing, and the image data subjected to the image processing is yellow (Y) or magenta (M). ), Cyan (C), and black (K) are converted into four-color material gradation data and output to the optical system unit 50.

The optical system unit 50 emits the laser light emitted from the semiconductor laser (not shown) to the polygon mirror via the f−θ lens (not shown) according to the input color material gradation data. In the polygon mirror, the incident laser beam is modulated according to the gradation data of each color, deflected and scanned, and the image forming units 21Y, 21M, 21C, and 21K are subjected to an imaging lens (not shown) and a plurality of mirrors. The photoconductor drum 22 is irradiated.

In the photoconductor drum 22 of the image forming units 21Y, 21M, 21C, and 21K, the surface charged by the charger 23 is scanned and exposed to form an electrostatic latent image. The formed electrostatic latent image is developed as a toner image of each color of yellow (Y), magenta (M), cyan (C), and black (K) in each image forming unit 21Y, 21M, 21C, 21K. Will be done. The toner images formed on the photoconductor drums 22 of the image forming units 21Y, 21M, 21C, and 21K are multiplex-transferred onto the intermediate transfer belt 31 which is an intermediate transfer body.

On the other hand, in the paper supply unit 60, the paper feeding roll 62 rotates in accordance with the timing of image formation to pick up the paper P stored in the paper storage unit 61 and transport the paper P through the transport path 63 by the transport rolls 64 and 65. Will be done. After that, the transfer roll 66 rotates according to the movement timing of the intermediate transfer belt 31 on which the toner image is formed, and the paper P is transferred to the secondary transfer position formed by the backup roller 35 and the secondary transfer roller 41. .. Toner images in which four colors are multiplexed are sequentially transferred in the sub-scanning direction on the paper P conveyed from the bottom to the top at the secondary transfer position using a pressure contact force and a predetermined electric field. Will be done. Then, the paper P on which the toner images of each color are transferred is subjected to a fixing process by heat and pressure by the fixing device 45, and then discharged and loaded on the first loading tray 71 or the second loading tray 72.

When there is a request for double-sided printing, the paper P on which an image is formed on one side is conveyed by the reverse transfer unit 80 so that the front and back sides are reversed, and is sent to the secondary transfer position again. Then, at the secondary transfer position, the toner image is transferred to the other surface of the paper P, and the image transferred by the fixing device 45 is fixed. After that, the paper P on which the images are formed on both sides is ejected and loaded on the first loading tray 71 or the second loading tray 72.

The user interface 300 operates as follows.
FIG. 3 is a diagram showing an example of an image displayed on the user interface 300.
In this case, the user interface 300 displays a list of icons for executing the function included in the image forming apparatus 1 as an image.

In the illustrated example, the user interface 300 is a copy, fax / internet fax, scanner (mail transmission), job flow, print anywhere, easy fax, scanner (PC save), external access, scanner (box save), box operation, Icons representing the functions of job memory and digital camera print are displayed, and these are listed.
Further, in the illustrated example, the user interface 300 displays a "back" button as a button for screen transition.

Then, the user performs a selection operation to select one of the icons or buttons in order to use the desired function.
Specifically, the selection operation is an operation in which the user touches an icon or a button when the user interface 300 is a touch panel. When the user touches the icon, the setting screen corresponding to the function associated with the icon is displayed. For example, if you touch the "Copy" icon, the screen for setting the number of copies to be made, the paper to be used, the black / white / color selection, and the enlargement / reduction settings will be displayed as the setting screen for copying the original. To. Then, after the setting, when the user presses a start button or the like (not shown), the actual copy operation is started. You can also touch the "Back" button to return to the original screen such as the home screen. The user interface 300 is not limited to this, and for example, when the user interface 300 is composed of a liquid crystal display and a physical key, the user selects an icon by pressing the arrow key, which is a physical key, and similarly. The operation of pressing the enter key, which is a physical key, can be a selection operation.

Further, the image forming apparatus 1 detects the position of the user's line of sight on the user interface 300 by analyzing the image taken by the camera 400 with the control device 500. Thereby, for example, it is determined which icon of the user interface 300 the user is looking at based on the detected position of the line of sight of the user. Then, for example, it is assumed that the icon selection operation is performed when staring at any icon for a predetermined time or when the user blinks a predetermined number of times while staring at any icon. .. That is, it can be used as a line-of-sight input.

4 (a) to 4 (d) are views showing a method of detecting the position of the user's line of sight from the image of the user's eyes taken by the camera 400.
Of these, FIGS. 4A to 4B show a method of detecting the position of the line of sight with visible light.
In the illustrated example, first, the positions of the inner corner G and the iris K in the user's eye M are obtained. In this case, when the position of the line of sight of the user changes, the positional relationship between the inner corner G and the iris K changes as shown in FIGS. 4 (a) and 4 (b). As a result, the position of the line of sight can be obtained from the positional relationship between the inner corner G of the eye and the iris K.
Further, FIGS. 4 (c) to 4 (d) show a method of detecting the position of the line of sight by infrared rays.
In the illustrated example, first, the positions of the pupil D and the corneal reflex H in the user's eye M are obtained. In this case, when the position of the user's line of sight changes, the positional relationship between the pupil D and the corneal reflex H changes as shown in FIGS. 4 (c) and 4 (d). As a result, the position of the line of sight can be obtained from the positional relationship between the pupil D and the corneal reflex H.

Further, in order to obtain the position of the line of sight more accurately, calibration is required. Specifically, as calibration, it is necessary to adjust the relationship between the positional relationship shown in FIG. 4 and the actual line-of-sight position. This is because the position of the line of sight detected differs depending on the characteristics of the user. The user's characteristics are the user's physical characteristics that affect the detection of the position of the line of sight. Specifically, for example, the height of the user. In addition, for example, the angle of the user's face, the size of the face, the width of the eyes, the size of the pupil, the distance between the eyes, and whether or not the user has a squint. Of these, the height of the user has a particularly large effect on the detection of the position of the line of sight.

FIG. 5 is a diagram showing the influence of the height of the user on the detection of the position of the line of sight.
FIG. 5 shows a case where the user is looking at the upper end U of the user interface 300. The user interface 300 has a depth of 30 cm. Further, the user's eyes M1 and M2 are located at a position 20 cm away from the user interface 300 in the horizontal direction. Further, in this example, the camera 400 is arranged near the lower end L of the user interface 300.

The figure shows the difference in the direction of the line of sight between a short user with a height of 140 cm and a tall user with a height of 190 cm. Of these, the position of the eyes of the short user is illustrated by M1. It is assumed that the eye M1 is 50 cm above the user interface 300 at this time. And the short user is looking in the direction of 45 ° with respect to the vertical direction. This is an angle of 23 ° when viewed from the camera 400.

On the other hand, the position of the eyes of a tall user is illustrated by M2. It is assumed that the eye M2 is 100 cm above the user interface 300 at this time. Tall users are looking in a direction of 26 ° with respect to the vertical direction. This is an angle of 15 ° when viewed from the camera 400. Therefore, even if the viewing location is the same, the state when viewed from the camera 400 differs depending on the height of the user.

Further, when the position of the user's line of sight moves from the upper end U to the lower end L of the user interface 300, the short user moves by 23 °, but the tall user moves by 15 °. That is, even if the amount of eye movement is the same, the shorter the user, the smaller the position of the line of sight, and the taller the user, the larger the position of the line of sight. It can be said that the sensitivity to the movement of the eyes is larger for the taller user and smaller for the shorter user.

As described above, the movement amount of the line-of-sight position with respect to the eye position and the movement amount of the line-of-sight position with respect to the eye movement amount greatly differ depending on the height of the user. Therefore, it is necessary to adjust this difference by performing calibration.
In the example of FIG. 5, the vertical direction of the user interface 300 has been described, but the same can be said for the horizontal direction.

To actually perform calibration, for example, marks are sequentially displayed at a plurality of positions on the user interface 300. Then, the user follows it with his / her eyes to grasp the relationship between the above-mentioned positional relationship and the position of the line of sight. When calibration is performed, the calibration value, which is a parameter for adjusting the relationship between the positional relationship and the line-of-sight position, is saved. Then, for that user, the position of the line of sight can be detected accurately by using this calibration value thereafter. This calibration value is an example of adjustment information for adjusting the position of the line of sight.

However, the image forming apparatus 1 is often used by an unspecified number of users. And the characteristics such as height of these users are different from each other. However, if calibration is performed for each user, the convenience is reduced. On the other hand, when a fixed calibration value is used, it is difficult to accurately detect the position of the line of sight due to the influence of the height of the user or the like as described above.

Therefore, in the present embodiment, a calibration value suitable for the user's characteristics such as height is prepared in advance, and a calibration value close to the user's characteristics is used for a user who has not been calibrated. We are trying to suppress. Hereinafter, the configuration of the control device 500 that realizes this will be described.

<Explanation of control device 500>
FIG. 6 is a block diagram showing a functional configuration example of the control device 500. Note that, in FIG. 6, among various functions of the control device 500, those related to the present embodiment are selected and shown.
The control device 500 is an example of an information processing device, and is also an example of a control unit that controls its own device (image forming device 1) according to the position of the user's line of sight on the user interface 300.
As shown in the figure, the control device 500 of the present embodiment includes the user authentication means 510, the calibration value acquisition means 520, the storage means 530, the feature information acquisition means 540, the image acquisition means 550, and the line-of-sight detection means 560. And the selection operation determination means 570.

The user authentication means 510 authenticates the user who uses the image forming apparatus 1. User authentication may be performed by, for example, a method in which the user inputs a user ID and password from the user interface 300. An NFC (Near Field Communication) or the like is separately provided in the image forming apparatus 1, and the user holds the ID card over the NFC. The method may be used. In the figure, the user authentication means 510 shows a case where a user is authenticated by a user ID and a password.

The calibration value acquisition means 520 acquires a calibration value for each user. The calibration value acquisition means 520 acquires the calibration value associated with the user ID, for example, based on the user ID (user authentication information). That is, the storage means 530 stores the user ID and the calibration value in correspondence with each other. Then, the calibration value acquisition means 520 refers to the storage means 530 and acquires the calibration value corresponding to the user ID. In this case, this user has already performed the calibration, and therefore the calibration value is prepared in advance.

If the user has not been calibrated and the calibration value is not prepared, the feature information acquisition means 540 acquires the feature information corresponding to this user.
The feature information is information on the features of the user that influences the detection of the position of the line of sight. Specifically, the feature information is, for example, the height information of the user described above. In addition, for example, information such as the angle of the user's face, the size of the face, the width of the eyes, the size of the pupil, the distance between the eyes, and whether or not the user is squinting may be used.
The feature information can be acquired, for example, by the user inputting from the user interface 300. If the feature information can be estimated from an image such as the height of the user, it can be obtained from an image taken by the camera 400, for example.

The image acquisition means 550 acquires an image signal of the user's image taken by the camera 400.

The line-of-sight detecting means 560 obtains the positional relationship as described with reference to FIG. 4 based on the user's image taken by the camera 400. Then, the line-of-sight detecting means 560 detects the position of the line of sight in the object to be viewed by the user from this positional relationship. In this case, the object is the user interface 300, and the line-of-sight detecting means 560 detects the position of the user's line of sight on the user interface 300.

At this time, the line-of-sight detecting means 560 detects the position of the user's line of sight on the user interface 300 according to the acquired feature information.
Specifically, the storage means 530 stores the calibration value according to the feature information. That is, the storage means 530 stores the feature information and the calibration value in correspondence with each other. Then, the calibration value acquisition means 520 acquires the calibration value with reference to the storage means 530. At this time, the calibration value acquisition means 520 can find the feature information closest to the feature information acquired by the feature information acquisition means 540 from the storage means 530 and acquire the calibration value accordingly. Further, a plurality of feature information close to the feature information acquired by the feature information acquisition means 540 may be found from the storage means 530, and the calibration values corresponding thereto may be weighted averaged. Further, when acquiring a calibration value based on a plurality of feature quantities, a weighting coefficient is set for each feature quantity. Then, for each feature amount, the feature information closest to the feature information acquired by the feature information acquisition means 540 is found from the storage means 530. Further, the difference between the feature information acquired by the feature information acquisition means 540 and the feature information stored in the storage means 530 is taken and multiplied by a weighting coefficient. As a result, the calibration value for the one with the smallest value after multiplication is selected.

Then, the line-of-sight detecting means 560 uses the acquired calibration value to determine the position of the user's line of sight on the user interface 300.

The selection operation determination means 570 determines that the user has performed a selection operation on the user interface 300 based on the position of the line of sight detected by the line-of-sight detection means 560. As described above, the selection operation determination means 570 blinks a predetermined number of times, for example, when staring at any icon or the like for a predetermined time, or when the user stares at any of the icons or the like. Occasionally, it is determined that the selection operation of the icon or the like has been performed. When the selection operation determination means 570 determines that the user has performed the selection operation, it outputs selection information indicating that the icon or the like has been selected.

As described above, in the control device 500 of the present embodiment, the line-of-sight detection means 560 detects the position of the line of sight by using the calibration value when there is a calibration value for adjusting the position of the line of sight. .. When the line-of-sight detection means 560 does not have a calibration value corresponding to the input feature information, the line-of-sight detection means 560 acquires a calibration value from the relationship between the feature information prepared in advance and the calibration value, and obtains the acquired calibration value. Use to detect the position of the line of sight.

<Explanation of operation of control device 500>
Next, the operation of the control device 500 will be described.
FIG. 7 is a flowchart illustrating the operation of the control device 500.
First, the user authentication means 510 authenticates the user who uses the image forming apparatus 1 (step 101).

Next, the calibration value acquisition means 520 refers to the storage means 530 and determines whether or not there is a calibration value for the authenticated user (step 102).

As a result, when the calibration value exists (Yes in step 102), the calibration value acquisition means 520 acquires the calibration value from the storage means 530 (step 103). After that, the process proceeds to step 106. That is, the line-of-sight detecting means 560 detects the position of the user's line of sight using the calibration value already prepared.

On the other hand, when the calibration value does not exist (No in step 102), the feature information acquisition means 540 acquires the feature information corresponding to the authenticated user (step 104).
Next, the calibration value acquisition means 520 refers to the storage means 530 and acquires the calibration value according to the acquired feature information (step 105).

Further, the image acquisition means 550 acquires an image of the user taken by the camera 400 (step 106).

Then, the line-of-sight detecting means 560 obtains the positional relationship as described with reference to FIG. 4 based on the user's image taken by the camera 400. Further, the line-of-sight detecting means 560 detects the position of the user's line of sight on the user interface 300 by using the calibration value acquired by the calibration value acquiring means 520 (step 107).

Further, the selection operation determination means 570 determines whether or not the user has performed the selection operation on the user interface 300 based on the position of the line of sight detected by the line-of-sight detection means 560 (step 108).

Then, when the selection operation determination means 570 determines that the user has performed the selection operation (Yes in step 108), the selection information is output (step 109).
On the other hand, when the selection operation determination means 570 does not determine that the user has performed the selection operation (No in step 108), the process returns to step 106.

Hereinafter, the present invention will be described in more detail with reference to Examples, but the present invention is not limited to these Examples as long as the gist thereof is not exceeded.

(Example 1)
Consider a case where four users, user A (height 152 cm), user B (height 164 cm), user C (height 178 cm), and user D (height 191 cm), use the image forming apparatus 1. At this time, as shown in Table 1 below, the user A and the user C have already performed the calibration, and the calibration value is stored in the storage means 530. On the other hand, the user B and the user D have not yet performed the calibration, and the calibration value is not stored in the storage means 530.

At this time, when the user A or the user C authenticates the user by the user authentication means 510 in order to use the image forming apparatus 1, the calibration value acquisition means 520 is stored in the storage means 530 for calibration. Get the value. Then, the line-of-sight detecting means 560 uses this calibration value to detect the position of the user's line of sight.

On the other hand, when the user B or the user D authenticates the user by the user authentication means 510 in order to use the image forming apparatus 1, the feature information acquisition means 540 acquires the feature information about the user B or the user D. To do. Then, the calibration value acquisition means 520 refers to the storage means 530 and acquires the calibration value according to the acquired feature information.

Table 2 shows the relationship between the feature information stored in the storage means 530 and the calibration value at this time. In this case, the feature information is the height information of the user. The height is set every 10 cm between 140 cm and 190 cm, and a calibration value is associated with each.

Since the height of the user B is 164 cm, the calibration value acquisition means 520 acquires, for example, the calibration value of 160 cm, which is the closest to 164 cm in Table 2. Further, since the user D has a height of 191 cm, the calibration value acquisition means 520 acquires, for example, the calibration value of 190 cm, which is the closest to 191 cm in Table 2.

After that, the same processing is performed for any of the users A to D. That is, the image acquisition means 550 acquires the image of the user taken by the camera 400. Then, the line-of-sight detecting means 560 detects the position of the user's line of sight on the user interface 300 by using the acquired calibration value based on the image of the user taken by the camera 400. Further, the selection operation determination means 570 determines whether or not the user has performed the selection operation on the user interface 300 based on the detected position of the line of sight.

At this time, it is preferable that the storage means 530 stores the user ID and the feature information in correspondence with each other for the user B and the user D. Table 3 shows an example of the correspondence between the user ID stored in the user ID and the feature information. This eliminates the need to acquire feature information thereafter.

According to the above-described embodiment, it is possible to accurately detect the position of the line of sight for each user who uses the image forming apparatus 1 without performing calibration for each user.

In the above-mentioned example, the image forming apparatus 1 has been taken as an example of the information processing apparatus, but the description is not limited to this. That is, the present embodiment can be applied to any device that detects the line of sight. For example, it can be applied to devices such as mobile phones, smartphones, and tablets that display application software icons side by side.
Further, in the above-described example, the user is looking at the user interface 300 and has detected the position of the line of sight on the user interface 300, but of course, it can be applied to the case of looking at other parts.
In addition, the effect of the user's height on the position of the line of sight becomes more problematic as the user's height is lower. Therefore, when the user's height information is greater than or equal to a predetermined height, a fixed calibration value is used. When the height information of the user is lower than the predetermined height, it is conceivable to use a calibration value that reduces the sensitivity to the movement of the line of sight.

<Program description>
The processing performed by the control device 500 is realized by the cooperation of software and hardware resources.

Therefore, the processing performed by the control device 500 is a feature information acquisition function for acquiring the feature information which is the feature information of the user which affects the detection of the position of the line of sight, and the user sees the feature information according to the acquired feature information. It can also be regarded as a program that realizes a line-of-sight detection function that detects the position of the line of sight in the target object.

1 ... Image forming device, 20 ... Image forming unit, 30 ... Intermediate transfer unit, 40 ... Secondary transfer unit, 45 ... Fixing device, 100 ... Image reading device, 200 ... Image recording device, 300 ... User interface, 400 ... Camera , 500 ... Control device, 510 ... User authentication means, 520 ... Calibration value acquisition means, 530 ... Storage means, 540 ... Feature information acquisition means, 550 ... Image acquisition means, 560 ... Line-of-sight detection means, 570 ... Selection operation determination means

Claims (7)

Feature information acquisition means for acquiring feature information, which is information on user features that affect the detection of the position of the line of sight,
A line-of-sight detection means for detecting the position of the line of sight in an object to be viewed by the user according to the acquired feature information.
Equipped with a,
The feature information is an information processing device that is information on the height of the user . Feature information acquisition means for acquiring feature information, which is information on user features that affect the detection of the position of the line of sight,
A line-of-sight detection means for detecting the position of the line of sight in an object to be viewed by the user according to the acquired feature information.
With
The line-of-sight detecting means detects the position of the line of sight by using the adjustment information when there is adjustment information for adjusting the position of the line of sight corresponding to the feature information acquired by the feature information acquiring means. when the adjustment information is not present acquires the adjustment information from the relationship between the feature information and the adjustment information prepared in advance, information for and detecting the position of the line of sight by using the obtained corresponding adjustment information Processing equipment. User authentication means for authenticating users and
A storage means for storing the user authentication information used by the user authentication means in association with the adjustment information, and
The information processing apparatus according to claim 2 , further comprising. The information processing device according to claim 3 , wherein the storage means further stores the user authentication information in association with the feature information. Feature information acquisition means for acquiring feature information, which is information on user features that affect the detection of the position of the line of sight,
A line-of-sight detection means for detecting the position of the line of sight in an object to be viewed by the user according to the acquired feature information.
With
The visual axis detecting means, is lower than the height information of the height of the user is predetermined, the information processing apparatus to reduce the sensitivity to movement of the line of sight. An image forming part that forms an image on the recording material,
A display unit that displays information about the own device as an image,
A control unit that controls its own device according to the position of the user's line of sight on the display unit,
With
The control unit
Feature information acquisition means for acquiring feature information, which is information on user features that affect the detection of the position of the line of sight,
A line-of-sight detecting means for detecting the position of the line of sight on the display unit according to the acquired feature information.
Equipped with a,
The feature information is an image forming apparatus that is information on the height of the user . On the computer
A feature information acquisition function that acquires feature information, which is information on user features that affect the detection of the position of the line of sight,
A line-of-sight detection function that detects the position of the line of sight on an object to be viewed by the user according to the acquired feature information.
Realized ,
The feature information is a program that is information on the height of the user .

Images