JP2017168039A - Image display device, image display method, and image display program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable a neighborhood person to browse a content, while enabling an operation by an operator.SOLUTION: An image display device includes a display part which displays the content, an operation receiving part 17 which receives the operation, a person part extraction part 31 which detects the operator for inputting the operation and the neighborhood person other than the operator, an operator position detection part 33 which detects the first relative position of the operator with respect to the display surface of the display part 120, a neighborhood person position detection part 35 which detects the second relative position of the neighborhood person with respect to the display surface of display means, a display position decision part 13 which decides a position where an operation screen for allowing the operator to input the operation is arranged on the display surface of the display part on the basis of the first relative position and the second relative position, and a display control part 15 which displays the operation screen in the decided position and changes the display mode of the content.SELECTED DRAWING: Figure 3

Description

  The present invention relates to an image display device, an image display method, and an image display program, and more particularly to an image display device that can accept an operation by an operator, an image display control method and an image display control program executed by the image display device.

  In recent years, digital signage for displaying content such as advertisements has been installed in public places. Some digital signage can form content images on paper. When the operator operates the digital signage, the operator is positioned near the display surface on which the content is displayed. Therefore, if the operator is positioned behind the operator operating the digital signage, a part of the display surface is There is a problem that a part of the content cannot be visually recognized by being hidden by the operator.

  For example, Japanese Patent Application Laid-Open No. 2014-016904 includes display means capable of displaying an image, operation detection means for detecting an operator's operation on the display means, and control means for controlling the display means, A content display unit that displays a content image on a display unit, an area specifying unit that specifies an area in which the content image is displayed or an area in which a predetermined image is displayed in the content image, in the image display area in the display unit; Based on the detection result by the operation detection means, an operation position specifying unit that specifies an operation position related to an operation by an operator in the display area, and a menu for content at the operation position specified by the operation position specifying unit. Menu display section for displaying an image, area for displaying the menu image, and area specification The menu image display position is changed so that the menu image is displayed in an area excluding the area specified by the area specifying unit in the display area when the area specified by A content display device including a control unit having a display position changing unit is described.

However, since the content display device described in Japanese Patent Application Laid-Open No. 2014-016904 displays a menu image in a region different from the region where the content is displayed, the operator must move to a position where the menu screen is displayed. There is a problem that must be done.
JP 2014-016904 A

  The present invention has been made to solve the above-described problems, and one of the objects of the present invention is to provide a display control device that allows a neighbor to view content while enabling operation by an operator. It is.

  Another object of the present invention is to provide a display control method that allows a neighbor to browse content while enabling an operation by an operator.

  Still another object of the present invention is to provide a display control program that allows a neighbor to browse content while enabling operation by an operator.

  The present invention has been made to solve the above-described problems. According to one aspect of the present invention, an image display device inputs a display unit that displays content, an operation reception unit that receives an operation, and an operation. Detecting means for detecting an operator who performs the operation, a neighbor other than the operator, an operator position detecting means for detecting a first relative position of the operator to the display surface of the display means detected by the detecting means, and detecting means Based on the detected neighbor relative position detecting means for detecting the second relative position of the neighbor relative to the display surface of the display means, and the detected first relative position and the detected second relative position, Display position determining means for determining the position where the operation screen for inputting the operation is arranged on the display surface of the display means, and displaying the operation screen at the determined position and changing the display mode of the content It includes a 示制 your means.

  According to this aspect, the content is displayed, the first relative position with respect to the display surface of the operator is detected, the second relative position with respect to the display surface of the neighbor is detected, and the first relative position and the second relative position are detected. Based on this, the position where the operation screen is arranged on the display surface is determined, the operation screen is displayed at the determined position, and the display mode of the content is changed. For this reason, since the position where the operation screen is arranged is determined by the relative position between the operator and the neighbor, the operator can operate the operation screen, and the operation screen is arranged at a position where the neighbor cannot see the operation screen. be able to. Moreover, since the display mode of the content is changed, the content can be displayed in a display mode that can be visually recognized by a neighbor. As a result, it is possible to provide a display control device that allows neighbors to browse content while enabling operations by the operator.

  Preferably, the display screen further includes blind spot area determining means for determining a blind spot area that is a blind spot of a neighbor on the display surface, and the display control means displays the content in a displayable area other than the determined blind spot area.

  According to this aspect, a blind spot area that is a blind spot of the neighbor is determined on the display surface, and the content is displayed in a displayable area other than the blind spot area, so that the neighbor can visually recognize the content.

  Preferably, the display control means determines the size of the operation screen based on the determined size of the blind spot area.

  According to this aspect, the operation screen can be displayed in a region that can be visually recognized by the operator but cannot be visually recognized by a neighbor.

  Preferably, the display control means changes the size of the content based on the size of the displayable area.

  According to this aspect, since the size of the content is changed based on the size of the displayable area, the entire content can be displayed.

  Preferably, the display control means determines a rectangular area extracting means for extracting a rectangular area from an area other than the blind spot area, and determines the extracted rectangular area as a displayable area.

  According to this aspect, rectangular content can be displayed in a rectangular area.

  Preferably, when a plurality of rectangular regions are extracted by the rectangular region extracting unit, the display control unit determines a rectangular region having the largest area among the plurality of rectangular regions as a displayable region.

  According to this aspect, since the content is displayed in the rectangular area having the largest area, the content can be displayed in as large a size as possible.

  Preferably, when a plurality of rectangular regions are extracted by the rectangular region extracting unit, the display control unit determines each of two or more rectangular regions that do not overlap each other as a displayable region among the plurality of rectangular regions.

  According to this aspect, since the content is displayed in two or more rectangular areas that do not overlap each other, the content can be displayed so as not to overlap.

  Preferably, when a plurality of rectangular areas are extracted by the rectangular area extracting means, the display control means determines a rectangular area closest to the neighbor's line of sight among the plurality of rectangular areas as a displayable area.

  According to this aspect, the content is displayed in the rectangular area closest to the neighbor's line of sight among the plurality of rectangular areas, so that the neighbor can easily see the content.

  Preferably, the display position determining means determines the position on the display surface of the operation screen so that the operator is positioned between the operation screen and the neighbor.

  According to this aspect, since the operator is located between the operation screen and the neighbor, the operator can visually recognize the operation screen, but the neighbor cannot hide the operator screen from the operator. it can.

  Preferably, when a plurality of operators are detected by the detecting unit, the display position determining unit determines a position on the display surface of each of the plurality of operation screens respectively corresponding to the plurality of operators.

  According to this aspect, when a plurality of operators are detected, the positions on the display surfaces of the plurality of operation screens respectively corresponding to the plurality of operators are determined, so that operations by a plurality of users can be accepted. it can.

  Preferably, the display control display means includes a movement display control means for moving the operation screen on the display surface in accordance with a change in position determined by the display position determination means after the operation screen is displayed on the display means. .

  According to this aspect, since the operation screen moves on the display surface according to the position change determined after the operation screen is displayed, the operation screen is moved to a position according to the movement of the operator or the neighbor. Can be displayed.

  Preferably, the display control means further includes movement prohibiting means for prohibiting movement of the operation screen by the movement display control means while the operation is accepted by the operation accepting means.

  According to this aspect, since the movement of the operation screen is prohibited while the operation by the operator is accepted, the operator can easily input the operation.

  Preferably, the camera further includes a line-of-sight detecting unit that detects a line of sight of a neighbor, and the display position determining unit determines the position of the operation screen on the condition that the detected line of sight is the display surface of the display unit.

  According to this aspect, since the position of the operation screen is determined on the condition that the line of sight of the neighbor is on the display surface, the display mode of the content can be changed when the neighbor is watching the content. it can.

  Preferably, when the position of the operation screen is not determined by the display position determination unit, the operation screen display unit prohibits display of the operation screen and further includes an undecided time processing unit that accepts an operation by the operator.

  According to this aspect, when the position of the operation screen is not determined, the display of the operation screen is prohibited and the operation by the operator can be accepted. According to this aspect, since the operation screen is not displayed, the neighbor can visually recognize the content and can receive the operation by the operator without displaying the operation screen.

  Preferably, an image forming unit that forms an image on the recording medium, and an image reading unit that reads the image formed on the recording medium are further provided, and the undecided time processing unit displays the image of the content displayed on the display unit. A setting value image forming unit that causes the image forming unit to form an image for describing a setting value for forming, and an image reading unit that analyzes the image formed by the image forming unit to recognize characters Character recognition means, and setting means for setting a setting value for causing the image forming means to form an image based on the recognized character.

  According to this aspect, the operation by the operator can be accepted without displaying the operation screen.

  Preferably, the undetermined time processing means includes an operating device specifying means for specifying an information communication device carried by the operator as an operating device, and a remote control means remotely controlled by the determined operating device.

  According to this aspect, since the remote control is performed by the information communication device carried by the operator, the operation by the operator can be accepted without displaying the operation screen.

  Preferably, when the position of the operation screen is not determined by the display position determination unit, the operation screen display unit further includes a content transmission unit that displays the operation screen at a predetermined position and transmits the content to the outside.

  According to this aspect, when the position of the operation screen is not determined, the operation screen is displayed at a predetermined position, and the content is transmitted to the outside. For this reason, if the neighbor receives the content and carries a device for display, the neighbor can visually recognize the content.

  According to another aspect of the present invention, a display control method is a display control method that is executed by an image display device that includes a display unit that displays content. The display control method receives an operation, and receives an operation. In a detection step for detecting an operator and a neighbor other than the operator, an operator position detection step for detecting a first relative position of the operator with respect to the display surface detected in the detection step, and a detection step The operator operates based on the detected neighbor position detecting step for detecting the second relative position of the neighbor with respect to the display surface of the display means, and the detected first relative position and the detected second relative position. A display position determining step for determining a position at which the operation screen for inputting the operation position is arranged on the display surface of the display means, the operation screen is displayed at the determined position, and Comprising a display control step of changing a display mode of Ntsu, the.

  According to this aspect, it is possible to provide a display control method that allows a neighbor to view content while enabling an operation by an operator.

  According to another aspect of the present invention, the display control program is a display control program that is executed by a computer that controls an image display device including a display unit that displays content, and an operation reception step that receives an operation; A detection step for detecting an operator who inputs an operation and a neighbor other than the operator, and an operator position detection step for detecting a first relative position of the display means of the display means of the operator detected in the detection step. , Based on the neighbor position detection step of detecting the second relative position of the display means of the neighbor detected in the detection step, the detected first relative position and the detected second relative position, A display position determining step for determining a position where an operation screen for an operator to input an operation is arranged on the display surface of the display means, and the determined position In displays an operation screen, to execute a display control step of changing the display mode of the content and the computer.

  According to this aspect, it is possible to provide a display control program that allows a neighbor to browse content while enabling an operation by an operator.

1 is a perspective view showing an external appearance of an MFP in one embodiment of the present invention. 2 is a diagram illustrating an example of a hardware configuration of an MFP. FIG. 3 is a diagram illustrating an example of functions of a CPU provided in the MFP according to the first embodiment. FIG. It is a figure which shows an example of the detailed function of a content position determination part. It is a figure which shows an example of the detailed function of an undecided time process part. It is a 1st figure for demonstrating a blind spot area | region. It is a 1st figure which shows an example of the blind spot area | region in a display surface. It is a 2nd figure for demonstrating a blind spot area | region. It is a 2nd figure which shows an example of the blind spot area | region in a display surface. It is a figure for demonstrating a some rectangular area. It is a flowchart which shows an example of the flow of the display control process in 1st Embodiment. It is a flowchart which shows an example of the flow of the operation screen display process in 1st Embodiment. It is a flowchart which shows an example of the flow of an undecided execution process. It is a figure which shows an example of the detailed function of the process part at the time of undecided in a modification. It is a flowchart which shows an example of the flow of an undecided execution process in a modification. It is a figure which shows an example of the function which CPU with which MFP in 2nd Embodiment has is provided. It is a flowchart which shows an example of the flow of the display control process in 2nd Embodiment. It is a flowchart which shows an example of the flow of the operation screen display process in 2nd Embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description, the same parts are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed description thereof will not be repeated.

  FIG. 1 is a perspective view showing an appearance of an MFP according to one embodiment of the present invention. Referring to FIG. 1, an MFP (Multi Function Peripheral) 100 is an example of an image display device, and includes a display unit 120 capable of displaying an image, a touch panel 130 provided so as to be superimposed on a display surface of the display unit 120, A camera 140 provided above the display surface of the display unit 120, a main body 101, and a base 121 that supports the display unit 120 are included.

  The base 121 includes a bottom plate 123 and struts 122 extending vertically from the bottom plate. The display unit 120 has a back surface fixed to an upper portion of the support column 122, and the display surface of the display unit 120 is substantially parallel to the vertical direction. By changing the position where the display unit 120 is fixed to the column 122, the height of the display surface of the display unit 120 from the ground can be changed. The height of the display surface of the display unit 120 from the ground is preferably equal to or higher than the eye level of an upright person. The main body 101 is disposed on the bottom plate 123 of the base 121. The main body 101 includes a document reading unit 150 that reads a document, an image forming unit 160 that forms an image on a sheet or the like based on image data, and a paper feeding unit 170 that supplies the sheet to the image forming unit 160.

  FIG. 2 is a diagram illustrating an example of a hardware configuration of the MFP. Referring to FIG. 2, MFP 100 includes a main circuit 110 to which display unit 120, touch panel 130, camera 140, document reading unit 150, image forming unit 160, and paper feeding unit 170 are connected.

  The main circuit 110 includes a CPU (Central Processing Unit) 111, a communication I / F (Interface) unit 112, a ROM (Read Only Memory) 113, a RAM (Random Access Memory) 114, and an EEPROM (Electrically Erasable Memory). ) 115, an HDD (Hard Disk Drive) 116 as a mass storage device, a facsimile unit 117, an external storage device 118 on which a CD-ROM (Compact Disk-ROM) 118A is mounted, a short-range communication unit 119, ,including. CPU 111 is connected to display unit 120, touch panel 130, camera 140, document reading unit 150, image forming unit 160, and paper feeding unit 170, and controls the entire MFP 100.

  The display unit 120 is a display device such as an LCD (Liquid Crystal Display), an organic ELD (Electro-Luminescence Display), a plasma display panel, or electronic paper. The display unit 120 is controlled by the CPU 111 and displays an image.

  The touch panel 130 is provided on the upper surface or the lower surface of the display unit 120. The touch panel 130 detects a position on the display surface of the display unit 120 instructed by the operator. The touch panel 130 outputs the detected position coordinates to the CPU 111.

  The CPU 111 identifies the position in the image displayed on the display unit 120 from the position of the display surface of the display unit 120 detected by the touch panel 130, and detects an operation by the operator. For example, when displaying a keyboard on the display unit 120, the CPU 111 detects an operation instructing input of a key if the position detected by the touch panel 130 is in an image of any key of the keyboard.

  The camera 140 includes a lens and a photoelectric conversion element. The light focused by the lens is imaged on the photoelectric conversion element, and the photoelectric conversion element photoelectrically converts the received light to output image data. The photoelectric conversion element is a CMOS (Complementary Metal Oxide Semiconductor) sensor, a CCD (Charge Coupled Device) sensor, or the like. The camera 140 is disposed above the display unit 120 at the center position in the horizontal direction of the display unit 120. The camera 140 is set so that the optical axis thereof is parallel to the normal line of the display surface of the display unit 120 and the angle of view includes a person who views the display surface of the display unit 120. Further, the camera 140 may be configured to change the optical axis and the angle of view of the camera 140 as long as the person who can visually recognize the display surface of the display unit 120 can be imaged. Further, although the camera 140 is provided above the display unit 120, the camera 140 may be provided below the display unit 120 and on the side of the display unit.

  The camera 140 includes a distance measuring unit 141 that can measure the distance to the subject. The distance measuring unit 141 irradiates infrared rays and measures the distance from the subject based on the time taken to receive the reflected light from the subject. If the image captured by the camera 140 includes a plurality of subjects, the distance measuring unit 141 measures the distance for each of the plurality of subjects and measures the distance between each of the plurality of subjects. The camera 140 outputs the distance between the image data obtained by imaging the subject and each of the one or more subjects to the CPU 111. Note that the two cameras 140 are arranged with the optical axes in parallel, and the distance measuring unit 141 calculates the distance to the subject in the image based on the two images captured by the two cameras 140, respectively. You may do it.

  The ROM 113 stores a program executed by the CPU 111 or data necessary for executing the program. The RAM 114 is used as a work area when the CPU 111 executes a program.

  Communication I / F unit 112 is an interface for connecting MFP 100 to a network. The CPU 111 communicates with an information communication device connected to the network via the communication I / F unit 112 to transmit / receive data. Further, the communication I / F unit 112 can communicate with an information communication device connected to the Internet via a network.

  The facsimile unit 117 is connected to a PSTN (Public Switched Telephone Network) and transmits / receives facsimile data. The facsimile unit 117 converts the image data read by the document reading unit 150 or the data stored in the HDD 116 into facsimile data, and transmits the facsimile data to the facsimile apparatus connected to the PSTN. Further, the facsimile unit 117 stores the received facsimile data in the HDD 116 or causes the image forming unit 160 to form an image of the facsimile data on a sheet.

  The short-range communication unit 119 performs wireless communication based on Bluetooth (registered trademark) standard GAP (Generic Access Profile) or the like. For example, the short-range communication unit 119 communicates when the distance between the communication partner device is equal to or less than the communicable distance. The distance that the short-range communication unit 119 can communicate is several meters. Further, the short-range communication unit 119 may perform communication using a near field communication (NFC) method. In this case, the distance that the short-range communication unit 119 can communicate is several tens of centimeters.

  The external storage device 118 is loaded with a CD-ROM 118A. The CPU 111 can access the CD-ROM 118 </ b> A via the external storage device 118. The CPU 111 can load a program recorded on the CD-ROM 118A attached to the external storage device 118 into the RAM 114 and execute it. Note that the program executed by the CPU 111 is not limited to the program recorded on the CD-ROM 118A, and the program stored in the HDD 116 may be loaded into the RAM 114 and executed. In this case, another information communication device connected to the network may rewrite a program stored in HDD 116 of MFP 100 or may write a new program. Further, MFP 100 may download a program from another information communication device connected to the network and store the program in HDD 116. The program here includes not only a program directly executable by the CPU 111 but also a source program, a compressed program, an encrypted program, and the like.

  FIG. 3 is a diagram illustrating an example of functions of the CPU provided in the MFP according to the first embodiment. The functions shown in FIG. 3 are functions formed in CPU 111 when CPU 111 provided in MFP 100 according to the first embodiment executes a display control program stored in ROM 113, EEPROM 115, HDD 116, or CD-ROM 118A. .

  With reference to FIG. 3, the CPU 111 includes an image analysis unit 11, a display position determination unit 13, a display control unit 15, an operation reception unit 17, a process execution unit 19, and an undecided time processing unit 21. Including. The image analysis unit 11 receives the distance between the image obtained by capturing the subject from the camera 140 and the subject, and detects the position of the subject by analyzing the image. The image analysis unit 11 includes a human part extraction unit 31, an operator position detection unit 33, a neighbor position detection unit 35, and a line-of-sight detection unit 37.

  Human part extraction unit 31 detects an operator who operates MFP 100. The human part extracting unit 31 extracts a part in which a person is represented from the image. For example, a portion similar to the shape of a person's face, neck, and shoulder in the image is determined as a portion where the person is represented. Further, a part having the same color as the face color determined in advance from the image may be determined as a part representing a person. Furthermore, a moving body part may be specified from a difference between two or more images input from the camera 140 at different times, and the moving body part may be determined as a part representing a person. The human part extracting unit 31 outputs position information indicating the position in the image of the part representing the person from the image to the operator position detecting unit 33 and the neighbor position detecting unit 35. When extracting a plurality of portions where a person is represented from the image, the human part extraction unit 31 uses, for each of the plurality of portions where the person is represented, position information indicating the position of the portion in the image as an operator position detection unit. 33 and the neighbor position detector 35.

  Operator position detection unit 33 receives one or more pieces of position information from human part extraction unit 31 and detects the position of the operator who operates MFP 100. The position is a relative position with respect to the display surface of the display unit 120 and is determined by a distance between the position in the image and the display surface of the display unit 120. Here, since the position is detected based on the image output by the camera 140, the imaging range of the camera 140 is regarded as the same as the display surface of the display unit 120, and the relative position with respect to the camera 140 is determined as the display surface of the display unit 120. Detected as a relative position with respect to. Operator position detection unit 33 identifies the operator as an operator who operates MFP 100 in response to detection of the position indicated by touch panel 130 by the operator. When one piece of position information is input from the human part extraction unit 31, the operator position detection unit 33 displays the position on the display surface specified by the position information and the distance input from the camera 140 using the MFP 100. Determine the position of the operator to operate. When two or more pieces of position information are input from the human part extraction unit 31, the operator position detection unit 33 has a plurality of distances respectively corresponding to a plurality of positions on the display surface specified by the two or more pieces of position information. Input from the camera 140. When two or more pieces of position information are input from the human part extraction unit 31, the operator position detection unit 33 is specified by position information closest to the position detected by the touch panel 130 from among the two or more pieces of position information. The position on display surface of display unit 120 and the distance input from camera 140 are determined as the position of the operator who operates MFP 100. Operator position detection unit 33 outputs a set of position information and distance indicating the position of the operator who operates MFP 100 to neighbor position detection unit 35 and display position determination unit 13.

  In addition, when a plurality of different positions are detected by the touch panel 130, the operator position detection unit 33 determines the positions of the plurality of operators. Specifically, the operator position detection unit 33, when a plurality of positions are detected by the touch panel 130 from two or more pieces of position information input from the human part extraction unit 31, each of the detected plurality of positions. , The position information closest to the position on the display surface of the display unit 120 and the distance input from the camera 140 are determined as the position of the operator who operates the MFP 100. For each of a plurality of operators who operate MFP 100, operator position detection unit 33 outputs a set of position information and distance indicating the position of the operator to neighbor position detection unit 35 and display position determination unit 13. .

  The neighbor position detection unit 35 receives a set of position information and distance indicating the position of the operator who operates the MFP 100 from the operator position detection unit 33, and receives one or more pieces of position information from the human part extraction unit 31. Then, the positions of neighbors other than the operator who operates MFP 100 are detected. When two or more pieces of position information are input from the human part extraction unit 31, the neighbor position detection unit 35 has a plurality of distances respectively corresponding to a plurality of positions in an image specified by the two or more pieces of position information. 140 is input. When two or more pieces of position information are input from the human part extraction unit 31, the neighbor position detection unit 35 receives information other than the position information included in the group input from the operator position detection unit 33 among the two or more pieces of position information. One or more pieces of position information are determined as position information of one or more neighbors. The neighbor position detection unit 35 determines the position in the image specified by the determined position information and the distance input from the camera 140 with respect to the position as the position of the neighbor. The neighbor position detection unit 35 outputs a set of position information and distance indicating the position of the neighbor to the line-of-sight detection unit 37 and the display position determination unit 13.

  The line-of-sight detection unit 37 receives a set of position information and distance indicating the position of the neighbor from the neighbor position detection unit 35 and detects the line of sight of the neighbor. The line-of-sight detection unit 37 detects the line of sight of the neighbor by analyzing the image input from the camera 140. Specifically, the eye area is extracted from the position specified by the position information of the neighbor in the image input from the camera 140, and the direction in which the pupil faces from the eye area is determined. The line-of-sight detection unit 37 displays the line-of-sight of the neighbor based on the distance of the neighbor, the position specified by the position information of the neighbor in the image input from the camera 140, and the determined direction. The position that intersects the display surface 120 is detected as the line-of-sight position. The line-of-sight detection unit 37 does not detect the line-of-sight position when the line of sight of the neighbor does not intersect the display surface of the display unit 120, in other words, when the neighbor is looking at a portion other than the display surface of the display unit 120. The line-of-sight detection unit 37 outputs the detected line-of-sight position to the display position determination unit 13 and the display control unit 15 when detecting the line-of-sight position, and indicates that the line-of-sight position is not detected when the line-of-sight position is not detected. The detection signal is output to the display position determination unit 13 and the display control unit 15.

  Display position determination unit 13 receives a set of position information indicating the position of the operator who operates MFP 100 from operator position detection unit 33 and a distance, and position information indicating the position of the neighbor from neighbor position detection unit 35. And a distance pair are input, and a line-of-sight position or an undetected signal is input from the line-of-sight detection unit 37.

  The display position determination unit 13 does not determine the display position when an undetected signal is input from the line-of-sight detection unit 37. This is because when a non-detection signal is input from the line-of-sight detection unit 37, the neighbor does not look at the display surface of the display unit 120, so it is not necessary to make the content described later visible to the neighbor. The display position determination unit 13 receives a set of position information and distance indicating the position of the operator who operates the MFP 100 from the operator position detection unit 33, and the neighbor position detection unit 35 receives the position of the neighbor. When a set of position information and distance indicating “” is not input, the display position is not determined. When the pair of the position information indicating the position of the neighbor and the distance is not input from the neighbor position detection unit 35, there is no neighbor, and when there is no neighbor, the content described later is transferred to the operator. It is enough if you can see. When the undetected signal is input from the line-of-sight detection unit 37 or when the combination of the position information indicating the position of the neighbor and the distance is not input from the neighbor position detection unit 35, the display position determination unit 13 displays A normal display signal is output to the control unit 15.

  When the line-of-sight position is input from the line-of-sight detection unit 37, the display position determination unit 13 determines the display position for displaying the operation screen. The display position is a position on the display surface of the display unit 120. Display position determination unit 13 displays the position at which the operation screen for the operator to input an operation is arranged on the display surface of display unit 120 based on the position of the operator who operates MFP 100 and the position of the neighbor. Determine as. Specifically, the display position determination unit 13 determines the relative position of the operator with respect to the camera 140 from the set of the position information indicating the position of the operator and the distance, and calculates the position information indicating the position of the neighbor and the distance. The relative position of the neighbor with respect to the camera 140 is determined from the set. Then, the position where the straight line connecting the relative position of the neighbor to the camera 140 and the operator relative to the camera 140 intersects the display surface of the display unit 120 is determined as the display position.

  The display position determination unit 13 receives a set of position information indicating the position of the operator who operates the MFP 100 from the operator position detection unit 33 and a distance, and also displays a neighbor's position from the neighbor position detection unit 35. Each time a set of position information indicating a position and a distance is input, the display position is determined. In other words, the display position determination unit 13 determines the display position according to a change in the relative position of the operator with respect to the camera 140 or the relative position of the neighbor with respect to the camera 140.

  The display position determination unit 13 includes a blind spot area determination unit 14. The blind spot area determination unit 14 determines a blind spot area that is a blind spot of a neighbor on the display surface of the display unit 120. The area of the operator included in the image output by the camera 140 has a width in the left-right direction. For this reason, the display position determination unit 13 determines that the rightmost end of the operator is a relative position with respect to the camera 140 and the leftmost end of the operator based on the combination of the image output from the camera 140, position information indicating the position of the operator, and the distance. Determines the relative position to the camera 140. Then, the position where the straight line connecting the relative position of the neighbor with respect to the camera 140 and the relative position of the rightmost end of the operator with respect to the camera 140 intersects the display surface of the display unit 120 and the relative position of the neighbor with respect to the camera 140 of the operator. A region between the position where the straight line connecting the relative positions of the leftmost end to the camera 140 intersects the display surface of the display unit 120 is determined as a blind spot region. When there are a plurality of operators, the blind spot area determination unit 14 determines a blind spot area for each of the plurality of operators.

  In addition, the display position determination unit 13 may receive a plurality of sets of position information and distance indicating the position of the neighbor from the neighbor position detection unit 35. The blind spot area determination unit 14 determines a blind spot area for each of a plurality of neighbors, and when there is an overlap area where all of the plurality of blind spot areas respectively corresponding to the plurality of neighbors overlap, the size of the overlap area is a predetermined size. If so, the overlapping area is determined to be a blind spot area, but if the size of the overlapping area is smaller than a predetermined size, the blind spot area is not determined. The predetermined size is determined by the size of the operation screen, and is the size after the operation screen is reduced to a size that can be reduced.

  When the blind spot area is determined by the blind spot area determination section 14, the display position determination section 13 outputs the display position and the blind spot area to the display control section 15. When the blind spot area is not determined by the blind spot area determining section 14, the display position determining section 13 outputs a determination impossibility signal to the undecided time processing section 21 and the display control section 15. The undetermined time processing unit 21 executes a series of predetermined processes in response to the input of the determination impossible signal from the display position determining unit 13. Details of the undetermined time processing unit 21 will be described later.

  The display control unit 15 controls the display unit 120. The display control unit 15 has a case where a display position and a blind spot area are input from the display position determination unit 13, a case where a normal display signal is input, and a case where a determination impossible signal is input. The display control unit 15 includes an operation screen display unit 41 and a content display unit 51.

  The operation screen display unit 41 does not display the operation screen on the display unit 120 when a determination impossible signal is input from the display position determination unit 13. When a normal display signal is input from the display position determining unit 13, the operation screen display unit 41 displays the operation screen on the display unit 120 by arranging the operation screen at a default position on the display surface of the display unit 120. . This is because the normal display signal is input when the neighbor is not looking at the display surface of the display unit 120 or when there is no neighbor, so it is sufficient that the operator can see the content.

  When the display position and the blind spot area are input from the display position determination unit 13, the operation screen display unit 41 arranges the operation screen at a position determined by the display position on the display surface of the display unit 120 and displays it on the display unit 120. indicate. When a plurality of operators are detected, the operation screen display unit 41 receives a plurality of sets of display positions and blind spot areas from the display position determination unit 13. When a plurality of sets of display positions and blind spot areas are input from the display position determination unit 13, the operation screen display unit 41 displays the same number of operation screens as the number of sets at the display position on the display surface of the display unit 120. It is arranged at a fixed position and displayed on the display unit 120.

  The operation screen display unit 41 includes a movement display control unit 43, a movement prohibition unit 45, and a size determination unit 47. When the operation screen does not fit in the blind spot area, the size determination unit 47 determines the reduction rate of the operation screen based on the size of the operation screen and the size of the blind spot area so that the operation screen fits in the blind spot area. The operation screen display unit 41 reduces the operation screen at the reduction rate determined by the size determination unit 47, arranges the reduced operation screen at a position determined by the display position on the display surface of the display unit 120, It is displayed on the display unit 120.

  The operation reception unit 17 receives an operation input by the operator on the operation screen displayed by the operation screen display unit 41. When the operator designates the display surface of the display unit 120, the position designated by the touch panel 130 is detected. If the position detected by touch panel 130 is within the operation screen, operation reception unit 17 receives an operation assigned to the position instructed on the operation screen. For example, when the operation screen includes an execution instruction button to which a command is assigned, if the position detected by the touch panel 130 is the execution instruction button, an operation for instructing execution of the command assigned to the execution instruction button is accepted. When the operation screen includes a transition button to which another operation screen is assigned, an operation for instructing display of the operation screen assigned to the transition button is received. Further, the operation reception unit 17 outputs operation identification information for identifying the received operation to the process execution unit 19 and the movement prohibition unit 45.

  The process execution unit 19 executes the process according to the operation specified by the operation identification information input from the operation reception unit 17. The processing executed by the processing execution unit 19 includes processing for reading an image of a document, processing for forming an image of data, processing for transmitting and receiving a facsimile, processing for receiving data from the outside, processing for transmitting data to the outside, and storing in the HDD 116. Including a process for managing the generated data. In particular, when content is displayed on the display unit 120, the processing execution unit 19 performs processing for forming an image of the content displayed on the display unit 120, and transmits the content displayed on the display unit 120 to the outside. Execute the process.

  Each time the display position is input from the display position determination unit 13, the movement display control unit 43 arranges the operation screen at a position determined by the display position on the display surface of the display unit 120 and displays the operation screen on the display unit 120. For this reason, the movement display control unit 43 moves the operation screen on the display surface of the display unit 120 in response to a change in the position determined by the display position input from the display position determination unit 13.

  The movement prohibition unit 45 prohibits movement of the operation screen by the movement display control unit 43 while the operator inputs an operation. Specifically, the movement prohibition unit 45 determines that the operator is inputting an operation until a predetermined time elapses after the operation identification information is input from the operation reception unit 17. The movement prohibition unit 45 outputs a movement prohibition signal to the movement display control unit 43 in response to the operation identification information input from the operation reception unit 17. The movement prohibition unit 45 sends a release signal to the movement display control unit when a predetermined time elapses without the operation identification information being input from the operation reception unit 17 since the operation identification information was last input from the operation reception unit 17. Output to 43.

  The movement display control unit 43 displays the operation screen even if the display position is input from the display position determination unit 13 until the release signal is input after the movement prohibition signal is input from the movement prohibition unit 45. The state in which the operation screen is displayed at the position determined by the display position input from the display position determination unit 13 is maintained before the movement prohibition signal is input from the movement prohibition unit 45 without being moved on the display screen 120. .

  The content display unit 51 displays content on the display unit 120. The content is an image. The image includes a still image and a moving image. The content display unit 51 determines a content to be displayed from among a plurality of contents stored in advance in the HDD 116. The content may be data stored in an external computer. The content to be displayed may be determined in advance or may be selected at random.

  When a non-decisable signal or a normal display signal is input from the display position determination unit 13, the content display unit 51 arranges the content determined as a display target in a predetermined region on the display surface of the display unit 120. Are displayed on the display unit 120. The predetermined region can be arbitrarily determined, and may be the entire display surface. When the normal display signal is input, the operation screen is displayed by the operation screen display unit 41. In this case, the operation screen display unit 41 superimposes the operation screen on the content displayed by the content display unit 51. To display.

  The content display unit 51 includes a content position determination unit 53 and a content scaling unit 55. When the display position and the blind spot area are input from the display position determination unit 13, the content position determination unit 53 determines the position where the content is displayed on the display surface of the display unit 120. The content position determination unit 53 determines a displayable area other than the blind spot area input from the display position determination unit 13, and determines a position for displaying the content within the determined displayable area. When the content does not fit in the displayable area determined by the content position determination unit 53, the content scaling unit 55 has a size that allows the content to fit in the displayable area based on the size of the content and the size of the displayable area. The content reduction rate is determined as follows. The content display unit 51 reduces the content at the reduction rate determined by the content scaling unit 55, and arranges the reduced content at the position determined by the content position determination unit 53 on the display surface of the display unit 120. And displayed on the display unit 120.

  When one displayable area is determined by the content position determination unit 53, the content display unit 51 displays content in one displayable area. When the content position determination unit 53 determines two or more displayable areas, the content display unit 51 displays the content in each of the two or more displayable areas. In this case, the content display unit 51 displays the content in each of two or more displayable areas. The content display unit 51 generates two or more divided contents by dividing the content into the same number as the two or more displayable areas, and displays the two or more divided contents in the two or more displayable areas, respectively. May be.

  FIG. 4 is a diagram illustrating an example of detailed functions of the content position determination unit. Referring to FIG. 4, content position determination unit 53 includes a rectangular region extraction unit 61, a maximum region determination unit 63, a plurality of places determination unit 65, and a line-of-sight direction determination unit 67.

  The rectangular area extraction unit 61 extracts a rectangular area from the area other than the blind spot area on the display surface of the display unit 120. If the area other than the blind spot area is rectangular on the display surface of the display unit 120, the rectangular area extraction unit 61 extracts the area other than the blind spot area itself as one rectangular area. When extracting one rectangular area, the rectangular area extracting unit 61 determines that rectangular area as a displayable area.

  When the area other than the blind spot area is composed of two rectangular areas on the display surface of the display unit 120, the rectangular area extracting unit 61 extracts the two rectangular areas as two rectangular areas, respectively. When the area other than the blind spot area is not rectangular in the display surface of the display unit 120, the rectangular area extraction unit 61 calculates a rectangular area with each side of the area other than the blind spot area as one side in the display surface of the display unit 120. Extract as a rectangular area. In this case, the same number of rectangular areas as the sides of the area other than the blind spot area in the display surface of the display unit 120 are extracted.

  When extracting two or more rectangular areas, each of the two or more rectangular areas is set to be effective among the maximum area determining unit 63, the multiple location determining unit 65, and the line-of-sight direction determining unit 67. Output to one. It may be determined in advance which of the maximum area determining unit 63, the plurality of site determining units 65, and the line-of-sight direction determining unit 67 is to be valid. For example, an administrator who manages MFP 100 may set MFP 100 to enable any one of maximum area determination unit 63, multi-location determination unit 65, and line-of-sight direction determination unit 67.

  The maximum area determination unit 63 determines a rectangular area having the largest area among two or more rectangular areas input from the rectangular area extraction unit 61 as a displayable area. The plurality of location determination unit 65 determines each of two or more rectangular regions that do not overlap each other from the two or more rectangular regions input from the rectangular region extraction unit 61 as displayable regions. Specifically, the plurality of places determination unit 65 selects a rectangular region having the maximum area from two or more rectangular regions input from the rectangular region extraction unit 61. Next, the plurality of places determination unit 65 selects a rectangular area that does not overlap with any rectangular area selected from the remaining two or more unselected rectangular areas and has the largest area. The plurality of places determination unit 65 repeats this selection to determine two or more selected rectangular areas as displayable areas. The line-of-sight direction determination unit 67 determines a rectangular area closest to the line-of-sight position input from the line-of-sight detection unit 37 among two or more rectangular areas input from the rectangular area extraction unit 61 as a displayable area.

  The undetermined time processing unit 21 executes a series of predetermined processes in response to the input of the determination impossible signal from the display position determining unit 13. Specifically, the undecided time processing unit 21 sets print conditions for forming the content image displayed on the display unit 120 and causes the processing execution unit 19 to execute processing for forming the content image.

  FIG. 5 is a diagram illustrating an example of detailed functions of the undetermined time processing unit. Referring to FIG. 5, the undecided time processing unit 21 includes a set value image forming unit 71, a set value acquiring unit 73, a setting unit 75, and an execution instruction unit 77. The set value image forming unit 71 controls the image forming unit 160 and the paper feeding unit 170 in response to the input of the non-decision signal from the display position determining unit 13, and sets the set value description image to the image forming unit 160. The sheet is formed on a sheet supplied from the sheet feeding unit 170. The setting value description image is an image having an area for describing a setting value that defines an image forming condition. The image forming conditions include, for example, the number of sheets on which an image is to be formed, designation of either a color to be printed in a plurality of colors or a monochrome to be printed in a single color, a magnification for enlarging or reducing the image, and the like.

  The operator who operates MFP 100 ejects the sheet on which the setting value description image is formed from MFP 100, so that the operator can obtain the sheet on which the setting value description image is formed. The set value to be defined can be described with a pen. After the operator describes the setting value on the paper on which the setting value description image is formed, and the operator inputs an operation for setting the paper on the original reading unit 150 and causing the MFP 100 to read the paper, the original reading unit 150 applies the operation to the paper. The formed set value description image and the handwritten image of the set value described by the operator are read.

  The setting value acquisition unit 73 analyzes the image output by the document reading unit 150 reading the paper, and recognizes the character of the handwritten image having the setting value described by the operator, thereby acquiring the setting value. The set value acquisition unit 73 outputs the acquired set value to the setting unit 75. The setting unit 75 sets a setting value input from the setting value acquisition unit 73. In this case, the setting unit 75 sets the content displayed on the display unit 120 as data to be subjected to image formation. Since the content displayed on the display unit 120 is stored in the HDD 116, the setting unit 75 sets the content stored in the HDD 116 as data to be subjected to image formation. After setting the setting value, the setting unit 75 outputs an image formation instruction to the process execution unit 19. The process execution unit 19 controls the image forming unit 160 and the paper feeding unit 170 in response to the input of an image formation instruction from the setting unit 75 and stores them in the HDD 116 according to the setting values set by the setting unit 75. The image of the content currently displayed on the display part 120 among the displayed data is formed. Thus, the sheet on which the content image displayed on display unit 120 is formed is discharged from MFP 100.

  FIG. 6 is a first diagram for explaining the blind spot area. FIG. 6 schematically shows the MFP 100 as viewed from above. Referring to FIG. 6, the operator is located at position P1 at a distance L1 from the display surface of display unit 120, and the neighbor is located at position P2 at a distance L2 from the display surface of display unit 120. However, L1 <L2. A display position is a position P3 where a straight line connecting the neighbor position P2 and the operator position P1 intersects the display surface of the display unit 120.

  Further, a straight line connecting the neighbor position P2 and the operator's right end position WR intersects at the display surface position P5 of the display unit 120, and connects the neighbor position P2 and the operator's left end position WL. The straight lines intersect at a position P4 on the display surface of the display unit 120. In this case, the area between the position P4 and the position P5 on the display surface of the display unit 120 is a blind spot area for a nearby operator.

  FIG. 7 is a first diagram illustrating an example of a blind spot area in the display surface. FIG. 7 shows a blind spot area in the case where the operator and the neighbor are at the positions shown in FIG. Referring to FIG. 7, display surface 120 </ b> A of display unit 120 can be displayed on a rectangular blind spot area 301 between a vertical straight line passing through position P <b> 4 and a vertical straight line passing through position P <b> 5, and on both sides of blind spot area 301. Regions 303 and 304.

  FIG. 8 is a second diagram for explaining the blind spot area. FIG. 8 schematically shows the MFP 100 viewed from above. Referring to FIG. 8, operator A is located at position P1 at a distance L1 from the display surface of display unit 120, operator B is located at position P1 ′ at a distance L1 ′ from the display surface of display unit 120, and the display is performed. A neighbor is located at a position P2 at a distance L2 from the display surface of the unit 120. However, L1 <L2 and L1 ′ <L2. A position P3 where the straight line connecting the position P2 of the neighbor and the position P1 of the operator A intersects the display surface of the display unit 120 is the display position for the operator A. A straight line connecting the position P2 of the neighbor and the position WR1 of the right end of the operator A intersects at the position P5 on the display surface of the display unit 120, and the position P2 of the neighbor and the position WL1 of the left end of the operator A A straight line connecting the lines intersects at a position P4 on the display surface of the display unit 120. In this case, the area between the position P4 and the position P5 on the display surface of the display unit 120 is a blind spot area for the operator A of the neighbor.

  Furthermore, a position P3 ′ at which a straight line connecting the position P2 of the neighbor and the position P1 ′ of the operator B intersects the display surface of the display unit 120 is a display position for the operator B. In addition, a straight line connecting the position P2 of the neighbor and the position WR2 of the right end of the operator B intersects at the position P5 ′ on the display surface of the display unit 120, and the position P2 of the neighbor and the position WL2 of the left end of the operator B A straight line connecting the two crosses at a position P4 ′ on the display surface of the display unit 120. In this case, the area between the position P4 'and the position P5' on the display surface of the display unit 120 is a blind spot area for the operator B of the neighbor.

  FIG. 9 is a second diagram illustrating an example of a blind spot area on the display surface. FIG. 9 shows a blind spot area in the case where the operator A, the operator B, and the neighbors are at the positions shown in FIG. Referring to FIG. 9, display surface 120A of display unit 120 has a rectangular blind spot area 301 between a vertical straight line passing through position P4 and a vertical straight line passing through position P5, and a vertical straight line passing through position P4 ′. And a rectangular blind area 301A between the vertical line passing through the position P5 ′ and displayable areas 303 and 303A on both sides of the blind area 301.

  FIG. 10 is a diagram for explaining a plurality of rectangular regions. Referring to FIG. 10, display surface 120 </ b> A of display unit 120 includes a blind spot area 401. Here, the blind spot area 401 is determined in consideration of the height direction using the eye heights of the neighbor and the operator. In this case, the area other than the blind spot area 401 on the display surface 120A is not a rectangle, and the area other than the blind spot area 401 on the display face 120A can be divided into a rectangle A to a rectangle E. The rectangle including the right side of the display surface 120A is a rectangle obtained by combining the rectangle D and the rectangle E. The rectangle including the left side of the display surface 120A is a rectangle obtained by combining the rectangle A and the rectangle B. The rectangle including the upper side of the display surface 120A is a rectangle obtained by combining the rectangle B, the rectangle C, and the rectangle D. In this case, if the area of the rectangle combining the rectangle D and the rectangle E including the right side of the display surface 120A is the maximum, the rectangular area including the rectangle D and the rectangle E including the right side of the display surface 120A can be displayed. To be determined. Further, the rectangle having the largest area that does not overlap with the rectangular area obtained by combining the rectangle D and the rectangle E including the right side of the display surface 120A is a rectangle obtained by combining the rectangle A and the rectangle B including the left side of the display surface 120A.

  FIG. 11 is a flowchart illustrating an example of the flow of display control processing according to the first embodiment. The display control process is a process executed by CPU 111 when CPU 111 provided in MFP 100 according to the first embodiment executes a display control program stored in ROM 113, EEPROM 115, HDD 116, or CD-ROM 118A. Referring to FIG. 11, CPU 111 displays the content on the entire display surface of display unit 120 (step S01). Content to be displayed is determined from among a plurality of contents stored in advance in HDD 116. The content to be displayed may be determined in advance or may be selected at random.

  In the next step S02, it is determined whether an operator who operates MFP 100 has been detected. The process waits until an operator is detected (NO in step S02). If an operator is detected (YES in step S02), the process proceeds to step S03. When an operator is detected based on an image output from the camera 140 and the touch panel 130 detects a position designated by the operator, the operator closest to the detected position is detected as an operator who operates the MFP 100. When the touch panel 130 detects a plurality of positions, an operator closest to the position is detected for each of the plurality of positions.

  In step S03, the position of the operator is acquired, and the process proceeds to step S04. The position in the image captured by the camera 140 of the operator detected in step S02 and the distance to the operator measured by the camera 140 are acquired. When a plurality of operators are detected, the position in the image captured by the camera 140 and the distance to the operator measured by the camera 140 are acquired for each of the plurality of operators.

  In step S04, it is determined whether or not a neighbor has been detected. If an operator other than the operator is included in the image captured by the camera 140, an operator other than the operator is detected as a neighbor. If a neighbor is detected, the process proceeds to step S05. If not, the process proceeds to step S08. In step S08, the operation screen is arranged at a position determined by default on the display surface of display unit 120 and displayed on display unit 120, and the process proceeds to step S09.

  In step S05, the position of the neighbor detected in step S04 is acquired, and the process proceeds to step S06. The position in the image captured by the neighbor's camera 140 detected in step S04 and the distance to the neighbor measured by the camera 140 are acquired as the position of the neighbor. When a plurality of neighbors are detected, the position in the image captured by the camera 140 and the distance to the neighbor measured by the camera 140 are acquired for each of the plurality of neighbors. In step S06, an operation screen display process is executed, and the process proceeds to step S07. Although details of the operation screen display process will be described later, this is a process of displaying the operation screen on the display unit 120, and there are cases where the operation screen is displayed and the operation screen is not displayed. The case where the operation screen is not displayed is a case where there is no blind spot area for displaying the operation screen, for example, when there are a plurality of neighbors.

  In step S07, it is determined whether an operation screen is displayed on display unit 120 or not. If the operation screen is displayed, the process proceeds to step S09; otherwise, the process proceeds to step S19. In step S19, an undecided execution process is executed, and the process ends. Details of the undecided execution process will be described later.

  In step S09, it is determined whether an operation has been accepted. If the touch panel 130 detects the position on the operation screen displayed on the display unit 120 in step S06 or step S08, the operation is accepted. If the operation is accepted, the process proceeds to step S10; otherwise, the process proceeds to step S15.

  In step S10, the timer is restarted, and the process proceeds to step S11. The timer is started when an operation screen is displayed on the display unit 120 in an operation screen display process to be described later, measures the elapsed time since the operation screen is displayed, and is restarted in step S10. The elapsed time since the last input by the user is counted.

  In step S11, the process branches according to the operation accepted in step S09. If the operation accepted in step S09 is an execution instruction operation, the process proceeds to step S12. If the operation accepted in step S09 is a setting operation, the process proceeds to step S13. The operation accepted in step S09 is a screen transition operation. If so, the process proceeds to step S14. In step S12, the image forming unit 160 and the paper feeding unit 170 are controlled to form the image of the content displayed on the display unit 120 in step S01 on a sheet, and the process ends. In step S13, a setting value determined by the setting operation is set, and the process proceeds to step S15. In step S14, the operation screen displayed in step S06 is switched to the operation screen display determined by the screen transition operation, and the process proceeds to step S15.

  In step S15, the operator's position is acquired in the same manner as in step S03, and the process proceeds to step S16. In step S16, the position of the neighbor is acquired in the same manner as in step S05, and the process proceeds to step S17. In step S17, it is determined whether or not the position of the operator or the position of the neighbor has changed. If at least one of the position of the operator or the position of the neighbor has changed, the process proceeds to step S18. If not, the process returns to step S09. In step S18, it is determined whether the timer value is equal to or greater than a threshold value TH. If the timer value is equal to or greater than threshold value TH, the process returns to step S06; otherwise, the process returns to step S09. When the operation screen is displayed on the display unit 120 and the threshold value TH has elapsed without receiving an operation, or when the operation screen has been last received and the threshold value TH has elapsed, the process proceeds to step S06. The operation screen is displayed again. In other words, when the threshold value TH elapses after the operation screen is displayed, or when the threshold value TH elapses after the operator inputs an operation, the operation screen is displayed again. Do not move the operation screen. Thereby, the operation | work which an operator inputs operation according to an operation screen can be made easy.

  If a plurality of operators are detected in step S02, the processing of steps S06 to S18 is executed for each of the plurality of operators, and an operation screen is displayed for each of the plurality of operators.

  FIG. 12 is a flowchart illustrating an example of a flow of operation screen display processing according to the first embodiment. The operation screen display process is a process executed in step S06 of the display control process in FIG. Referring to FIG. 12, CPU 111 detects the line-of-sight position of the neighbor (step S <b> 21). The line of sight of the neighbor is detected by analyzing the image input from the camera 140. Specifically, the eye area of the neighbor is extracted from the image input from the camera 140, and the direction in which the pupil faces from the eye area is determined. Based on the distance of the neighbor, the position of the neighbor in the image input from the camera 140, and the determined direction, the position at which the line of sight of the neighbor intersects the display surface of the display unit 120 is determined as the line-of-sight position. Detect as. In the next step S <b> 22, it is determined whether or not the line-of-sight position is within the display surface of the display unit 120. If the line-of-sight position is within the display surface of display unit 120, the process proceeds to step S23; otherwise, the process proceeds to step S33. In step S33, the operation screen is arranged at a position determined by default on the display surface of the display unit 120 and displayed on the display unit 120, and the process returns to the display control process.

  In step S23, a blind spot area is determined. On the display surface of the display unit 120, a blind spot area that is a blind spot of a neighbor is determined. Specifically, the operator area included in the image output by the camera 140 has a width in the left-right direction. For this reason, from the combination of the image output from the camera 140, position information indicating the position of the operator, and the distance, the operator's rightmost end is the relative position to the camera 140 and the operator's leftmost is the relative position to the camera 140 decide. Then, the position where the straight line connecting the relative position of the neighbor with respect to the camera 140 and the relative position of the rightmost end of the operator with respect to the camera 140 intersects the display surface of the display unit 120 and the relative position of the neighbor with respect to the camera 140 of the operator. A region between the position where the straight line connecting the relative positions of the leftmost end to the camera 140 intersects the display surface of the display unit 120 is determined as a blind spot region. A blind spot area is determined for each of a plurality of neighbors, and when there is an overlap area where all of the plurality of blind spot areas respectively corresponding to the plurality of neighbors overlap, if the size of the overlap area is equal to or larger than a predetermined size, the overlap area Is determined as a blind spot area, but if the size of the overlapping area is smaller than a predetermined size, the blind spot area is not determined. The predetermined size is determined by the size of the operation screen, and is the size after the operation screen is reduced to a size that can be reduced. When there are a plurality of operators, the blind spot area is determined for each of the plurality of operators.

  In the next step S24, it is determined whether or not a blind spot area has been determined. If the blind spot area is determined, the process proceeds to step S25. If not, the process returns to the display control process. When the blind spot area is not determined, there are a plurality of neighbors.

  In step S25, a display position is determined within the blind spot area determined in step S23, and the process proceeds to step S26. In step S26, it is determined whether or not an operation screen can be arranged in the blind spot area. If the operation screen can be arranged in the blind spot area, the process proceeds to step S28; otherwise, the process proceeds to step S27. In step S27, the size of the operation screen is reduced to a size that fits within the blind spot area, and the process proceeds to step S28. In step S28, an operation screen is displayed in the visual region, and the process proceeds to step S29.

  In step S29, a timer is started and the process proceeds to step S30. In step S30, a displayable area is determined. The displayable area is an area other than the blind spot area. Specifically, if the area other than the blind spot area is rectangular, the area other than the blind spot area itself is determined as the displayable area. When the area other than the blind spot area is composed of two rectangular areas, one of the two rectangular areas having a larger area is determined as a displayable area. If the area other than the blind spot area is not rectangular, a rectangular area having one side of each side of the area other than the blind spot area is extracted as a rectangular area, and the rectangular area having the largest area is determined as the displayable area.

  In step S31, the content is reduced so that the size of the content is within the displayable area determined in step S30, and the process proceeds to step S32. In step S32, the content is displayed in the displayable area, and the process returns to the display control process.

  In addition, when the area other than the blind spot area is composed of two rectangular areas, each of the two rectangular areas is set as a displayable area, and when a plurality of rectangular areas are extracted from the area other than the blind spot area, Each rectangular area may be set as a displayable area. In this case, the content may be divided into the number of displayable areas, and a plurality of parts obtained by dividing the content may be displayed in each of the plurality of displayable areas, or a plurality of displays may be performed without dividing the content. You may make it display on each possible area | region.

  FIG. 13 is a flowchart illustrating an example of the flow of execution processing when not determined. The undecided execution process is a process executed in step S19 of FIG. Referring to FIG. 13, CPU 111 prints the setting value description image (step S41), and advances the process to step S42. The image forming unit 160 and the paper feeding unit 170 are controlled to form a setting value description image on a sheet. The setting value description image is an image having an area for describing a setting value that defines an image forming condition. An operator who operates MFP 100 can hold a sheet on which a setting value description image is formed.

  In step S42, the process waits until an original reading instruction is accepted (NO in step S42). When the original reading instruction is accepted (YES in step S42), the process proceeds to step S43. An operator who operates MFP 100 can use a pen to describe setting values that define image forming conditions on paper. When the operator describes the setting value on the sheet on which the setting value description image is formed, sets the sheet in the document reading unit 150 and inputs an operation for causing the MFP 100 to read the document, the document reading instruction is accepted.

  In step S43, the original reading unit 150 recognizes characters in an image output by reading the original. The set value is acquired by recognizing the character of the handwritten image of the set value described by the operator in the image. In the next step S44, a set value acquired by character recognition is set, and the process proceeds to step S46. In step S46, the image forming unit 160 and the paper feeding unit 170 are controlled to form an image of the content displayed on the display unit 120 according to the setting value set in step S45. Thus, the sheet on which the content image displayed on display unit 120 is formed is discharged from MFP 100.

<Modification of undecided processing unit>
FIG. 14 is a diagram illustrating an example of a detailed function of the undecided time processing unit in the modification. Referring to FIG. 14, undecided time processing unit 21 </ b> A includes a remote control unit 81. The remote control unit 81 communicates with a portable information device such as a smartphone or PDA (Personal Digital Assistant) carried by the operator via the communication I / F unit 112 or the short-range communication unit 119, and is remotely operated by the portable information device. Is done. For example, the remote control unit 81 displays a two-dimensional code indicating information for starting communication on the display unit 120 in response to the input of the determination impossible signal from the display position determination unit 13. The information for starting communication includes, for example, a network address assigned to MFP 100. The network address is, for example, an IP address or a MAC address assigned to the MFP 100. The network address may be a URL (Uniform Resource Locator) assigned to a remote operation screen to be described later.

  When the operator operates the portable information device to read the two-dimensional code, the portable information device acquires information for starting communication with the MFP 100. Thereby, a connection can be requested from MFP 100 to portable information device. The remote control unit 81 controls the communication I / F unit 112 or the short-range communication unit 119 to establish a communication path with the portable information device in response to receiving a connection request from the portable information device.

  Remote control unit 81 includes an operation screen transmission unit 83, a set value reception unit 85, and a setting unit 87. When a communication path is established with the portable information device, the operation screen transmission unit 83 transmits a remote operation screen to the portable information device. The remote operation screen here is a remote operation screen for setting a setting value that defines image forming conditions. The portable information device that receives the remote operation screen displays the remote operation screen and returns a setting value input by the operator according to the remote operation screen.

  The set value receiving unit 85 controls the communication I / F unit 112 or the short-range communication unit 119 to receive the set value returned by the portable information device. The setting value receiving unit 85 outputs the setting value received from the portable information device to the setting unit 87.

  The setting unit 87 sets a setting value input from the setting value receiving unit 85. In this case, the setting unit 87 sets the content displayed on the display unit 120 as data to be image-formed. Since the content displayed on the display unit 120 is stored in the HDD 116, the setting unit 87 sets the content stored in the HDD 116 as data to be subjected to image formation. The setting unit 87 outputs an image formation instruction to the processing execution unit 19 after setting the set value. The process execution unit 19 controls the image forming unit 160 and the paper feeding unit 170 in response to the input of an image formation instruction from the setting unit 87, and stores them in the HDD 116 according to the setting values set by the setting unit 87. The image of the content currently displayed on the display part 120 among the displayed data is formed. Thus, the sheet on which the content image displayed on display unit 120 is formed is discharged from MFP 100.

  FIG. 15 is a flowchart illustrating an example of the undecided execution process flow in the modification. The undecided execution process in the modification is a process executed in step S19 of FIG. Referring to FIG. 15, CPU 111 displays the two-dimensional code on display unit 120 (step S51), and the process proceeds to step S52. The two-dimensional code includes an IP address or a MAC address assigned to MFP 100. When the operator operates the portable information device to read the two-dimensional code, the portable information device requests the MFP 100 to connect using the IP address or the MAC address included in the two-dimensional code.

  In step S52, it is determined whether communication I / F unit 112 or short-range communication unit 119 has received a connection request. The process waits until a connection request is received (NO in step S52). If a connection request is received (YES in step S52), the process proceeds to step S53. In step S53, a communication path is established with the portable information device that has transmitted the connection request, and the process proceeds to step S54.

  In step S54, the remote operation screen is transmitted via the communication path established in step S53. The remote operation screen is a remote operation screen for setting image forming conditions. The portable information device that receives the remote operation screen displays the remote operation screen and returns a setting value input by the operator according to the remote operation screen.

  In the next step S55, it is determined whether or not a setting value is received from the portable information device. If the set value is received, the process proceeds to step S56; otherwise, the process proceeds to step S57. In step S56, the setting value received from the portable information device is set, and the process returns to step S54. In step S57, it is determined whether an execution instruction has been received from the portable information device. If an execution instruction is received, the process proceeds to step S58; otherwise, the process proceeds to step S59. In step S58, the image forming unit 160 and the paper feeding unit 170 are controlled to form the image of the content displayed on the display unit 120 according to the set value when set in step S56, and the process returns to the display control process. In step S59, it is determined whether or not the communication path established in step S53 has been disconnected. If the communication path is disconnected, the process returns to the display control process. If not, the process returns to step S55.

<Second Embodiment>
MFP 100 according to the second embodiment differs from MFP 100 according to the first embodiment in that a content image is transmitted to a neighbor when the position for displaying the operation screen cannot be determined. In the following, MFP 100 according to the second embodiment will be described mainly with respect to differences from MFP 100 according to the first embodiment.

  FIG. 16 is a diagram illustrating an example of functions of a CPU included in the MFP according to the second embodiment. The function shown in FIG. 16 is different from the function shown in FIG. 3 in that the display position determination unit 13 and the undecided time processing unit 21 are changed to a display position determination unit 13A and a content transmission unit 21B, respectively. . Other functions are the same as those shown in FIG. Therefore, description is not repeated here.

  The display position determination unit 13A is different from the display position determination unit 13 shown in FIG. 3 in that the display position determination unit 13 and the undecided time processing unit 21 and the display when the blind spot region is not determined by the blind spot region determination unit 14 In contrast to outputting a determination impossible signal to the control unit 15, the display position determination unit 13A outputs a determination impossible signal to the content transmission unit 21B when the blind spot region determination unit 14 does not determine the blind spot region, and the display control unit 15 outputs a normal display signal. When the normal display signal is input, the display control unit 15 displays the operation screen at a position determined by default. Therefore, when there are a plurality of neighbors, the operation screen is displayed on the display unit 120.

  The content transmission unit 21B transmits the content displayed on the display unit 120 to the outside. For example, the content transmission unit 21B controls the short-range communication unit 119 to transmit content. The content transmission unit 21B transmits the content by broadcast, for example, without specifying the transmission destination of the content. As a result, when the neighbor carries the portable information device, the portable information device receives the content. Therefore, if the content is displayed on the portable information device, the neighbor is notified of the content. Can do.

  In addition, content transmitting unit 21B transmits device identification information for identifying MFP 100 and identification information for identifying the content via short-range communication unit 119 without transmitting the content itself. Also good. In response to receiving the device identification information and the identification information transmitted from the short-range communication unit 119, the portable information device transmits the identification information to the MFP 100 specified by the device identification information. The content transmission unit 21B transmits the content identified by the identification information via the communication I / F unit 112 in response to the communication I / F unit 112 receiving the identification information. Thereby, a content can be notified to a neighbor. In addition, the content is not stored in MFP 100 but is stored in association with identification information on a server connected to the Internet, and the portable information device transmits the identification information received from short-range communication unit 119 to the server. Then, the content may be received from the server.

  FIG. 17 is a flowchart illustrating an example of the flow of display control processing according to the second embodiment. Referring to FIG. 17, the difference from the display control process shown in FIG. 11 is that step S06, step S07, and step S19 are changed to step S06A, step S07A, and step S19A, respectively. Since other processes are the same as those shown in FIG. 11, the description thereof will not be repeated here.

  In step S06A, the operation screen display process in the second embodiment is executed. FIG. 18 is a flowchart illustrating an example of a flow of operation screen display processing according to the second embodiment. Referring to FIG. 18, the difference from the operation screen display process in the first embodiment shown in FIG. 12 is that step S24 is changed to step S24A. Since other processes are the same as those shown in FIG. 12, the description thereof will not be repeated here. In step S24A, it is determined whether a blind spot area has been determined. If the blind spot area is determined, the process proceeds to step S25; otherwise, the process proceeds to step S33. In step S33, the operation screen is displayed at a position determined by default. For this reason, when a blind spot area is not determined, such as when there are a plurality of neighbors, an operation screen is displayed on the display unit 120.

  Returning to FIG. 17, in step S07A, it is determined whether or not a blind spot area has been determined in the operation screen display processing executed in step S06A. If the blind spot area is determined, the process proceeds to step S09; otherwise, the process proceeds to step S19A. In step S19A, the short-range communication unit 119 is controlled to transmit the content displayed on the display unit 120 to the outside. As a result, when the neighbor carries the portable information device, the portable information device receives the content. Therefore, if the content is displayed on the portable information device, the neighbor is notified of the content. Can do.

  As described above, the MFP 100 according to the first embodiment functions as an image display device, displays content on the display unit 120, detects the first relative position of the operator with respect to the display surface of the display unit 120, and performs operations. The second relative position of a neighbor other than the person with respect to the display surface of the display unit 120 is detected, and the position where the operation screen is arranged on the display surface of the display unit 120 is determined based on the first relative position and the second relative position. The operation screen is displayed at the determined position, and the display mode of the content is changed. For this reason, the operator can operate the operation screen, and the operation screen can be arranged at a position where a neighbor cannot visually recognize the operation screen.

  In addition, as a change in display mode, MFP 100 determines a blind spot area that is a blind spot of a neighbor on the display surface of display unit 120 and displays content in a displayable area other than the blind spot area. For this reason, since the content is displayed in a displayable area other than the blind spot area that is the blind spot of the neighbor on the display surface of the display unit 120, the content can be displayed in such a manner that the neighbor can visually recognize the content. Therefore, it is possible to allow the neighbor to browse the content while enabling the operation by the operator.

  Further, since MFP 100 determines the size of the operation screen based on the size of the blind spot area, MFP 100 can display the operation screen in an area that can be viewed by the operator but cannot be viewed by the neighbor.

  Further, since MFP 100 changes the size of the content based on the size of the displayable area, the entire content can be displayed.

  In addition, since the MFP 100 extracts a rectangular area from an area other than the blind spot area and determines the extracted rectangular area as a displayable area, the rectangular content can be displayed in the rectangular area.

  Further, when a plurality of rectangular regions are extracted, MFP 100 displays the content in the rectangular region having the largest area among the plurality of rectangular regions, so that the content can be displayed as large as possible.

  Further, when a plurality of rectangular areas are extracted, MFP 100 displays the contents in two or more rectangular areas that do not overlap each other among the plurality of rectangular areas, so that the contents can be displayed so as not to overlap. In this case, one content may be displayed in two or more rectangular areas, or the content is divided into the same number as two or more rectangular areas, and the divided two or more contents are displayed in two or more rectangular areas, respectively. You may make it do.

  Further, when a plurality of rectangular areas are extracted, MFP 100 displays the content in the rectangular area closest to the neighbor's line of sight among the plurality of rectangular areas, so that the neighbor can easily see the content.

  In addition, since MFP 100 determines the position on the display surface of the operation screen so that the operator is positioned between the operation screen and the neighbor, the operator can visually recognize the operation screen. It is possible to hide the operation screen from being hidden.

  Further, when a plurality of operators are detected, MFP 100 determines a position on the display surface of each of the plurality of operation screens corresponding to each of the plurality of operators, and displays the plurality of operation screens. The operation by can be accepted.

  In addition, after displaying the operation screen, MFP 100 determines and displays the position of the operation screen in accordance with the change in the relative position of the operator or the neighbor. An operation screen can be displayed at the position.

  Further, since MFP 100 does not move the operation screen while the operator is operating, the operator can easily input the operation.

  Further, since MFP 100 determines the position of the operation screen on the condition that the line of sight of the neighbor is on the display surface of display unit 120, the display mode of the content is changed when the neighbor is looking at the content. If the neighbor is not watching the content, the display mode of the content is not changed. This is because it is sufficient that the operator can visually recognize the content and the operation screen when the neighbor is not watching the content.

  In addition, when there are a plurality of neighbors and the position of the operation screen cannot be determined, the MFP 100 prints the setting value description image without displaying the operation screen, and the user reads the paper on which the setting value is described by handwriting. A set value is extracted by character recognition, and an image of content displayed on the display unit 120 is formed according to the extracted set value. For this reason, the operation by the operator can be accepted without displaying the operation screen.

  Further, MFP 100 in the modified example displays the operation screen because it is remotely controlled by the information communication device carried by the operator without displaying the operation screen when there are a plurality of neighbors and the position of the operation screen cannot be determined. The operation by the operator can be accepted without doing so.

  In addition, when the position of the operation screen is not determined, MFP 100 in the second embodiment displays the operation screen at a predetermined position on the display surface of display unit 120 and controls near field communication unit 119. , Send content to the outside. For this reason, since the information communication device carried by the neighbor receives the content, the neighbor can view the content by displaying the content on the information communication device.

  In the embodiment described above, MFP 100 has been described as an example of an image display device. However, a display control method for causing MFP 100 to execute the display control processing illustrated in FIGS. It goes without saying that the invention can be understood as a display control program that causes the CPU 111 that controls the MFP 100 to execute the control method.

  The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

  100 MFP, 101 main body, 110 main circuit, 111 CPU, 112 communication I / F unit, 113 ROM, 114 RAM, 115 EEPROM, 116 HDD, 117 facsimile unit, 118 external storage device, 118A CD-ROM, 119 near field communication Section, 120 display section, 121 base, 122 column, 123 bottom plate, 130 touch panel, 140 camera, 141 distance measuring section, 150 document reading section, 160 image forming section, 170 sheet feeding section, 11 image analysis section, 13, 13A Display position determination unit, 14 blind spot area determination unit, 15 display control unit, 17 operation reception unit, 19 process execution unit, 21 undecided processing unit, 21A undecided processing unit, 21B content transmission unit, 31 person partial extraction unit , 33 Operator position detector, 35 Neighbor position detector, 37 Detection unit, 41 Operation screen display unit, 43 Movement display control unit, 45 Movement prohibition unit, 47 Size determination unit, 51 Content display unit, 53 Content position determination unit, 55 Content scaling unit, 61 Rectangular area extraction unit, 63 Maximum Area determination unit, 65 Multi-location determination unit, 67 Gaze direction determination unit, 71 Setting value image forming unit, 73 Setting value acquisition unit, 75 Setting unit, 77 Execution instruction unit, 81 Remote control unit, 83 Operation screen transmission unit, 85 Setting value receiving unit, 87 setting unit.

Claims (19)

Display means for displaying content;
An operation accepting means for accepting an operation;
Detection means for detecting an operator who inputs an operation and a neighbor other than the operator;
An operator position detecting means for detecting a first relative position of the operator detected by the detecting means with respect to the display surface of the display means;
A neighbor position detection means for detecting a second relative position of the neighbor detected by the detection means with respect to the display surface of the display means;
Based on the detected first relative position and the detected second relative position, a display position for determining a position at which an operation screen for the operator to input an operation is arranged on the display surface of the display means A determination means;
An image display device comprising: display control means for displaying the operation screen at the determined position and changing a display mode of the content. A blind spot area determining means for determining a blind spot area to be a blind spot of the neighbor in the display surface,
The image display device according to claim 1, wherein the display control unit displays the content in a displayable area other than the determined blind spot area.   The image display device according to claim 2, wherein the display control unit determines the size of the operation screen based on the size of the determined blind spot area.   The image display device according to claim 2, wherein the display control unit changes a size of the content based on a size of the displayable area. The display control means includes a rectangular area extracting means for extracting a rectangular area from an area other than the blind spot area;
The image display device according to claim 2, wherein the extracted rectangular area is determined as the displayable area.   6. The display control unit according to claim 5, wherein when a plurality of rectangular regions are extracted by the rectangular region extraction unit, the display control unit determines a rectangular region having the maximum area among the plurality of rectangular regions as the displayable region. Image display device.   6. The display control means, when a plurality of rectangular areas are extracted by the rectangular area extracting means, determines two or more rectangular areas that do not overlap each other among the plurality of rectangular areas as the displayable area. The image display device described in 1.   The display control means determines, when the rectangular area extraction means extracts a plurality of rectangular areas, a rectangular area closest to the neighbor's line of sight among the plurality of rectangular areas as the displayable area. 5. The image display device according to 5.   The said display position determination means determines the position in the said display surface of the said operation screen so that the said operator may be located between the said operation screen and the said neighbor. The image display device described.   The said display position determination means determines the position in the said display surface of each of the some operation screen respectively corresponding to the said some operator, when a some operator is detected by the said detection means. 10. The image display device according to any one of 9.   The display control display means is a moving display control means for moving the operation screen on the display surface in accordance with a change in position determined by the display position determination means after the operation screen is displayed on the display means. The image display apparatus in any one of Claims 1-10 containing these.   The image display according to claim 11, wherein the display control unit further includes a movement prohibiting unit that prohibits the movement display control unit from moving the operation screen while the operation is received by the operation receiving unit. apparatus. A line-of-sight detection means for detecting the line of sight of the neighbor, further comprising:
The image display device according to claim 1, wherein the display position determining unit determines the position of the operation screen on condition that the detected line of sight is a display surface of the display unit.   The processing further when the display position determining means does not determine the position of the operation screen, further comprising undecided time processing means for prohibiting display of the operation screen by the operation screen display means and accepting an operation by the operator. The image display device according to any one of 13. Image forming means for forming an image on a recording medium;
An image reading means for reading an image formed on the recording medium,
The undecided time processing means includes a set value image forming means for causing the image forming means to form an image for describing a set value for forming an image of the content displayed on the display means,
A character recognizing unit that recognizes a character by analyzing an image obtained by the image reading unit reading an image formed by the image forming unit;
The image display apparatus according to claim 14, further comprising: a setting unit that sets a setting value for causing the image forming unit to form an image based on the recognized character. The undecided time processing means includes an operating device specifying means for specifying an information communication device carried by the operator as an operating device,
The image display device according to claim 14, further comprising remote control means that is remotely controlled by the determined operation device.   When the position of the operation screen is not determined by the display position determination unit, the operation screen display unit further includes a content transmission unit that displays the operation screen at a predetermined position and transmits the content to the outside. The image display device according to claim 1. A display control method executed by an image display device provided with display means for displaying content,
An operation accepting step for accepting an operation;
A detection step of detecting an operator who inputs an operation and a neighbor other than the operator;
An operator position detecting step of detecting a first relative position of the operator detected in the detecting step with respect to the display surface of the display means;
A neighbor position detecting step of detecting a second relative position of the neighbor detected in the detecting step with respect to the display surface of the display means;
Based on the detected first relative position and the detected second relative position, a display position for determining a position at which an operation screen for the operator to input an operation is arranged on the display surface of the display means A decision step;
A display control step of displaying the operation screen at the determined position and changing a display mode of the content. A display control program that is executed by a computer that controls an image display device including display means for displaying content,
An operation accepting step for accepting an operation;
A detection step of detecting an operator who inputs an operation and a neighbor other than the operator;
An operator position detecting step of detecting a first relative position of the operator detected in the detecting step with respect to the display surface of the display means;
A neighbor position detecting step of detecting a second relative position of the neighbor detected in the detecting step with respect to the display surface of the display means;
Based on the detected first relative position and the detected second relative position, a display position for determining a position at which an operation screen for the operator to input an operation is arranged on the display surface of the display means A decision step;
A display control program for causing the computer to execute a display control step of displaying the operation screen at the determined position and changing a display mode of the content.

Images