JP2021086410A - Device, display control method, and program - Google Patents

Abstract

To provide an operation assistance function in accordance with operation status of a user.SOLUTION: An image processing apparatus 10 includes: a user operation detection unit 35 (one example of user operation detection means) which detects user operation received with respect to an operation panel 305; a device information acquisition unit 36 (one example of device information acquisition means) which acquires device information including sensor signals output from a peripheral device (one example of a sensor) arranged in a specific position of the image processing apparatus 10; and a display control unit 33 (one example of display control means) which causes the operation panel 305 to display an operation screen (e.g., operation assistance screen 700) in which processing to be executed by the image processing apparatus 10 can be selected, on the basis of history information indicating a history of detected user operations and the acquired device information.SELECTED DRAWING: Figure 11

Description

The present invention relates to devices, display control methods and programs.

Devices equipped with image processing functions such as MFPs (Multifunction Peripheral / Printer / Product) are equipped with an operation panel having a touch panel function in order to allow the user to select and operate processing functions and setting menus.

Further, there is also known a technique of assisting a user in operation by changing the display form of an operation screen displayed on the operation panel in order to reduce the time and effort of user operation. For example, Patent Document 1 discloses a content in which an operation image for executing a specific process in which the establishment of a start condition is detected is displayed differently from an operation image for executing other processes. There is.

However, the conventional method has a problem that the operation assist according to the real-time operation situation of the user cannot be performed.

In order to solve the above-mentioned problems, the invention according to claim 1 is a device provided with an operation means for accepting a predetermined input operation, and the user operation detection means for detecting the user operation accepted for the operation means. A device information acquisition means for acquiring device information including a sensor signal output from a sensor arranged at a specific position of the device, history information indicating the detected history of the user operation, and the acquired device. It is a device including a display control means for causing the operation means to display an operation screen on which a process to be executed by the device can be selected based on information.

According to the present invention, it is possible to provide an operation assist function according to a user's operation status.

It is a figure for demonstrating the outline of an example of the image processing apparatus which concerns on embodiment. It is a figure which shows an example of the hardware composition of the image processing apparatus which concerns on embodiment. It is a figure which shows an example of the software structure of the image processing apparatus which concerns on embodiment. It is a figure which shows an example of the functional structure of the image processing apparatus which concerns on embodiment. (A) is a conceptual diagram showing an example of an operation history management table according to an embodiment, and (B) is a conceptual diagram showing an example of an operation assist management table according to an embodiment. It is a conceptual diagram which shows an example of the authority information management table which concerns on embodiment. It is a flowchart which shows an example of the operation assist processing in the image processing apparatus which concerns on embodiment. It is a figure which shows an example of the selection screen displayed on the image processing apparatus which concerns on embodiment. It is a figure for demonstrating an example of the screen transition of the display screen displayed on the image processing apparatus which concerns on embodiment. It is a flowchart which shows an example of the user use function guessing process in the image processing apparatus which concerns on embodiment. (A) is a diagram showing an example of an operation assist screen displayed on the image processing device according to the embodiment, and (B) is a diagram showing an example of a recommendation screen displayed on the image processing device according to the embodiment. Is. It is a flowchart which shows an example of the operation assist processing in the image processing apparatus which concerns on modification 1 of embodiment. It is a flowchart which shows an example of the user use function guessing process in the image processing apparatus which concerns on modification 2 of embodiment. It is a flowchart which shows an example of the operation assist processing in the image processing apparatus which concerns on modification 3 of embodiment. It is a flowchart which shows an example of the user use function guessing process in the image processing apparatus which concerns on modification 4 of embodiment.

Hereinafter, modes for carrying out the invention will be described with reference to the drawings. In the description of the drawings, the same elements are designated by the same reference numerals, and duplicate description will be omitted.

● Embodiment ●
● Outline of Image Processing Device FIG. 1 is a diagram for explaining an outline of an example of an image processing device according to an embodiment. The image processing device 10 shown in FIG. 1 provides an operation screen for assisting the user operation in order to reduce the time and effort required for the user operation on the operation panel 305.

The image processing device 10 is, for example, an image forming device having an image processing function such as an MFP (Multifunction Peripheral / Printer / Product), a facsimile, a scanner, or a printer, and a communication function. The image processing device 10 includes an operation unit 30 and a main body unit 50 that are connected by a cable so as to be capable of data communication. The image processing device 10 is an example of the device.

The operation unit 30 receives a screen display and a user operation by the user. The main body unit 50 performs an operation according to the input received by the operation unit 30. The operation unit 30 and the main body 50 may be connected wirelessly (including infrared rays) instead of the wired connection using a cable.

It is conceivable that the image processing device 10 makes the operation unit 30 independent of the main body 50 so that the operation response is not lowered even if the load of the main body 50 is high. For example, the operation unit 30 is equipped with an OS (Operating System) independent of the main body 50 and operates independently of the main body 50.

Further, the main body 50 includes an ADF (Auto Document Feeder) 531 that automatically feeds a document to be image-processed, a motion sensor 533 that detects a user around the image processing device 10, and an image. It has a paper feed tray 535 for replenishing the paper used in the processing.

The ADF531, the motion sensor 533, and the paper feed tray 535 are peripheral devices arranged at specific positions on the main body 50. Such a peripheral device is provided with a sensor for detecting the processing or the state of the main body 50. Peripheral devices output device information including sensor signals. The ADF531 outputs, for example, the presence / absence of an installed document, the paper size, the opening / closing of the ADF531, and the like as device information. Further, the motion sensor 533 outputs, for example, the presence / absence of a person (user) as a detection result as device information. Further, the paper feed tray 535 outputs, for example, the paper size, the remaining amount of paper, the presence / absence of paper, the opening / closing information of the paper feed tray 535, and the like as device information.

The peripheral device of the image processing device 10 shown in FIG. 1 is not limited to this, and other peripheral devices (sensors) according to the application may be provided. The image processing device 10 can output device information including sensor signals such as the remaining amount of toner and the insertion / removal of a LAN cable from these peripheral devices. Further, the image processing device 10 may be a portable handy printer, a handy scanner, or the like, as well as a device installed at a specific place by an administrator or a user.

● Hardware Configuration FIG. 2 is a diagram showing an example of the hardware configuration of the image processing apparatus according to the first embodiment. In the hardware configuration shown in FIG. 2, components may be added or deleted as needed. The image processing device 10 includes a main body unit 50 capable of realizing various functions such as a printer function, a copy function, a scanner function, and a fax function, and an operation unit 30 that receives an input according to a user operation. The operation unit 30 is an operation unit 30 (operation panel) of the main body unit 50 having a control unit of the main body unit 50 and a control unit independent of the main body unit 50.

It should be noted that accepting a user's operation is a concept including accepting information (including a signal indicating a coordinate value of a screen) input according to the user's operation. The main body unit 50 and the operation unit 30 are connected to each other so as to be able to communicate with each other via a dedicated communication path 200. For the communication path 200, for example, a USB (Universal Serial Bus) standard can be used, but any standard may be used regardless of whether it is wired or wireless. The main body unit 50 performs an operation according to the input received by the operation unit 30. Further, the main body 50 can communicate with an external device via, for example, a communication network, and can perform an operation according to an instruction received from the external device.

First, the hardware configuration of the main body 50 will be described. As shown in FIG. 3, the main body 50 includes a CPU (Central Processing Unit) 501, a ROM (Read Only Memory) 502, a RAM (Random Access Memory) 503, an HDD (Hard Disk Drive) 504, and a scanner engine. It includes a 505, a printer engine 506, a FAX control unit 507, a communication I / F (Interface) 508, and a connection I / F 509.

The CPU 501 controls the operation of the entire main body 50. The ROM 502 is a non-volatile memory that stores a program used to drive the CPU 501. The RAM 503 is a volatile memory used as a work area of the CPU 501. HDD 504 is a non-volatile storage medium for storing various data or programs. The CPU 501 controls the operation of the entire main body 50 by executing a program stored in the ROM 502, the HDD 504, or the like with the RAM 503 as a work area (work area). Further, the CPU 501 realizes various functions such as the above-mentioned printer function, copy function, scanner function, and fax function by the scanner engine 505, the printer engine 506, or the FAX control unit 507.

The communication I / F 508 is an interface for communicating with an external device via a communication network. The connection I / F 509 is an interface for communicating with the operation unit 30 via the communication path 200. The device I / F 511 is an interface for connecting to peripheral devices such as the ADF531, the motion sensor 533, and the paper feed tray 535 shown in FIG. Further, the main body 50 is provided with a bus line 510. The bus line 510 is an address bus, a data bus, or the like for electrically connecting each component such as the CPU 501.

Subsequently, the hardware configuration of the operation unit 30 will be described. As shown in FIG. 3, the operation unit 30 includes a CPU 301, a ROM 302, a RAM 303, a flash memory 304, an operation panel 305, a communication I / F 306, and a connection I / F 307.

The CPU 301 controls the operation of the entire operation unit 30. The ROM 302 is a non-volatile memory that stores a program used to drive the CPU 301. The RAM 303 is a volatile memory used as a work area of the CPU 301. The CPU 301 controls the operation of the entire operation unit 30 by executing a program stored in the ROM 302, the flash memory 304, or the like, using the RAM 303 as a work area (work area). Further, the CPU 301 realizes various functions such as displaying information (image) according to the input received by the user operation.

The operation panel 305 accepts various inputs according to user operations and displays various information. For example, various types of information include information according to the received input, information indicating the operating status of the image processing device 10, information indicating the setting state, and the like. The operation panel 305 is composed of, for example, a liquid crystal display (LCD) equipped with a touch panel function, an organic EL (Electro Luminescence) display device equipped with a touch panel function, or the like. Further, the operation panel 305 may be provided with an input unit such as a hardware key or a display unit such as a lamp in addition to or in place of the operation panel 305. The operation panel 305 is an example of an operation means. The communication I / F 306 is an interface for communicating with an external device or the like via a communication network. The connection I / F 307 is an interface for communicating with the main body 50 via the communication path 200. Further, the operation unit 30 includes a bus line 308. The bus line 308 is an address bus, a data bus, or the like for electrically connecting each component such as the CPU 301.

Each of the above programs may be a file in an installable format or an executable format, and may be recorded and distributed on a computer-readable recording medium. Examples of recording media include CD-R (Compact Disc Recordable), DVD (Digital Versatile Disk), Blu-ray (registered trademark) disc, SD card, USB memory and the like. In addition, the recording medium can be provided domestically or internationally as a program product. For example, the image processing device 10 realizes the display control method according to the present invention by executing the program according to the present invention.

● Software configuration Next, the outline of the software configuration of the image processing apparatus 10 will be described with reference to FIG. FIG. 3 is a diagram showing an example of the software configuration of the image processing device according to the embodiment. As shown in FIG. 3, the main body 50 has an application layer 551, a service layer 552, and an OS (Operating System) layer 553. The application layer 551, the service layer 552, and the OS layer 553 are various software stored in the ROM 502, HDD 504, or the like. The main body 50 provides various functions by executing these softwares by the CPU 501.

The software of the application layer 551 is application software (hereinafter, referred to as "application") for operating hardware resources to provide a predetermined function. For example, the application is a copy application for providing a copy function, a scanner application for providing a scanner function, a fax application for providing a fax function, a printer application for providing a printer function, and the like.

The software of the service layer 552 intervenes between the application layer 551 and the OS layer 553. The software of the service layer 552 is software for providing the application with an interface for using the hardware resources included in the main body 50. Specifically, the software of the service layer 552 is software for providing a function of accepting an operation request for a hardware resource and arbitrating the operation request. The operation request received by the service layer 552 is, for example, a request for reading by the scanner engine 505 or printing by the printer engine 506.

The function of the interface by the service layer 552 is provided not only to the application layer 551 of the main body 50 but also to the application layer 351 of the operation unit 30. That is, the application layer 351 of the operation unit 30 has a function of utilizing the hardware resources of the main unit 50 (for example, the scanner engine 505, the printer engine 506, the FAX control unit 507, etc.) through the interface function of the service layer 552. It can be realized.

The software of the OS layer 553 is basic software (operating system) for providing basic functions for controlling the hardware included in the main body 50. The software of the service layer 552 converts the usage request of the hardware resource from various applications into a command that can be interpreted by the OS layer 553 and passes it to the OS layer 553. Then, when the command is executed by the software of the OS layer 553, the hardware resource operates according to the request of the application.

The operation unit 30 has an application layer 351 and a service layer 352 and an OS layer 353. The hierarchical structure of the application layer 351 and the service layer 352 and the OS layer 353 included in the operation unit 30 is the same as that of the main body unit 50. However, the functions provided by the application of the application layer 351 and the types of operation requests that the service layer 352 can accept are different from those of the main body 50. The application of the application layer 351 is software for operating the hardware resource included in the operation unit 30 to provide a predetermined function. The application layer 351 includes software for providing UI (User Interface) functions for performing operations and displays related to the functions (printer function, copy function, scanner function, fax function) provided in the main body 50. included.

In the image processing device 10, the software of the OS layer 553 of the main body 50 and the software of the OS layer 353 of the operation unit 30 are different from each other in order to maintain the independence of the functions. That is, the main body 50 and the operation 30 operate independently of each other in different operating systems. For example, the software of the OS layer 553 of the main body 50 may adopt LINUX (registered trademark), and the software of the OS layer 353 of the operation unit 30 may adopt Android (registered trademark).

As described above, in the image processing device 10 shown in FIG. 3, since the main body 50 and the operation unit 30 operate on different operating systems, the communication between the main body 50 and the operation unit 30 is common. It is performed as communication between different devices, not as interprocess communication within the device. For example, the operation of transmitting the input received by the operation unit 30 (instruction content by user operation) to the main unit 50 (command communication), the operation of the main unit 50 notifying the operation unit 30 of an event, and the like are performed between different devices. It is done as a communication of. As a result, the operation unit 30 can use the function of the main body unit 50 by performing command communication with the main body unit 50. The events notified from the main body 50 to the operation unit 30 include the execution status of the operation in the main body 50, the contents set on the main body 50 side, and the like.

Further, in the image processing device 10 shown in FIG. 3, the power supply to the operation unit 30 is performed from the main body unit 50 via the communication path 200. Therefore, the power supply control of the operation unit 30 can be performed separately (independently) from the power supply control of the main body unit 50.

● Functional configuration Next, the functional configuration of the image processing apparatus according to the embodiment will be described with reference to FIGS. 4 to 7. FIG. 4 is a diagram showing an example of the functional configuration of the image processing device according to the embodiment. Note that FIG. 4 shows a functional configuration related to the processing or operation described later in the image processing apparatus 10.

As described above, the image processing device 10 is composed of an operation unit 30 and a main body unit 50. Of these, the operation unit 30 includes an input / output unit 31, a reception unit 32, a display control unit 33, a judgment unit 34, a user operation detection unit 35, a device information acquisition unit 36, a user use function estimation unit 37, and an operation assist determination unit 38. , Communication unit 39, determination unit 41, authentication unit 42, and storage / reading unit 49. Each of these parts is a function realized or a means for functioning by operating any of the components shown in FIG. 2 by an instruction from the CPU 301 according to a program developed on the RAM 303. Further, the operation unit 30 has a storage unit 3000 constructed by the ROM 302 or the flash memory 304 shown in FIG.

The input / output unit 31 is a function realized by a command from the CPU 301 shown in FIG. 2 and a connection I / F 307, and exchanges various data or information with the main body unit 50. The input / output unit 31 transmits, for example, a predetermined request signal corresponding to the request received by the user's input operation to the main body unit 50.

The reception unit 32 is a function realized by the instruction from the CPU 301 shown in FIG. 2 and the operation panel 305, and receives various inputs to the operation panel 305 from the user. The display control unit 33 is a function realized by the instruction from the CPU 301 shown in FIG. 2 and the operation panel 305, and causes the operation panel 305 to display various screens such as an operation screen that accepts an input operation by the user. The display control unit 33 uses, for example, a WEB browser to display a Web page created by HTML (HyperText Markup Language) or the like on the operation panel 305. The determination unit 34 is a function realized by an instruction from the CPU 301 shown in FIG. 2 and performing various determinations in the operation unit 30.

The user operation detection unit 35 is a function realized by an instruction from the CPU 301 shown in FIG. 2 and detects a user operation in response to a user input received by the reception unit 32. The user operation detection unit 35 detects, for example, information on the operation interval, the number of operations, or the screen transition according to the user operation as the user operation. The device information acquisition unit 36 is a function realized by a command from the CPU 301 shown in FIG. 2 and acquires device information input from the main body 50 via the input / output unit 31.

The user use function estimation unit 37 is a function realized by an instruction from the CPU 301 shown in FIG. 2 and estimates the function of the image processing device 10 that the user wants to use. The user use function estimation unit 37 estimates the user use function according to, for example, the detection result by the user operation detection unit 35 or the device information acquired by the device information acquisition unit 36.

The operation assist determination unit 38 is a function realized by the instruction from the CPU 301 shown in FIG. 2 and determines the operation assist to be provided to the user. The operation assist determination unit 38 determines the display form of the display screen for performing the operation assist, for example, according to the estimation result by the user use function estimation unit 37.

The communication unit 39 is a function realized by a command from the CPU 301 shown in FIG. 2 and a communication I / F 306, and is a function of communicating various data or information with an external device via a communication network. The determination unit 41 is a function realized by an instruction from the CPU 301 shown in FIG. 2 and determines the operation proficiency level of the user who operates the operation panel 305. The authentication unit 42 is a function realized by an instruction from the CPU 301 shown in FIG. 2 and performs an authentication process of a user who operates the operation panel 305.

The storage / reading unit 49 is a function executed by an instruction from the CPU 301 shown in FIG. 2 to store various data in the storage unit 3000 or read various data from the storage unit 3000.

-Operation history management table FIG. 5 (A) is a conceptual diagram showing an example of the operation history management table according to the embodiment. In the storage unit 3000, an operation history management DB 3001 configured by an operation history management table as shown in FIG. 5A is constructed. In this operation history management table, as history information indicating the history of user operations, the date and time when the user operation is accepted and the screen transition information indicating the screen transition of the display screen generated by the user operation are stored in association with each other. .. In addition, a series of operation histories of the user during the operation are accumulated and stored in the operation history management table. For example, in the example of FIG. 5A, the user operation 1 and the user operation 2 are stored as history information in the operation history management table.

-Operation Assist Management Table FIG. 5B is a conceptual diagram showing an example of the operation assist management table according to the embodiment. The storage unit 3000 is constructed with an operation assist management DB 3003 configured by an operation assist management table as shown in FIG. 5 (B). In this operation assist management table, various conditions for performing the operation assist of the assist item are associated and stored with respect to the assist item indicating the operation item for which the operation assist is performed. The conditions for performing the operation assist include the threshold value T of the operation interval of the user operation, the threshold value C of the number of operations of the user operation, the information indicating a specific screen transition on the display screen, and the sensor that transmits the sensor signal used for the operation assist ( Peripheral device) location and user authority information indicating the user's authority are included. If there is an assist item that satisfies the condition for performing operation assist based on the operation history of the user operation and the device information acquired from the peripheral device, the user use function estimation unit 37 infers the item as the user use function. To do.

○ Authority information management table FIG. 6 is a conceptual diagram showing an example of the authority information management table according to the embodiment. In the storage unit 3000, an authority information management DB 3005 configured by an authority information management table as shown in FIG. 6 is constructed. In this authority information management table, each password and authority information indicating the operation authority for the image processing device 10 are associated with the user ID for identifying the user and managed. For example, in the authority information management table shown in FIG. 6, the password of the user ID "tm01" is "abab", and the user authority is "general". The password of the user ID "tm02" is "aabb", and the user authority is "administrator". Based on this authority information management table, the image processing device 10 can make the content of the operation assist different for each user according to the operation authority.

Subsequently, the main body unit 50 has an input / output unit 51, an image processing unit 52, a device information acquisition unit 53, a communication unit 54, and a storage / reading unit 59. Each of these parts is a function realized or a means for functioning by operating any of the components shown in FIG. 2 by an instruction from the CPU 501 according to a program developed on the RAM 503. Further, the main body unit 50 has a storage unit 5000 constructed by the ROM 502 or HDD 504 shown in FIG.

The input / output unit 51 is a function realized by a command from the CPU 501 shown in FIG. 2 and a connection I / F 509, and exchanges various data or information with the operation unit 30. The input / output unit 51 receives, for example, a predetermined request signal transmitted from the operation unit 30. Further, the input / output unit 51 transmits the device information acquired by the device information acquisition unit 53 to the operation unit 30, for example.

The image processing unit 52 is a function realized by the instruction from the CPU 501 shown in FIG. 2 and the scanner engine 505, the printer engine 506, or the FAX control unit 507, and executes various image processing by the image processing device 10. The device information acquisition unit 53 receives device information including a command from the CPU 501 shown in FIG. 2 and a sensor signal from a sensor arranged at a specific position of the image processing device 10 realized by the device I / F 511. It is a function to acquire.

The communication unit 54 is a function realized by a command from the CPU 501 shown in FIG. 2 and a communication I / F 508, and communicates various data or information with an external device via a communication network. The storage / reading unit 59 is a function executed by an instruction from the CPU 501 shown in FIG. 2 to store various data in the storage unit 5000 or read various data from the storage unit 5000.

● Processing or operation of the embodiment Next, the processing or operation of the image processing apparatus according to the embodiment will be described with reference to FIGS. 7 to 11. FIG. 7 is a flowchart showing an example of operation assist processing in the image processing apparatus according to the embodiment.

First, the display control unit 33 of the operation unit 30 displays the selection screen 600 shown in FIG. 8 on the operation panel 305 (step S31). The selection screen 600 shown in FIG. 8 includes a selection button that is pressed when the user performs various processes using the image processing device 10. Further, the selection screen 600 includes a "home" button 610 that is pressed when the screen transitions to the home screen, which is the initial screen of the menu display, and a screen where the previous user operation has been performed. A "back" button 630 to be pressed is included. The user can perform a desired operation by, for example, touching the touch panel type operation panel 305 and selecting a button corresponding to a desired process. The selection screen 600 is an example of an operation screen.

Next, the user operation detection unit 35 of the operation unit 30 detects the user operation when the reception unit 32 accepts the selection of a predetermined button included in the selection screen 600 (step S12). In addition, the user operation detection unit 35 monitors the operation interval of the detected user operation. Here, the operation interval of the user operation is, for example, the time from when the user first touches the operation panel 305 to when the next button is selected, or after the display screen is dimmed in the power saving state and then the next button. Is the time until is selected. Further, the user operation detection unit 35 stores the detected user operation as an operation history in the operation history management DB 3001 (see FIG. 5A).

Next, the determination unit 34 of the operation unit 30 determines whether the operation interval monitored by the user operation detection unit 35 is equal to or greater than the threshold value T (step S13). Specifically, the determination unit 34 refers to the threshold value T of the operation interval stored in the operation assist management DB 3003 (see FIG. 5B), and determines whether or not the operation interval of the user operation has passed the threshold value T. When the operation interval is equal to or greater than the threshold value T (YES in step S13), the determination unit 34 shifts the process to step S17. On the other hand, when the operation interval is shorter than the threshold value T (NO in step S13), the determination unit 34 shifts the process to step S14.

Next, the storage / reading unit 49 of the operation unit 30 reads and acquires the operation history stored in the operation history management DB 3001 (step 14). Then, the determination unit 34 of the operation unit 30 determines whether the number of user operations is equal to or greater than the threshold value C in order to realize the operation assist for the user during the operation with higher accuracy (step S15). Specifically, the determination unit 34 refers to the threshold value C of the number of operations stored in the operation assist management DB 3003, and determines whether the number of operations of the user operation exceeds the threshold value C. When the number of operations is equal to or greater than the threshold value C (YES in step S15), the determination unit 34 shifts the process to step S17.

On the other hand, when the number of operations is less than the threshold value C (NO in step S15), the determination unit 34 shifts the process to step S16. Then, the determination unit 34 determines whether or not a specific screen transition has been performed according to the user operation (step S16). Specifically, the determination unit 34 determines whether the screen transition of the display screen corresponds to a specific screen transition stored in the operation assist management DB 3003. Here, the specific screen transition corresponds to the case where an operation of returning to the original screen is performed after the transition to another screen. When a specific screen transition is performed according to the user operation (YES in step S16), the determination unit 34 shifts the process to step S17. On the other hand, when the determination unit 34 does not perform a specific screen transition according to the user operation (NO in step S16), the determination unit 34 determines that the operation assist for the operating user is unnecessary, and processes from step S12. repeat.

Here, an example of a specific screen transition of the display screen displayed on the operation unit 30 will be described with reference to FIG. FIG. 9 is a diagram for explaining an example of screen transition of the display screen displayed on the image processing device according to the embodiment. First, the display control unit 33 of the operation unit 30 displays a selection screen for detailed copy functions as shown on the screen B when the "copy" button on the screen A shown in FIG. 9 is selected by the user. The display screen is changed to. Further, when the user selects the "back" button 630 on the screen B, the display control unit 33 returns the display screen from the screen B to the original screen A. In this case, the operation unit 30 detects the screen transition of "screen A-> screen B-> screen A" and determines whether the detected screen transition corresponds to a specific screen transition stored in the operation assist management DB 3003. ..

Next, the user use function estimation unit 37 of the operation unit 30 executes a process of estimating the user use function, which is a desired function that the user wants to use (step S17). Here, the details of the user use function estimation process by the image processing device 10 will be described with reference to FIG. FIG. 10 is a flowchart showing an example of user use function estimation processing in the image processing apparatus according to the embodiment.

First, the device information acquisition unit 36 of the operation unit 30 acquires device information of peripheral devices arranged in the image processing device 10 (step S171). Specifically, the device information acquisition unit 53 of the main body unit 50 acquires device information including sensor signals from sensors of each peripheral device via the device I / F 511. Further, the input / output unit 51 of the main body unit 50 transmits the acquired device information to the operation unit 30. Then, the device information acquisition unit 36 of the operation unit 30 acquires the device information received by the input / output unit 31.

Next, the determination unit 34 of the operation unit 30 determines the presence or absence of the original document to be processed by using the device information acquired by the device information acquisition unit 36 (step S172). Specifically, when the sensor signal included in the device information transmitted from the ADF531 indicates that there is a document (YES in step S172), the determination unit 34 shifts the process to step S173. Then, when there is a document to be processed, the user use function estimation unit 37 of the operation unit 30 is likely to perform a copy, a scanner, or a fax function operation on the document to be processed next, so that the user use function can be used. It is presumed to be either copy, scan or fax (step S173).

On the other hand, when the sensor signal included in the device information transmitted from the ADF531 indicates that there is no document (NO in step S172), the determination unit 34 shifts the process to step S174. Then, the user use function estimation unit 37 of the operation unit 30 is likely to perform some setting operation on the image processing device 10 when there is no original to be processed. Therefore, the user use function is a setting menu. Guess (step S174). Next, when the setting operation is performed, the user use function estimation unit 37 has a high possibility that one of the peripheral devices has been operated immediately before, so that the sensor included in the device information acquired in step S171 The presence or absence of a signal change is specified (step S175). Here, the change in the sensor signal is, for example, a change in the value of the sensor signal output from various peripheral devices such as opening / closing the paper feed tray 535, opening / closing the ADF531, replenishing toner, and inserting / removing the LAN cable.

Then, the determination unit 34 determines whether or not there is a setting operation according to the change in the sensor signal by searching the operation assist management DB 3003 (step S176). Specifically, the determination unit 34 changes the sensor signal when there is an assist item associated with the sensor (sensor position) in which the sensor signal changes among the assist items stored in the operation assist management DB 3003. It is determined that there is a setting operation according to. When the setting operation corresponding to the change of the sensor signal exists (YES in step S176), the determination unit 34 shifts the process to step S177. On the other hand, when the setting operation corresponding to the change of the sensor signal does not exist (NO in step S176), the determination unit 34 terminates the user use function estimation process assuming that the user use function cannot be estimated.

Then, the user use function estimation unit 37 of the operation unit 30 estimates the user use function, which is a desired function that the user wants to use, based on the estimation result in step S173 or the processing result in steps S174 to S176. Step S177). Specifically, the user use function estimation unit 37 refers to the assist items and various conditions stored in the operation assist management DB 3003, and corresponds to the operation history of the user operation and the type of device information acquired in step S171. Guess user-used functions by searching for assist items. In the case of the example shown in FIGS. 5 (A) and 5 (B), the screen transition information included in the operation history is a specific screen transition (screen A → screen) associated with the assist item “scan”. Since it corresponds to B → screen A), the user use function estimation unit 37 estimates that the scan function is a user use function.

Thereby, the image processing device 10 can estimate the desired function that the user wants to use in real time based on the operation history of the user operation and the sensor signal from the peripheral device provided in the image processing device 10.

Returning to FIG. 7, the description of the operation assist process in the image processing device 10 will be continued. The determination unit 34 of the operation unit 30 determines whether or not the function used by the user can be estimated according to the result of the process shown in FIG. 10 (step S18). When the function used by the user can be estimated (YES in step S18), the determination unit 34 shifts the process to step S19. The operation assist determination unit 38 of the operation unit 30 determines the display form according to the user use function estimated by the user use function estimation unit 37 (step S19). Then, the display control unit 33 of the operation unit 30 causes the operation panel 305 to display the operation assist screen 700, which is the operation screen of the display form determined by the operation assist determination unit 38 (step S20).

FIG. 11A is a diagram showing an example of an operation assist screen displayed on the image processing device according to the embodiment. The operation assist screen 700 shown in FIG. 11A is an example in which the scanning function is estimated as the user-used function by the user-used function estimating unit 37. In addition to the selection screen 600 shown in FIG. 8, the operation assist screen 700 includes an enlargement button 750 in which a function menu showing a scan function is enlarged and displayed. In the case of the normal selection screen 600 as shown in FIG. 8, it is necessary for the user to voluntarily select a function to be used from a plurality of buttons indicating different function menus. On the other hand, on the operation assist screen 700, for example, when the user is uncertain about the operation, the menu estimated as the function that the user wants to use is enlarged and displayed by using the display form determined by the operation assist process. This makes it possible to assist the user's operation. Although an example of a display form in which the enlargement button 750 is displayed on the operation assist screen 700 shown in FIG. 11A has been described, the image processing device 10 is a button corresponding to the inferred user-use function. The color of the button may be changed or blinked to assist the operation of the button in a display form that can be distinguished by the user. The operation assist screen 700 is an example of the operation screen.

On the other hand, when the determination unit 34 of the operation unit 30 cannot estimate the function used by the user (NO in step S18), the determination unit 34 shifts the process to step S20. Then, the display control unit 33 of the operation unit 30 displays the recommendation screen 800 (step S20). FIG. 11B is a diagram showing an example of a recommendation screen displayed on the image processing device according to the embodiment. The recommendation screen 800 shown in FIG. 11 (B) recommends a function that can be roughly predicted in the process of guessing when the user use function cannot be guessed by the user use function guessing unit 37 (recommended function). It is a display screen for presenting to the user as. On the recommendation screen 800, in addition to the selection screen 600 shown in FIG. 8, a recommendation area 850 is additionally displayed. In the recommendation area 850, "Is the function you want to use" FAX "? , Etc., an "OK" button 851 that is pressed when responding to the recommendation message, and a "close" button 853 that is pressed when the recommendation area 850 is closed without responding to the recommendation message. When the user selects the "OK" button 851 included in the recommendation area 850, the image processing device 10 executes operation assist for the function indicated in the recommendation message. The display form of the recommendation screen 800 is not limited to this, and the recommendation message may be displayed by using a banner, push notification, or the like. The recommendation screen 800 is an example of an operation screen.

● Effects of the Embodiment As described above, the image processing device 10 is based on the operation history such as the operation interval of the user operation, the number of operations, or the information of the screen transition according to the user operation, and the peripheral devices included in the image processing device 10. The content of the operation assist to be provided to the user is determined by using the sensor signal of. As a result, the image processing device 10 can provide operation assist according to the real-time operation status of the user.

● Modification example of the embodiment ●
● Modification example 1
Next, the image processing apparatus according to the modified example of the embodiment will be described with reference to FIGS. 12 to 15. In each modification, the same configuration and the same function as those in the above embodiment are designated by the same reference numerals, and the description thereof will be omitted. First, the processing of the image processing apparatus according to the first modification of the embodiment will be described with reference to FIG. The process executed by the image processing device 10 according to the first modification is a process of setting the operation assist level according to the operation proficiency level for each user.

FIG. 12 is a flowchart showing an example of operation assist processing in the image processing apparatus according to the first modification of the embodiment. Since the processing of steps S31 to S34 is the same as the processing of steps S11 to S14 shown in FIG. 7, the description thereof will be omitted.

The determination unit 41 of the operation unit 30 determines the proficiency level of the user's operation based on the operation history acquired in step S34 (step S35). Specifically, the determination unit 41 refers to the screen transition information included in the operation history, and determines whether or not any screen transition has been performed within a predetermined time. Then, when the screen transition is performed within the predetermined time, the determination unit 41 determines that the user is accustomed to the operation, and determines that the operation proficiency of the user during the operation is high. On the other hand, if the screen transition is not performed within the predetermined time, the determination unit 41 determines that the user is unfamiliar with the operation, and determines that the operation proficiency of the user during the operation is low.

Then, when the determination unit 41 determines that the operation proficiency of the user during the operation is high (YES in step S36), the determination unit 41 shifts the process to step S38. Since the processing after step S38 is the same as the processing after step S17 shown in FIG. 7, the description thereof will be omitted.

On the other hand, when the determination unit 41 determines that the operation proficiency of the user during the operation is low (NO in step S36), the determination unit 41 shifts the process to step S37. Then, the operation unit 30 stops the operation assist and ends the operation assist process (step S37). When the operation assist function is stopped in step S37, the image processing device 10 assists the operation when the human sensor 533 or the like detects that the user who is operating is no longer in front of the image processing device 10. The configuration may be such that the function is restarted.

As a result, the image processing device 10 according to the first modification is unnecessary by providing detailed operation assist to a user having a low operation proficiency level and not performing an operation assist to a user having a high operation proficiency level. Operation assist can be suppressed and usability can be improved.

● Modification example 2
Next, the processing of the image processing apparatus according to the second modification of the embodiment will be described with reference to FIG. The process executed by the image processing device 10 according to the second modification detects the change immediately before the sensor signal included in the device information acquired by the device information acquisition unit 36 of the operation unit 30 and determines the operation assist. It is a process.

FIG. 13 is a flowchart showing an example of user use function estimation processing in the image processing apparatus according to the second modification of the embodiment. First, the device information acquisition unit 36 of the operation unit 30 acquires device information of peripheral devices arranged in the image processing device 10 (step S271). Specifically, the device information acquisition unit 53 of the main body unit 50 acquires device information including sensor signals from sensors of each peripheral device via the device I / F 511. Further, the input / output unit 51 of the main body unit 50 transmits the acquired device information to the operation unit 30. Then, the device information acquisition unit 36 of the operation unit 30 acquires the device information received by the input / output unit 31.

Next, the determination unit 34 determines whether or not there is a change in the sensor signal included in the device information acquired immediately before among the device information acquired in step S171 (step S272). Here, the device information acquired immediately before is the device information most recently acquired by the device information acquisition unit 53.

If there is a change in the sensor signal immediately before (YES in step S272), the determination unit 34 shifts the process to step S273. On the other hand, if there is no change in the sensor signal immediately before (NO in step S273), the determination unit 34 repeats the process from step S271.

Next, the determination unit 34 determines whether or not there is an assist item corresponding to the change in the immediately preceding sensor signal by searching the operation assist management DB 3003 (see FIG. 5B) (step S273). Specifically, when the determination unit 34 has an assist item associated with a sensor (sensor position) having a change in the sensor signal among the assist items stored in the operation assist management DB 3003, the determination unit 34 immediately before the sensor signal. It is judged that there is an assist item according to the change of. When the assist item corresponding to the change of the immediately preceding sensor signal exists (YES in step S273), the determination unit 34 shifts the process to step S274. Since the processing after step S274 is the same as the processing after step S172 shown in FIG. 10, the description thereof will be omitted. On the other hand, when the assist item corresponding to the change of the immediately preceding sensor signal does not exist (NO in step S273), the determination unit 34 repeats the process from step S271.

As a result, the image processing device 10 according to the second modification can improve the estimation accuracy of the user-used function by preferentially using the change in the sensor signal immediately before for the determination of the operation assist. Further, the image processing device 10 according to the second modification uses, for example, only the sensor signal that has changed immediately before for the determination of the operation assist, thereby suppressing the communication amount of the sensor signal and the amount of information used for the operation assist processing. Therefore, the processing load of the image processing device 10 can be reduced.

● Modification example 3
Next, the processing of the image processing apparatus according to the third modification of the embodiment will be described with reference to FIG. The process executed by the image processing device 10 according to the third modification is a process of setting the guess level of the user use function guessing process based on the user authority using the user authentication result.

FIG. 14 is a flowchart showing an example of operation assist processing in the image processing apparatus according to the third modification of the embodiment. First, the reception unit 32 of the operation unit 30 accepts the input of user information on the user's operation panel (step S51). Here, the user information includes, for example, a user ID and a password. Next, the authentication unit 42 of the operation unit 30 authenticates the user using the user information received in step S51 (step S52). Specifically, the storage / reading unit 49 searches the authority information management DB 3005 (see FIG. 6) for the user ID / password set corresponding to the user ID / password set received in step S51. If there is a corresponding pair, the authentication unit 42 determines that the requesting user is a legitimate user. If there is no corresponding pair, the authentication unit 42 determines that the requesting user is an unjust (illegal) user. If it is not valid, the display control unit 33 notifies (displays) that it is not valid to the operation panel 305, but the case where it is valid will be described below.

Next, the authentication unit 42 sets the user authority when the authenticated user uses the operation assist function (step S53). Specifically, in the storage / reading unit 49, the authentication unit 42 reads out the authority information associated with the user ID and password of the authenticated user in the authority information management DB 3005. Then, the authentication unit 42 sets the user authority shown in the read authority information.

Then, the image processing device 10 executes the operation assist process according to the user authority set by the authentication unit 42 (step S54). In this case, the image processing device 10 has the assist items stored in the operation assist management DB 3003 (see FIG. 5B) within the range of the assist items associated with the user authority set in step S53. Execute operation assist processing. The details of the operation assist process in step S54 are the same as those shown in FIG.

As a result, the image processing device 10 according to the third modification of the embodiment can limit the operation assist functions that can be provided by the user authority, so that the processing load on the operation assist process can be reduced.

● Modification example 4
Next, the processing of the image processing apparatus according to the modified example 4 of the embodiment will be described with reference to FIG. The process executed by the image processing device 10 according to the fourth modification is a process for assisting the resolution of the abnormality of the image processing device 10 by utilizing the device information transmitted from the peripheral device.

FIG. 15 is a flowchart showing an example of user use function estimation processing in the image processing apparatus according to the fourth modification of the embodiment. The processing of steps S371 to S373 is the same as the processing of steps S271 to S273 shown in FIG. In particular, in the process of step S371, the device information acquisition unit 36 has not only a user-operable ADF531 or a paper feed tray 535, but also a peripheral sensor capable of detecting an abnormal state of the image processing device 10. Get the sensor signal from the device. Further, in step S373, the user use function estimation unit 37 determines whether or not the assist item exists by using the operation assist management table in which the abnormality resolution assist item is stored. Then, the user use function estimation unit 37 of the operation unit 30 estimates the content of the abnormality resolution assist when there is an assist item corresponding to the change in the sensor signal (step S374).

As a result, the image processing device 10 according to the modified example 4 of the embodiment determines an abnormal state such as a hardware failure or malfunction of the image processing device 10 based on the acquired device information, thereby asking the operation panel 305. Not only the operation assist of the user operation but also the abnormality resolution assist of the image processing device 10 can be performed. Conventionally, there is known a technique of displaying a guide, a guide, or the like for solving an abnormality in an image processing device on an operation panel. However, the image processing device 10 according to the modified example 4 is, for example, , When the user repeats the same operation many times, or when the abnormality is not resolved even if the operation is performed according to the displayed operation assist screen 700, various device information transmitted from the peripheral device is utilized. Therefore, the content of the abnormality resolution assist can be changed.

● Summary ●
As described above, the device according to the embodiment of the present invention is an image processing device 10 provided with an operation panel 305 (an example of an operation means) that accepts a predetermined input operation. The image processing device 10 includes a user operation detection unit 35 (an example of a user operation detecting means) that detects a user operation received by the operation panel 305, and a peripheral device (an example of a user operation detecting means) arranged at a specific position of the image processing device 10. The device information acquisition unit 36 (an example of the device information acquisition means) that acquires the device information including the sensor signal output from the sensor), the history information indicating the history of the detected user operation, and the acquired device information. Based on this, it includes a display control unit 33 (an example of display control means) for displaying an operation screen (for example, an operation assist screen 700) on the operation panel 305 on which a process to be executed by the image processing device 10 can be selected. As a result, the image processing device 10 can provide operation assist according to the real-time operation status of the user.

Further, the image processing device 10 (an example of a device) according to an embodiment of the present invention is a user-use function guessing unit that guesses a desired process based on history information indicating a history of user operations and acquired device information. 37 (an example of the estimation means) and an operation assist determination unit 38 (an example of the determination means) for determining the display form of the operation screen (for example, the operation assist screen 700) for performing the operation assist according to the estimated processing. , And display the operation screen of the determined display form on the operation panel 305 (an example of the operation means). Further, the image processing device 10 changes the display form of the operation screen displayed on the operation panel 305 (for example, the selection screen 600) to the determined display form (for example, the operation assist screen 700). As a result, the image processing device 10 can improve the accuracy of the operation assist required by the user by changing the display form of the operation screen displayed on the operation panel 305 according to the real-time operation status of the user.

Further, the image processing device 10 (an example of an apparatus) according to an embodiment of the present invention includes a determination unit 41 (an example of a determination means) for determining the presence or absence of a screen transition within a predetermined time. Then, when the determination unit 41 determines that there is a screen transition, the operation assist determination unit 38 (an example of the determination means) ends the operation assist. As a result, the image processing device 10 provides detailed operation assist to a user who is unfamiliar with screen operation and has low operational proficiency, and does not provide operation assist to a user who is accustomed to screen operation and has high operational proficiency. As a result, unnecessary operation assist can be suppressed and usability can be improved.

Further, the image processing device 10 (an example of an apparatus) according to an embodiment of the present invention includes an authentication unit 42 (an example of an authentication means) that performs an authentication process of a user who has performed a predetermined input operation. Then, the user use function guessing unit 37 (an example of the guessing means) guesses a desired process according to the authority of the authenticated user. As a result, the image processing device 10 can limit the operation assist functions that can be provided by the user authority, so that the processing load on the operation assist process can be reduced.

Further, in the image processing device 10 (an example of the device) according to the embodiment of the present invention, the user use function guessing unit 37 (an example of the guessing means) has an abnormality of the image processing device 10 based on the acquired device information. The operation assist determination unit 38 (an example of the determination means) estimates the state, and determines the display form of the operation screen (for example, the operation assist screen 700) for performing the operation assist according to the estimated abnormal state. As a result, the image processing device 10 determines not only the operation assist of the user operation on the operation panel 305 but also the operation assist of the user operation on the operation panel 305 by determining an abnormal state such as a hardware failure or malfunction of the image processing device 10 based on the acquired device information. , It is possible to assist the abnormality elimination of the image processing device 10.

● Supplement ●
Each function of the embodiment described above can be realized by one or more processing circuits. Here, the "processing circuit" in the present specification is a processor programmed to execute each function by software such as a processor implemented by an electronic circuit, or a processor designed to execute each function described above. It shall include devices such as ASICs (Application Specific Integrated Circuits), DSPs (digital signal processors), FPGAs (field programmable gate arrays), and conventional circuit modules.

Further, the various tables of the embodiment described above may be generated by the learning effect of machine learning, and by classifying the data of each related item by machine learning, the table is not used. You may. Here, machine learning is a technology for making a computer acquire learning ability like a human being, and the computer autonomously generates an algorithm necessary for judgment such as data identification from learning data taken in advance. However, it refers to a technology that applies this to new data to make predictions. The learning method for machine learning may be any of supervised learning, unsupervised learning, semi-supervised learning, enhanced learning, and deep learning, and may be a learning method that combines these learning methods. Machine learning It doesn't matter how you learn for.

Although the devices, display control methods, and programs according to one embodiment of the present invention have been described so far, the present invention is not limited to the above-described embodiments, and additions, changes, deletions, etc. of other embodiments, etc. , It can be changed within the range that can be conceived by those skilled in the art, and is included in the scope of the present invention as long as the action / effect of the present invention is exhibited in any of the embodiments.

10 Image processing device (example of equipment)
30 Operation unit 33 Display control unit (an example of display control means)
35 User operation detection unit (an example of user operation detection means)
36 Equipment information acquisition unit (an example of equipment information acquisition means)
37 User-use function guessing unit (an example of guessing means)
38 Operation assist determination unit (example of determination means)
50 Main body 305 Operation panel (example of operation means)
531 ADF (an example of a sensor)
533 Motion sensor (example of sensor)
535 Paper tray (example of sensor)
600 selection screen (example of operation screen)
700 Operation assist screen (example of operation screen)
800 recommendation screen (example of operation screen)

JP-A-2018-158529

Claims (13)

A device equipped with an operating means for accepting a predetermined input operation.
A user operation detecting means for detecting a user operation received for the operating means, and a user operation detecting means.
A device information acquisition means for acquiring device information including a sensor signal output from a sensor arranged at a specific position of the device, and a device information acquisition means.
A display control means for causing the operation means to display an operation screen on which a process to be executed by the device can be selected based on the history information indicating the detected history of the user operation and the acquired device information.
Equipment equipped with. A guessing means for guessing a desired process based on the history information and the device information, and
It is provided with a determination means for determining the display form of the operation screen for performing operation assist according to the inferred processing.
The device according to claim 1, wherein the display control means causes the operation means to display an operation screen having a determined display form. The device according to claim 2, wherein the display control means changes the display form of the operation screen displayed on the operation means to the display form determined by the determination means. The history information includes at least one of the detected operation interval of the user operation and the number of operations within a predetermined time.
The device according to claim 2 or 3, wherein the guessing means guesses a desired process based on at least one of the operation interval and the number of operations. The history information includes screen transition information indicating a specific screen transition according to the detected user operation.
The device according to any one of claims 2 to 4, wherein the guessing means guesses a desired process based on the screen transition information. The device according to any one of claims 2 to 5, wherein the guessing means guesses a desired process based on an arrangement position of the sensor that has transmitted a sensor signal included in the acquired device information. The device according to claim 6, wherein the guessing means guesses a desired process based on a change in the sensor signal included in the acquired device information. The device according to any one of claims 2 to 7, and further
Equipped with a determination means for determining the presence or absence of screen transition within a predetermined time
The determination means is a device that terminates the operation assist when it is determined that there is a screen transition. The device according to any one of claims 2 to 8, further
An authentication means for performing an authentication process of the user who has performed the input operation is provided.
The guessing means is a device that guesses a desired process according to the authority of an authenticated user. The guessing means guesses the abnormal state of the device based on the acquired device information, and estimates the abnormal state of the device.
The device according to any one of claims 2 to 9, wherein the determination means determines a display form of the operation screen for performing an operation assist according to the estimated abnormal state. The device according to any one of claims 1 to 10, wherein the device is an image processing device capable of performing image processing. A display control method executed by a device having an operation means for accepting a predetermined input operation.
A user operation detection step for detecting a user operation received for the operation means, and
A device information acquisition step for acquiring device information including a sensor signal from a sensor arranged at a specific position of the device, and
A display control step for causing the operation means to display an operation screen on which a process to be executed by the device can be selected based on the detected history information indicating the history of the user operation and the acquired device information.
Display control method to execute. For devices equipped with an operating means that accepts predetermined input operations
A user operation detection step for detecting a user operation received for the operation means, and
A device information acquisition step for acquiring device information including a sensor signal from a sensor arranged at a specific position of the device, and
A display control step for causing the operation means to display an operation screen on which a process to be executed by the device can be selected based on the detected history information indicating the history of the user operation and the acquired device information.
A program that executes.

Images