JP6911317B2 - Programs and information processing systems executed on information processing terminals - Google Patents

Description

The present invention relates to a program of an information processing terminal that customizes a screen displayed by an image forming apparatus.

For example, Patent Document 1 discloses a program of an information processing terminal that customizes a screen displayed on a printer display. More specifically, the program can add the UI component to the screen by dragging and dropping the UI component prepared in advance.

Japanese Unexamined Patent Publication No. 2011-154474

However, in the program described in Patent Document 1, the UI component to be displayed on the screen of the printer can be selected only from the UI components prepared in advance. Therefore, there is a problem that the degree of freedom of screen customization is low.

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide an information processing terminal program and an information processing system that can flexibly customize the screen displayed by the image forming apparatus.

Various disclosures are made herein. The program, which is one of the disclosure examples, can be executed by an information processing terminal including a memory, a terminal display, and an input interface. The memory stores screen instruction data indicating an icon screen to be displayed on the device display of the image forming apparatus. The screen instruction data indicates an operation icon to be displayed on the icon screen and an operation to be executed by the image forming apparatus when the operation icon is specified. The program has a first display process for displaying a customized screen in which the arrangement of the operation icons is common to the icon screen in a part area of the terminal display according to the screen instruction data, and an external area different from the customized screen. The first reception process that accepts the drag operation for moving the target icon displayed by the program on the customization screen through the input interface, and the position on the customization screen where the target icon was moved in the first reception process. , A customization process for customizing the screen instruction data and the screen instruction data customized by the customization process are output to the image forming apparatus so that the symbol indicated by the target image data corresponding to the target icon is displayed. The information processing terminal is made to execute the output processing to be performed.

According to the above configuration, the icon screen can be customized with the target image data corresponding to the target icon by dragging the target icon displayed on the terminal display by the external program onto the customization screen. As a result, the number of customization options is greatly increased, and the icon screen can be customized more flexibly.

FIG. 1 is a block diagram of an MFP 10 and an information processing terminal 50 constituting the information processing system 100 according to the present embodiment. 2A and 2B are examples of screen instruction data stored in the memory 62, where FIG. 2A shows standard screen data and FIG. 2B shows custom screen data corresponding to FIG. 10A. FIG. 3 is an example of screen instruction data stored in the memory 62, in which (A) is the custom screen data corresponding to FIGS. 10 (B) and 11 (B), and FIG. 11 (B) is FIG. 11 (A). ) Corresponds to the custom screen data. FIG. 4 is a flowchart of the main process. FIG. 5 is a flowchart of the icon editing process. FIG. 6 is a flowchart of the icon registration process. FIG. 7 is a diagram showing a state in which the list screen 120 and the customization screen 130 are simultaneously displayed on the display 53. 8A and 8B are display examples of the customization screen, in which FIG. 8A shows a state in which the background symbol of FIG. 7 is changed, and FIG. 8B shows a state in which the symbol of the operation icon 134 in FIG. 7 is changed. 9A and 9B are display examples of the display 53, in which FIG. 9A shows an operation selection screen and FIG. 9B shows a parameter selection screen. 10A and 10B are display examples of the customization screen. FIG. 10A shows a state in which the icon 122 is dragged onto the reserved area image 135 in FIG. 7, and FIG. 10B shows an icon 139 on the insertion area image 136 in FIG. Indicates the dragged state. 11A and 11B are display examples of the customization screen. FIG. 11A shows a state in which the operation icon 139 of FIG. 10A is dragged onto the operation icon 133, and FIG. 11B shows an operation icon 139 of FIG. 10A. Is dragged onto the insertion area image 136. 12A and 12B are display examples of icon screens, in which FIG. 12A shows an icon screen corresponding to FIG. 2A, and FIG. 12B shows an icon screen corresponding to FIG. 3A.

Hereinafter, embodiments of the present invention will be described with reference to the drawings as appropriate. It goes without saying that the embodiments described below are merely examples of the present invention, and the embodiments of the present invention can be appropriately changed without changing the gist of the present invention. For example, the execution order of each process described later can be appropriately changed without changing the gist of the present invention.

FIG. 1 is a schematic view of an information processing system 100 according to the present embodiment. The information processing system 100 shown in FIG. 1 includes an MFP (abbreviation of Multi-Function Peripheral) 10 and an information processing terminal 50. The MFP 10 and the information processing terminal 50 are configured to be able to communicate with each other through the communication network 101. Specific examples of the communication network 101 are not particularly limited, but may be, for example, the Internet, a wired LAN, a wireless LAN, or a combination thereof. Alternatively, the MFP 10 and the information processing terminal 50 may be connected by a USB cable or the like.

As shown in FIG. 1, the MFP 10 includes a printer 11, a scanner 12, a fax unit 13, a display 23, an input I / F 24, a communication I / F 25, a CPU 31, a memory 32, and a communication bus 33. Mainly equipped with. The components constituting the MFP 10 are connected to each other through the communication bus 33. The MFP 10 is an example of an image forming apparatus.

The printer 11 executes a printing operation of recording an image represented by image data on a sheet. As the recording technique of the printer 11, known techniques such as an inkjet technique and an electrophotographic technique can be adopted. The scanner 12 executes a scanning operation of reading an image recorded on a document and generating scan data. The FAX unit 13 executes a FAX operation of transmitting image data by fax to an external device and receiving the image data by fax from the external device.

The display 23 includes a liquid crystal display, an organic EL display, and the like, and includes a display screen for displaying various information. The display 23 is an example of a device display.

The input I / F 24 is a user interface that accepts input operations by the user. Specifically, the input I / F 24 has a button, and outputs various operation signals associated with the pressed button to the CPU 31. Further, the input I / F 24 may have a film-like touch sensor superimposed on the display screen of the display 23. The operation of designating an object displayed on the display screen of the display 23 and the operation of inputting a character string or a number string are examples of user operations. The “object” is, for example, a character string, an icon, a button, a link, a radio button, a check box, a pull-down menu, or the like displayed on the display 23. The input I / F 24 is an example of the second input interface.

The input I / F24 realized as a touch sensor outputs position information indicating a position on the display screen touched by the user. The term "touch" in the present specification includes all operations of bringing the input medium into contact with the display screen. Further, even if the input medium does not touch the display screen, the "hover" or "floating touch" that brings the input medium close to the position where the distance from the display screen is very small is used as the above-mentioned concept of "touch". May be included. Further, the input medium may be a user's finger, a touch pen, or the like. The user operation of tapping the position of the object displayed on the display 23 is an example of the user operation of designating the object.

The communication I / F 25 is an example of a second communication interface capable of communicating with an external device through the communication network 101. That is, the MFP 10 transmits various data to the external device through the communication I / F 25, and receives various data from the external device through the communication I / F 25. Hereinafter, the operation of the communication I / F 25 transmitting data to the external device is referred to as an upload operation, and the operation of the communication I / F 25 receiving data from the external device is referred to as a download operation. The specific communication procedure of the communication I / F25 is not particularly limited, but for example, Wi-Fi (registered trademark of Wi-Fi Alliance) can be adopted. When the MFP 10 and the information processing terminal 50 are connected by a USB cable, the communication I / F 25 may be a USB interface to which the USB cable can be attached and detached.

Further, the MFP 10 can execute a compound operation in which a plurality of operations are combined. In the copy operation, which is an example of the combined operation, the printer 11 records the image indicated by the scan data generated by the scanner 12 in the scanning operation on the sheet in the printing operation. In the scan PC operation, which is another example of the combined operation, the communication I / F 25 transmits the scan data generated by the scanner 12 in the scan operation to the information processing terminal 50 in the upload operation. In the scan mail operation, which is still another example of the combined operation, the communication I / F 25 transmits the scan data generated by the scanner 12 in the scan operation to the SMTP server (not shown) in the upload operation.

The operations that the MFP 10 can perform are identified by the operation ID. For example, the print operation executed by the printer 11 is identified by the operation ID "Print", the scan operation executed by the scanner 12 is identified by the operation ID "Scan", and the FAX operation executed by the FAX unit 13 is the operation ID "FAX". The upload operation and download operation executed by the communication I / F25 are identified by the operation IDs "Upload" and "Download", the copy operation is identified by the operation ID "Copy", and the scan PC operation is identified by the operation ID "Scan". It is identified by "to PC", and the scan mail operation is identified by the operation ID "Scan to Email".

The MFP 10 executes the above-described operation according to the operation parameters. The operation parameter is information indicating the execution condition of the operation. The operation parameter includes a setting value for each of one or more items (for example, resolution, format, size, color, destination). The items of operation parameters differ depending on the corresponding operation. The operation parameters of the copy operation can include, for example, the setting values of the items "size" and "color". The operation parameters of the scan PC operation can include, for example, the setting values of the items "resolution" and "format". The operation parameters of the scan mail operation can include, for example, the setting values of the items "format" and "destination".

The setting value of the item "resolution" indicates the scanning resolution of the original. The setting value of the item "Format" indicates the file format of the scan data. The item "size" indicates the size of the sheet on which the image is recorded. The item "color" indicates the color used in the printing operation. The item "destination" indicates the e-mail address to which the scan data is sent. However, the operation parameters of each operation may not include all of the above-mentioned items, or may further include items other than the above-mentioned items.

The CPU 31 controls the overall operation of the MFP 10. The CPU 31 acquires and executes various programs described later from the memory 32 based on various information output from the input I / F 24, various information acquired from the external device through the communication I / F 25, and the like. The CPU 31 and the memory 32 form an example of the second controller.

The memory 32 stores the OS 34 and the device program 35. The device program 35 may be a single program or a collection of a plurality of programs. Further, the memory 32 stores data or information necessary for executing the device program 35. The memory 32 is composed of, for example, a RAM, a ROM, an EEPROM, an HDD, a portable storage medium such as a USB memory attached to and detached from the MFP 10, a buffer included in the CPU 31, or a combination thereof.

The memory 32 may be a computer-readable storage medium. A computer-readable storage medium is a non-transitory medium. In addition to the above examples, the non-transitory medium also includes a recording medium such as a CD-ROM or a DVD-ROM. The non-transitory medium is also a tangible medium. On the other hand, an electric signal that carries a program downloaded from a server on the Internet is a computer-readable signal medium, which is a kind of computer-readable medium, but is a storage that can be read by a non-transitory computer. Not included in the medium. The same applies to the memory 62 of the information processing terminal 50, which will be described later.

The memory 32 stores, for example, screen instruction data indicating the icon screen shown in FIG. The screen instruction data is stored in the memory 32 by the manufacturer, for example, before the shipment of the MFP 10. The screen instruction data stored in the memory 32 at the time of shipment may be referred to as "standard screen data". The format of the screen instruction data is not particularly limited, but may be, for example, XML format information.

The screen instruction data includes, for example, one or more icon records indicating an operation icon displayed on the icon screen and an operation to be executed by the MFP 10 when the operation icon is specified through the input I / F 24. The icon record includes, for example, an icon ID, icon image data, text data, an operation ID, and an operation parameter.

The icon ID is identification information that identifies an operation icon displayed on the icon screen. The icon ID is also used as position information indicating the position of the operation icon on the icon screen. For example, the icon ID "001" indicates the position at the left end of the icon screen, the icon ID "002" indicates the position at the center of the icon screen, and the icon ID "003" indicates the position at the right end of the icon screen. In other words, the icon ID indicates the arrangement order of the plurality of operation icons arranged on the icon screen. However, the information indicating the position and arrangement order of the operation icons may be included in the icon record separately from the icon ID.

In the icon screen according to the present embodiment, as shown in FIG. 12, for example, a plurality of operation icons can be arranged in the left-right direction. The left side is the front of the arrangement order, and the right side is the back of the arrangement order. However, the number and arrangement order of the operation icons displayed on the icon screen is not limited to the example of FIG. Further, by making the icon screen scrollable, more operation icons may be displayed.

The icon image data is image data indicating the design of the operation icon. The icon image data includes a non-designated image data and a designated image data. The non-designated image data is an example of the first image data showing the design of the operation icon before being designated through the input I / F24. The designated image data is an example of the second image data showing the design of the operation icon after the input I / F24 is designated. It is desirable that the non-designated image data and the designated image data can be recognized by the user of the MFP 10 as representing the same operation at first glance, and show different symbols from each other.

The text data indicates a character string described on the icon screen in association with the operation icon. More specifically, the text data indicates a character string that the user of the MFP 10 can recognize at a glance the operation to be executed when the corresponding operation icon is specified. The character string represented by the text data is described on the icon screen at a position adjacent to the corresponding operation icon (for example, adjacent to the lower part of the operation icon). The action ID identifies the action to be performed when the corresponding action icon is specified. The operation parameter indicates the execution condition of the operation identified by the corresponding operation ID.

The icon record including the icon ID "003" does not include the icon image data, the text data, the operation ID, and the operation parameter. That is, as shown in FIG. 12 (A), the operation icon is not displayed at the position indicated by the icon ID "003" on the icon screen shown by the screen instruction data shown in FIG. 2 (A). This icon record indicates that a new operation icon can be added to the position on the icon screen indicated by the icon ID "003". The position on the icon screen indicated by the icon ID “003” in FIG. 2A is an example of the reserved area.

The record including the icon ID "999" includes background image data indicating a background pattern of the icon screen. This record contains only background image data instead of icon image data, text data, operation ID, and operation parameters. That is, the icon screen is a screen in which a symbol indicating an operation icon is combined on a background symbol that substantially matches the size of the display area of the display 23 (hereinafter, referred to as “display area size”).

As shown in FIG. 1, the information processing terminal 50 mainly includes a display 53, an input I / F 54, a communication I / F 55, a CPU 61, a memory 62, and a communication bus 63. The display 53, the input I / F 54, the communication I / F 55, the CPU 61, the memory 62, and the communication bus 63 included in the information processing terminal 50 are the display 23, the input I / F 24, the communication I / F 25, the CPU 31, and the communication bus 63 included in the MFP 10. Since the configuration is the same as that of the memory 32 and the communication bus 33, the description thereof will be omitted. The information processing terminal 50 is, for example, a mobile phone, a smartphone, a tablet terminal, a PC, or the like. The display 53 is an example of a terminal display, the input I / F 54 is an example of the first input interface, the communication I / F 55 is an example of the first communication interface, and the CPU 61 and the memory 62 are examples of the first controller. ..

The memory 62 stores the OS 64, the terminal program 65, and the list program 66. Although not shown, the memory 62 can store the designated device ID. The designated device ID identifies the MFP10 designated by the user (hereinafter, referred to as "designated device") among the plurality of MFPs 10 that can communicate through the communication I / F55. Further, the input I / F 54 may be composed of, for example, a keyboard and a mouse.

The OS 64 can simultaneously display a plurality of screens (that is, windows) generated by each of the plurality of programs installed in the information processing terminal 50 on the display 53. That is, the OS 64 has a so-called multi-window function. The screen displayed on the display 53 may be one displayed by the OS 64 itself (for example, a desktop screen).

Further, the OS 64 displays a pointer image (for example, an image representing an arrow) 110 on the display 53 as shown in FIG. 7, and moves the position of the pointer image 110 on the display 53 following the movement of the mouse. The terminal program 65 is notified of the position of the pointer image 110. Further, the OS 64 indicates that, for example, a plurality of buttons mounted on a keyboard and two buttons mounted on a mouse (hereinafter, referred to as "left button" and "right button") are pressed by a terminal program. Notify 65.

The terminal program 65 is a program for customizing the icon screen of the MFP 10. More specifically, the terminal program 65 edits the screen instruction data according to the user operation through the input I / F 54, and outputs the edited screen data to the MFP 10. The list program 66 is an example of an external program (for example, an explorer) that displays a list screen 120 including a plurality of icons 121 to 125 in a part area of the display 53, as shown in FIG. 7, for example.

The icons 121 to 125 are associated with, for example, one of a plurality of image data stored in the memory 62. The image data associated with the icons 121 to 125 indicates an arbitrary image such as a photograph taken by a camera or an image generated by using a paint program (not shown). That is, the plurality of image data stored in the memory 62 may indicate images having different sizes (number of pixels in the vertical direction and number of pixels in the horizontal direction). Further, the image data associated with the icons 121 to 125 may be stored in, for example, a storage server (not shown) on the Internet.

The operation of the information processing system 100 according to the present embodiment will be described with reference to FIGS. 4 to 6.

The flowchart of the present specification basically shows the processing of the CPUs 31 and 61 according to the instructions described in the program. That is, the processes such as "judgment", "extraction", "selection", "calculation", "determination", "specification", and "control" in the following description represent the processes of the CPUs 31 and 61. The processing by the CPU 61 also includes hardware control via the OS 64. Further, the "data" in the present specification is represented by a bit string that can be read by a computer. Then, data having the same substantial meaning and contents but different formats are treated as the same data. The same applies to "information" in the present specification.

[Main processing]
The terminal program 65 executes the main process shown in FIG. 4, for example, in response to being activated by a user's instruction through the input I / F 54. First, the terminal program 65 acquires the standard screen data of the designated device. Further, the terminal program 65 further acquires information indicating the display area size of the display 23 of the designated device and the icon size of the operation icon arranged on the icon screen (for example, the number of pixels in the vertical direction and the number of pixels in the horizontal direction). You may.

As an example, the terminal program 65 may read screen instruction data from the memory 62. As another example, the terminal program 65 may receive screen instruction data from the designated device through the communication I / F 55. As another example, the terminal program 65 may transmit model information indicating the model name of the designated device to a server (not shown) through the communication I / F 55. Then, the terminal program 65 may receive the screen instruction data stored in the server memory in association with the model information from the server through the communication I / F 55.

Next, the terminal program 65 causes the customization screen 130 to be displayed in a part of the area of the display 53, for example, as shown in FIG. 7 (S11). The customization screen 130 includes an icon display area 131 and a [send] icon 132. Then, the terminal program 65 accepts the user operation for the customization screen 130 through the input I / F 54 (S12 to S15). The process of S11 is an example of the first display process, and the processes of S12 to S15 are an example of the first reception process.

In the icon display area 131, the operation icons 133 and 134 are arranged at positions indicated by the acquired screen instruction data (hereinafter, referred to as “icon area”). That is, in the icon display area 131, the arrangement of the operation icons 133 and 134 is common to the icon screen shown in FIG. 12 (A). Further, the icon display area 131 may further include the reserved area image 135 and the insertion area images 136 and 137. The reserved area image 135 and the insertion area images 136 and 137 may not be included in the icon image indicated by the corresponding screen instruction data.

The reserved area image 135 is an image showing a reserved area indicated by an icon record including the icon ID “003”. The insertion area images 136 and 137 are images showing an insertion area in which a new operation icon can be inserted. The insertion area images 136 and 137 are arranged at positions adjacent to the front of the operation icons 133 and 134 in the arrangement order. The insertion area images 136 and 137 may be displayed only when the pointer image 110 is located on the insertion area, and may be hidden in other cases.

Further, the icon display area 131 includes a background area. The background area is an area in which the background pattern indicated by the background image data is displayed. In other words, the background area is an area of the icon display area 131 that is different from the operation icons 133 and 134, the reserved area image 135, and the insertion area image 136 and 137.

Next, the terminal program 65 executes the icon editing process shown in FIG. 5 in response to receiving the drag operation with the end position on the icon display area 131 through the input I / F 54 (S12: Yes) (S12: Yes). S16). Hereinafter, the details of the icon editing process, which is an example of the customization process, will be described with any position of the icons 121 to 125 included in the list screen 120 as the start position and the position on the icon display area 131 as the end position.

In the drag operation, the left mouse button is pressed at the position of the icon displayed on the display 53 (hereinafter, referred to as “start position”), and the pointer image 110 is moved while pressing the left button. This is an operation to release the pressing of the left button at an end position different from the start position. In other words, the drag operation is an operation of moving the pointer image 110 between the start position and the end position on the display 53 while pressing the left button. In other words, the drag operation is an operation of moving the icon displayed at the start position (hereinafter, referred to as “target icon”) to the end position.

Then, the terminal program 65 acquires from the OS 64 an event notification indicating that the pointer image 110 being dragged has been moved onto the customization screen 130 and an event notification indicating that the drag operation has been completed on the customization screen 130. can do. Further, the terminal program 65 can acquire the file path name of the image data stored in the memory 62 in association with the target icon by executing the API provided by the OS 64 after acquiring the above-mentioned event notification. ..

As an example, the terminal program 65 receives a drag operation with the position of the icon 125 as the start position and the background area of the customization screen 130 shown in FIG. 7 as the end position through the input I / F 54 (S21: Yes), background image data is generated using the image data associated with the icon 125 (hereinafter, referred to as “target image data”) (S22). The process of S22 is an example of the third generation process.

More specifically, the terminal program 65 secures a first memory area corresponding to the display area size in the memory 62. More specifically, the terminal program 65 secures in the memory 62 a first memory area having a size capable of expanding the pixel values of all the pixels constituting the image of the display area size. Next, the terminal program 65 sets default pixel values for all the pixels in the reserved first memory area. The default pixel value is a pixel value of a predetermined color (for example, white or black) as the background color of the icon screen to be displayed on the display 23.

Next, the terminal program 65 acquires the target image data from the memory 62 or a server (not shown). Next, the terminal program 65 sets the terminal program 65, for example, so that one of the horizontal length and the vertical length of the image indicated by the target image data matches the display area size and the other is smaller than the display area size. Enlarge or reduce while maintaining the aspect ratio of the image. Then, the terminal program 65 expands the pixel value of the enlarged or reduced image into the first memory area.

As a result, in the first memory area, for example, as shown in FIG. 8A, the “cloud” pattern shown in the target image data and the pixel having the default pixel value shown by hatching are adjacent to each other in the left-right direction. Background image data indicating the background pattern to be used is expanded. Next, the terminal program 65 replaces the background image data included in the icon record indicated by the icon ID “999” with the background image data expanded in the first memory area in S22 (S23). That is, the drag operation with the background area as the end position is an operation of replacing the background image data included in the screen instruction data.

Next, returning to FIG. 4, the terminal program 65 causes the customization screen 130 shown in FIG. 8A to be displayed on the display 53 instead of the customization screen 130 shown in FIG. 7 (S11). The customization screen 130 shown in FIG. 8A is different from FIG. 7 in that the background symbol shown by the background image data generated in S22 is displayed in the background area. That is, the drag operation with the background area as the end position is an operation of changing the background pattern of the customization screen 130.

As another example, the terminal program 65 accepts a drag operation with the position of the icon 124 as the start position and the position of the operation icon 134 of the customization screen 130 shown in FIG. 7 as the end position through the input I / F 54. Correspondingly (S21: No), the non-designated image data and the designated image data are generated using the target image data associated with the icon 124 (S24). The process of S24 is an example of the first generation process and the second generation process.

More specifically, the terminal program 65 secures a second memory area corresponding to the icon size in the memory 62. More specifically, the terminal program 65 secures a second memory area in the memory 62 having a size capable of expanding the pixel values of all the pixels constituting the icon-sized image. Next, the terminal program 65 sets the transmittance of all the pixels in the secured second memory area to 100%. In other words, the terminal program 65 sets transparent color pixel values (hereinafter, referred to as “transparent pixels”) in all the pixels of the secured second memory area 171.

Next, the terminal program 65 acquires the target image data from the memory 62 or a server (not shown). Next, the terminal program 65 vertically and horizontally the image so that one of the horizontal length and the vertical length of the image indicated by the target image data matches the specified size and the other is smaller than the specified size. Enlarge or reduce while maintaining the ratio. Then, the terminal program 65 expands the pixel value of the enlarged or reduced image into the second memory area. As a result, the non-designated image data is expanded in the second memory area.

Next, the terminal program 65 secures a third memory area having the same size as the second memory area in the memory 62. Next, the terminal program 65 copies the non-designated image data of the second memory area to the third memory area. Then, the terminal program 65 generates the designated time image data from the non-designated image data expanded in the third memory area. The designated image data indicates a symbol having the same contour line as the symbol shown in the non-designated image data but different in brightness, brightness, and saturation at least one.

More specifically, the terminal program 65 converts the pixel value of the non-designated image data expanded in the third memory area from the RGB color space to the Lab (abbreviation of Lab color space) color space. That is, the terminal program 65 converts the pixel value of the non-designated image data from a format represented by a plurality of color values to a format represented by dimensions L and complementary color dimensions a and b. Next, the terminal program 65 increases or decreases the value of the dimension L, which means lightness, by a predetermined method for each pixel converted into the Lab color space. Then, the terminal program 65 reversely converts the pixel values in which the dimension L is increased or decreased from the Lab color space to the RGB color space. As a result, the designated image data is expanded in the third memory area.

However, the method of generating the image data at the time of designation is not limited to the above-mentioned example. As another example, the terminal program 65 may convert the pixel value of the non-designated image data in the third memory area from the RGB color space to the YCbCr color space. Then, the terminal program 65 may increase or decrease the value of the luminance component Y by a predetermined method for each pixel converted into the YCbCr color space. Further, the terminal program 65 may inversely convert the pixel value obtained by increasing or decreasing the value of the luminance component Y from the YCbCr color space to the RGB color space. As a result, the designated image data is expanded in the third memory area.

Next, the terminal program 65 generated icon image data of the icon record including the icon ID “002” in S24 according to the position of the operation icon 134 at the end position of the drag operation (S25: Yes). Replace with icon image data (S26). On the other hand, the terminal program 65 does not change the icon ID, text data, operation ID, and operation parameter of the icon record. That is, the drag operation with the icon area as the end position is an operation of replacing only the icon image data among the icon records corresponding to the operation icons at the end position.

Next, returning to FIG. 4, the terminal program 65 causes the customization screen 130 shown in FIG. 8B to be displayed on the display 53 instead of the customization screen 130 shown in FIG. 7 (S11). The customization screen 130 shown in FIG. 8B is different from FIG. 7 in that the operation icon 134 symbol is changed. That is, the drag operation with the icon area as the end position is an operation of changing only the design of the operation icon without changing the operation associated with the operation icon at the end position.

As yet another example, the terminal program 65 accepts a drag operation with the position of the icon 122 as the start position and the position of the reserved area image 135 of the customization screen 130 shown in FIG. 7 as the end position through the input I / F 54. Accordingly (S21: No), the non-designated image data and the designated image data are generated by the same method as described above using the target image data associated with the icon 122 (S24). Next, the terminal program 65 skips the process of S28 and executes the icon registration process shown in FIG. 6 according to the end position of the drag operation being the reserved area (S25: No & S27: reservation) (S25: No & S27: reserved). S29).

First, the terminal program 65 causes the operation selection screen shown in FIG. 9A to be displayed on the display 53 (S31). The operation selection screen is added to a pull-down menu 141 that accepts one of a plurality of operation IDs "Print", "Scan", "Copy", "Scan to PC", and "Scan to Email", and an operation icon. It includes a text box 142 that accepts input of a character string, an [OK] icon 143, and a [Cancel] icon 144. Then, the terminal program 65 accepts the user operation for the operation selection screen through the input I / F 54 (S32). The process of S32 is an example of the second reception process.

Next, the terminal program 65 responds to, for example, receiving the designation of the operation ID "Scan to Email", the input of the character string "scan mail", and the designation of the [OK] icon 143 through the input I / F54. (S32: OK), the parameter selection screen shown in FIG. 9B is displayed on the display 53 (S33). Then, the terminal program 65 accepts the user operation for the parameter selection screen through the input I / F 54 (S34). The process of S34 is an example of the second reception process.

The parameter selection screen has a pull-down menu 145 that accepts one of a plurality of setting values "PDF", "JPEG", and "TIFF" corresponding to the item "format", and E which is a setting value of the item "destination". It includes a text box 146 that accepts input of an e-mail address, an [OK] icon 147, and a [Cancel] icon 148. That is, the operation parameters that can be specified through the parameter selection screen vary according to the operation ID specified through the operation selection screen.

Next, the terminal program 65 specifies, for example, the setting value "TIFF" of the item "format", the input of the setting value "aaa@brother.jp" of the item "destination", and the designation of the [OK] icon 147. (S34: OK), the icon image data generated in S24, and the operation ID and text data acquired in S32 and S34, as shown in FIG. 2 (B). And the set value are registered in the icon record including the icon ID “003” indicating the reserved area (S35). That is, the drag operation with the reserved area as the end position is an operation of registering the icon image data, the text data, the operation ID, and the operation parameter specified by the user in the icon record indicating the reserved area.

Next, returning to FIG. 4, the terminal program 65 causes the customization screen 130 shown in FIG. 10A to be displayed on the display 53 instead of the customization screen 130 shown in FIG. 7 (S11). The customization screen 130 shown in FIG. 10A is different from FIG. 7 in that the operation icon 139 is arranged at the position of the reserved area image 135. That is, the drag operation with the reserved area as the end position is an operation of newly displaying the operation icon associated with the operation specified by the user in the reserved area.

As yet another example, the terminal program 65 accepts a drag operation with the position of the icon 122 as the start position and the position of the insertion area image 136 of the customization screen 130 shown in FIG. 7 as the end position through the input I / F 54. Accordingly (S21: No), the non-designated image data and the designated image data are generated by the same method as described above using the target image data associated with the icon 122 (S24).

Next, the terminal program 65 has an icon record corresponding to the operation icons 133 and 134 after the insertion area image 136 in the arrangement order according to the end position of the drag operation being the insertion area (S25: No & S27: insertion). The operation ID of is moved down so as to indicate the position behind the sequence (S28). More specifically, the terminal program 65 replaces the icon ID "001" with "002" indicating the position behind the arrangement order in the screen instruction data shown in FIG. 2A, and replaces the icon ID "002" with "002". Replace with "003" indicating the position behind the sequence order.

Next, the terminal program 65 executes the icon registration process (S29). Then, the terminal program 65 acquires the icon image data generated in S24 and the icon image data in S32 and S34 as shown in FIG. 3A in response to the reception of the same user operation as described above in S32 and S34. The text data, the operation ID, and the operation parameter are registered in the icon record including the icon ID “001” indicating the position immediately after the insertion area image 136 in the arrangement order (S35).

That is, the drag operation with the insertion area as the end position lowers the operation ID of the icon code indicating the position behind the insertion area, and inserts the icon image data, text data, operation ID, and operation parameter specified by the user. This is an operation to register in the icon record indicating the position immediately after the area.

Next, returning to FIG. 4, the terminal program 65 causes the customization screen 130 shown in FIG. 10B to be displayed on the display 53 instead of the customization screen 130 shown in FIG. 7 (S11). In the customization screen 130 shown in FIG. 10B, the operation icons 133 and 134 are moved backward (that is, on the right side) in the arrangement order as shown by the broken line arrows, and the insertion area image 136 in the arrangement order. It differs from FIG. 7 in that the operation icon 139 indicated by the newly generated icon image data is displayed at the position immediately after. That is, in the drag operation with the insertion area as the end position, the position of the operation icon behind the insertion area is moved down, and the operation icon associated with the operation specified by the user is newly displayed at the position immediately after the insertion area. It is an operation to make it.

Note that FIGS. 7 to 10 have described an example of a drag operation in which a region different from the customization screen 130 on the display 53 is set as the start position and the end position is on the customization screen 130. Next, with reference to FIG. 11, a case where a drag operation with the start position and the end position on the customization screen 130 shown in FIG. 10A is accepted through the input I / F 54 will be described. The process of accepting such a drag operation is an example of the third acceptance process.

As an example, the terminal program 65 inputs a drag operation with the position of the operation icon 139 as the start position and the position of the operation icon 133 as the end position on the customization screen 130 shown in FIG. 10A through the input I / F 54. Depending on the acceptance, the icon IDs "001" and "003" shown in FIG. 2 (B) are replaced as shown in FIG. 3 (B). The operation icon 139 is an example of the first icon, and the operation icon 133 is an example of the second icon. That is, the drag operation for moving the first icon to the position of the second icon is an operation for replacing the icon ID of the icon record indicating the operation icons 133 and 139 at the start position and the end position.

Further, the terminal program 65 shifts the customization screen 130 in which the positions of the operation icons 133 and 139 are replaced to the customization screen 130 shown in FIG. 10 (A) as shown by the broken line arrow in FIG. 11 (A). Instead, it is displayed on the display 53. That is, the drag operation for moving the first icon to the position of the second icon is an operation for replacing the positions of the operation icons 133 and 139 at the start position and the end position. These processes are examples of replacement processes.

As another example, the terminal program 65 inputs a drag operation with the position of the operation icon 139 as the start position and the position of the insertion area image 136 as the end position on the customization screen 130 shown in FIG. 10 (A). As shown in FIG. 3 (A), the icon ID shown in FIG. 2 (B) is replaced from "001" to "002" and changed from "002" to "003" according to the reception through F54. Replace and replace "003" with "001". That is, in the drag operation of moving the operation icon to the insertion area, the operation ID of the icon code indicating the position behind the insertion area is moved down, and the icon ID indicating the position immediately after the insertion area is changed to the operation icon icon of the start position. It is an operation to assign to a record.

Further, in the terminal program 65, as shown by the broken line arrow in FIG. 11B, the operation icons 133 and 134 are moved backward (that is, on the right side) in the arrangement order, and immediately after the insertion area image 136 in the arrangement order. The customization screen 130 in which the operation icon 139 of the start position is moved to the position of is displayed on the display 53 instead of the customization screen 130 shown in FIG. 10 (A).

The operation icon 133 is an example of the first icon, the operation icon 134 is an example of the second icon, and the operation icon 139 is an example of the third icon. That is, the drag operation of moving the third icon to the insertion area immediately before the first icon moves the positions of the first icon and the second icon behind the insertion area to the position immediately before the moved first icon. 3 This is an operation to move the icon. These processes are an example of the sort process.

Next, returning to FIG. 4, in the terminal program 65, for example, on the customization screen 130 shown in FIG. 11B, the pointer image 110 moves from a position different from the operation icon 139 onto the operation icon 139 (so-called mouse over). ) (S13: Yes), the symbol of the operation icon 139 is changed from the symbol indicated by the non-specified image data "custom02.normal.jpeg" to the specified image data "custom02.pushed.jpeg". Switch to the indicated symbol (S17 → S11). The process of S17 is an example of the first switching process.

Further, in the terminal program 65, for example, on the customization screen 130 shown in FIG. 11B, the pointer image 110 is moved (so-called mouse out) from the operation icon 139 to a position different from the operation icon 139. (S14: Yes), the symbol of the operation icon 139 is switched from the symbol indicated by the specified image data "custom02.pushed.jpeg" to the symbol indicated by the non-specified image data "custom02.normal.jpeg" (S18). → S11). The process of S18 is an example of the second switching process.

Further, the terminal program 65 receives the designation of the [send] icon 132 shown in FIG. 11 (B) through the input I / F 54 (S15: Yes), and the screen instruction shown in FIG. 3 (A). Data (hereinafter, may be referred to as “custom screen data”) is directly or indirectly output to the designated device (S19). The process of S19 is an example of an output process and a transmission process.

As an example of direct output, the terminal program 65 may transmit custom screen data to the designated device through the communication I / F 55. As an example of indirectly outputting, the terminal program 65 may notify an output program (not shown) installed in the information processing terminal 50 of a pointer indicating the start address of the memory area in which the custom screen data is stored. Then, the output program may read the custom screen data from the memory area indicated by the pointer notified from the terminal program 65, and transmit the read custom screen data to the designated device through the communication I / F 55.

On the other hand, although not shown, the device program 35 of the MFP 10 which is a designated device receives custom screen data from the information processing terminal 50 through the communication I / F 25. This process is an example of a receive process. Then, the device program 35 stores the received custom screen data in the memory 32 separately from the standard screen data. Further, the device program 35 causes the display 23 to display the icon screen shown in FIG. 12, for example, in response to the power of the MFP 10 being turned on.

More specifically, the device program 35 determines whether or not the custom screen data is stored in the memory 32. In response to the determination that the custom screen data is not stored in the memory 32, the device program 35 reads the standard screen data shown in FIG. 2 (A) from the memory 32, and according to the read standard screen data, FIG. 12 ( The icon screen shown in A) is displayed on the display 23. The icon screen shown in FIG. 12A includes operation icons 133 and 134 and a [start] icon 150. Then, the device program 35 accepts the user operation for the icon screen through the input I / F24. This process is an example of the second reception process.

Further, the device program 35, for example, receives the designation of the operation icon 133 through the input I / F 54, and receives the symbol of the operation icon 133 from the symbol indicated by the non-designated image data as the designated image data. Change to the indicated pattern. Further, the device program 35 receives the designation of the [start] icon 150 through the input I / F24 in the state where the operation icon 133 is specified, and receives the icon record corresponding to the operation icon 133 from the screen instruction data. read out. Then, the device program 35 executes the operation indicated by the operation ID of the read icon record according to the execution conditions indicated by the operation parameters. This process is an example of an execution process.

On the other hand, the device program 35 reads the custom screen data shown in FIG. 3A from the memory 32 in response to the determination that the custom screen data is stored in the memory 32, and the figure is taken according to the read custom screen data. The icon screen shown in 12 (B) is displayed on the display 23. This process is an example of the second display process. The icon screen shown in FIG. 12B is different from FIG. 12A in that the operation icons 139, 133, and 134 are arranged in this order. Further, the process executed by the user operation on the icon screen is common to the case of FIG. 12A.

[Action and effect of this embodiment]
According to the above embodiment, by dragging the icons 122, 124, 125 displayed on the display 53 by the list program 66 to the customization screen 130, the icon screen is displayed with the target image data corresponding to the icons 122, 124, 125. Can be customized. As a result, the number of customization options is greatly increased, and the icon screen can be customized more flexibly.

For example, as shown in FIG. 8A, by dragging the icon 125 to the background area, the process of changing the background pattern of the icon screen can be realized by an intuitive operation. Further, as shown in FIG. 8B, by dragging the icon 124 to the position of the operation icon 134, the process of changing the operation icon 134 to a desired symbol can be realized by an intuitive operation. ..

Further, as shown in FIG. 10A, by dragging the icon 122 to the position of the reserved area image 135, the process of newly adding the operation icon 139 of the desired symbol is realized by an intuitive operation. be able to. Further, as shown in FIG. 10B, the process of inserting the operation icon 139 of the desired symbol at the desired position by dragging the icon 122 to the position of the insertion area image 136 can be performed by an intuitive operation. It can be realized. Further, by the icon registration process, a desired operation can be assigned to the newly displayed operation icon 139. This makes it possible to customize the icon screen more flexibly.

Further, according to the above embodiment, the user is made to specify the image data to be used as the operation icon 139, and the icon image data is generated from the designated image data. At this time, since the aspect ratio of the symbol indicated by the target image data is maintained, it is possible to prevent the symbol from being distorted at the time of enlargement or reduction. As a result, the operation icon 139 of the desired symbol can be displayed on the icon screen. Similarly, when the background image data is generated, the aspect ratio of the symbol indicated by the target image data is maintained, so that the symbol can be suppressed from being distorted at the time of enlargement or reduction. As a result, a desired background pattern can be displayed on the icon screen.

Further, according to the above embodiment, if only one target image data is specified by the user, the display mode of the operation icon 139 before and after the designation can be changed. That is, the icon screen can be easily and flexibly customized. Then, by moving the pointer image 110 over or out of the operation icon 139 on the customization screen 130, the user can easily confirm the change in the display mode of the operation icon 139 before and after the designation. can.

Further, according to the above embodiment, as shown in FIG. 11, by dragging the operation icon 139 on the customization screen 130, the positions of the operation icons 133, 134, and 139 are replaced or rearranged. As a result, the process of replacing or rearranging the positions of the operation icons on the icon screen can be realized by an intuitive operation.

In the above embodiment, an example of customizing the so-called top screen that is displayed first when the power of the MFP 10 is turned on has been described, but the customizable screen is not limited to the top screen, and the display 23 of the MFP 10 is used. Any screen displayed on may be customized. It should be noted that the "first screen displayed" does not include, for example, a so-called splash screen for displaying the manufacturer's logo, a screen for asking for consent to the rules, and the like.

Further, in the MFP 10 and the information processing terminal 50 of the above embodiment, each process executed by the controller of the present invention is realized by executing various programs stored in the memories 32 and 62 by the CPUs 31 and 61. Explained. However, the configuration of the controller is not limited to this, and a part or all of the controller configuration may be realized by hardware such as an integrated circuit.

Further, the present invention can be realized not only as the MFP 10 and the information processing terminal 50, but also as a program for causing the MFP 10 and the information processing terminal 50 to execute processing. Then, the program may be recorded and provided on a non-transitory recording medium. The non-transitory recording medium may include a storage unit mounted on a server that can be connected to the MFP 10 and the information processing terminal 50 through a communication network, in addition to a CD-ROM, a DVD-ROM, and the like. Then, the program stored in the storage unit of the server may be distributed as information or a signal indicating the program through a communication network such as the Internet.

10 ... MFP
11 ... Printer 23, 53 ... Display 24, 54 ... Input I / F
25, 55 ... Communication I / F
31,61 ... CPU
32, 62 ... Memory 34, 64 ... OS
35 ... Device program 50 ... Information processing terminal 65 ... Terminal program 66 ... List program

Claims (12)

A program that can be executed by an information processing terminal that includes a memory, a terminal display, and an input interface.
The memory stores screen instruction data indicating an icon screen to be displayed on the device display of the image forming apparatus, and the screen instruction data includes an operation icon to be displayed on the icon screen and a plurality of the above on the icon screen. Indicates the order of the operation icons,
The program
The icon screen displayed on the device display of the image forming apparatus and the operation icon are arranged in common , and are arranged at a plurality of positions adjacent to the front of each of the plurality of operation icons in the arrangement order. A customized screen including an insertion area image that is temporarily displayed at least in a drag operation is displayed in a part of the terminal display according to the screen instruction data , and the customized screen is displayed in the area. a first display processing Ru is displayed by placing the send icon together with the operation icon,
A first reception process that accepts a drag operation for moving a target icon displayed by an external program in an area different from the customization screen onto the insertion area image of any one of the customization screens through the input interface.
Target image data corresponding to the target icon moved on the insertion area image by moving the operation icon behind the arrangement order from the insertion area image to which the target icon has been moved to the rear of the arrangement order. A customization process for customizing the screen instruction data so that the operation icon of the symbol indicated by is newly displayed at a position immediately after the insertion area image in the arrangement order.
A program that causes the information processing terminal to execute an output process of outputting the screen instruction data customized by the customization process to the image forming apparatus in response to receiving an operation of the transmission icon through the input interface. The program according to claim 1, wherein the program replaces the symbol of the operation icon with the symbol indicated by the target image data in the customization process when the target icon is moved to the position of the operation icon. The screen instruction data indicates a reserved area on which the new operation icon can be added on the icon screen.
The customization screen includes a reserved area image showing the reserved area.
In the customization process when the target icon is moved to the position of the reserved area image, the program newly displays the operation icon of the symbol indicated by the target image data at the position of the reserved area image. The program according to claim 1. The screen instruction data further indicates an operation to be executed by the image forming apparatus when the operation icon is specified.
The program is used in the customization process.
The information processing terminal is made to execute a second reception process for receiving a user operation for designating one of the plurality of operations that can be executed by the image forming apparatus through the input interface.
A claim in which an operation ID for identifying the operation specified in the second reception process is registered in the screen instruction data as an operation to be executed by the image forming apparatus when the newly displayed operation icon is specified. The program described in any one of 1 to 3. The program is used in the customization process.
While maintaining the aspect ratio of the image indicated by the target image data, the image is enlarged or reduced to the size of the icon of the operation icon or less, and transparent pixels are added to the outer edge of the enlarged or reduced image to create an image of the icon size. The information processing terminal is made to execute the first generation process for generating the icon image data indicating the above.
The program according to any one of claims 1 to 4 , wherein the icon image data generated in the first generation process is registered in the screen instruction data. The icon image data is
The first image data showing the design of the operation icon when not specified by the user, and
Includes the second image data indicating the design of the operation icon when specified by the user.
The program is used in the customization process.
The first generation process for generating the first image data from the target image data, and
The information processing terminal is made to execute the second generation process of generating the second image data from the first image data, and the second image data has the same outline as the symbol shown in the first image data. And show a pattern in which at least one of brightness, lightness, and saturation is different.
The fifth aspect of claim 5, wherein the icon image data including the first image data generated in the first generation process and the second image data generated in the second generation process is registered in the screen instruction data. program. The input interface accepts a user operation instructing the movement of the pointer image displayed on the terminal display.
The program
In response to the pointer image being moved onto the motion icon from a position different from the motion icon, the symbol of the motion icon is changed from the symbol shown in the first image data to the symbol shown in the second image data. The first switching process to switch to, and
In response to the movement of the pointer image from the operation icon to a position different from the operation icon, the symbol of the operation icon is changed from the symbol shown in the second image data to the symbol shown in the first image data. The program according to claim 6 , wherein the information processing terminal executes the second switching process of switching to. The screen instruction data indicates a background pattern of the icon screen.
The program according to any one of claims 1 to 7 , wherein the background symbol is replaced with a symbol indicated by the target image data in the customization process when the target icon is moved to the position of the background symbol. program. The program is used in the customization process.
While maintaining the aspect ratio of the image indicated by the target image data, the image is enlarged or reduced to the size of the display area of the device display or less, and pixels of a predetermined color are added to the outer edge of the enlarged or reduced image. , The information processing terminal is made to execute a third generation process for generating background image data indicating an image of the display area size.
The program according to claim 8 , wherein the background image data generated in the third generation process is registered in the screen instruction data. The icon screen includes the first icon and the second icon, which are the operation icons.
The program
A third reception process that accepts a drag operation for moving the operation icon displayed on the customization screen through the input interface, and
In response to the movement of the first icon to the position of the second icon in the third reception process, the information processing terminal is subjected to a replacement process of replacing the display positions of the first icon and the second icon. The program according to any one of claims 1 to 9 to be executed. In the icon screen, the first icon, the second icon, and the third icon, which are the operation icons, are arranged in this order.
In the customization screen , the plurality of insertion area images are adjacent to the front of the arrangement order for each of the plurality of operation icons.
The program
A third reception process that accepts a drag operation for moving the operation icon displayed on the customization screen through the input interface, and
In response to the movement of the third icon to the insertion area image adjacent to the front of the first icon in the third reception process, the first icon and the second icon are moved to the rear of the arrangement order. It is allowed, and the program according to any of claims 1 1 0 that executes a rearrangement process of moving the third icon at a position adjacent to the front of the first icon moves, to the information processing terminal. An information processing terminal including a memory, a terminal display, a first input interface, a first communication interface, and a first controller, and an image forming device including a device display, a second input interface, a second communication interface, and a second controller. It is an information processing system that is equipped
The memory stores screen instruction data indicating an icon screen to be displayed on the device display, and the screen instruction data includes an operation icon to be displayed on the icon screen and the image formation when the operation icon is specified. An operation to be executed by the device and an arrangement order of a plurality of the operation icons on the icon screen are shown.
The first controller is
The icon screen displayed on the device display of the image forming apparatus and the operation icon are arranged in common , and are arranged at a plurality of positions adjacent to the front of each of the plurality of operation icons in the arrangement order. A customized screen including an insertion area image that is temporarily displayed at least in a drag operation is displayed in a part of the terminal display according to the screen instruction data , and the customized screen is displayed in the area. a first display processing Ru is displayed by placing the send icon together with the operation icon,
A first reception process that accepts a drag operation for moving a target icon displayed in an area different from the customization screen onto the insertion area image on any of the customization screens through the first input interface.
The operation icon behind the arrangement order from the insertion area image to which the target icon was moved in the first reception process is moved to the rear of the arrangement order, and the target icon is moved onto the insertion area image. A customization process for customizing the screen instruction data so that the operation icon of the symbol indicated by the target image data corresponding to the above is newly displayed at a position immediately after the insertion area image in the arrangement order.
In response to receiving the operation of the transmission icon through the first input interface, the transmission process of transmitting the screen instruction data customized by the customization process to the image forming apparatus through the first communication interface is executed. ,
The second controller is
A reception process for receiving the screen instruction data from the information processing terminal through the second communication interface, and
A second display process for displaying the icon screen indicated by the screen instruction data on the device display, and
A second reception process for accepting a user operation for designating the operation icon displayed on the device display through the second input interface, and
An information processing system that executes an execution process that executes the operation associated with the operation icon specified in the second reception process.

Images