JP6086015B2 - Image processing apparatus, method by which image processing apparatus transmits image data to terminal apparatus, and program - Google Patents

Description

  The present disclosure relates to an image processing device, and more particularly, to an image processing device that transmits image data to a terminal device.

  Conventionally, a terminal device is used as a remote terminal of an image processing device. The image processing apparatus transmits display data of screen information for displaying an operation screen of the image processing apparatus to the terminal device. In response to this, the terminal device displays the image information.

  Various devices that generate and transmit screen information to other devices such as terminal devices have been proposed. For example, Japanese Unexamined Patent Application Publication No. 2011-113344 (Patent Document 1) discloses a program generation device that generates a control program for causing a mobile terminal to display screen information including characters. When the device generates the control program, and the screen information includes a character represented by a typeface that is not compatible with the mobile terminal, an image representing the character is represented by the typeface type. Image data to be shown is generated in advance. Thereby, when the control program is executed in the portable terminal, the characters represented by the type of font that the portable terminal does not support are displayed according to the image data generated in advance.

JP 2011-113344 A

  In the technique disclosed in Patent Literature 1, the program generation device generates image data such that characters of a font that the mobile terminal does not support are displayed on the mobile terminal, and the mobile terminal Sending to the terminal. Here, in the image data transmission destination device (such as a portable terminal), although the above characters are displayed, they are displayed in a collapsed manner or due to the thickness of the line. Due to resolution or the like, it may be displayed in a manner that is difficult for the user to see. In the screen information, not only characters but also images may be displayed in a manner that is difficult to see on the terminal side.

  However, with the technique disclosed in Patent Document 1, the image data transmission source device cannot recognize the display mode of the character on the portable terminal (terminal device) that is the transmission destination of the image data. Therefore, there has been a demand for a technique for displaying image data in a transmission source apparatus so that images and characters are displayed in an easy-to-view manner on the transmission destination apparatus.

  The present disclosure has been conceived in view of such circumstances, and the purpose of the present disclosure is to enable an image processing apparatus to transmit image data that allows an image or a character to be displayed in an easily viewable manner to the terminal apparatus. Is to do.

  According to an aspect, an image processing apparatus capable of communicating with a terminal device is provided. An image processing apparatus includes: a first transmission unit configured to transmit, to a terminal device, first data that is image data of an operation screen for combining and displaying an image and a character; From the reception means configured to receive the second data which is the screen data for displaying the first data in the terminal device, and the comparison result between the first data and the second data A second transmission unit configured to generate third data, which is image data of an operation screen for combining and displaying an image and a character, and to transmit the third data to the terminal device; Prepare.

  Preferably, the second transmission unit generates the third data when the difference between the image displayed by the first data and the image displayed by the second data is equal to or larger than a predetermined amount. Is configured to do.

  Preferably, the comparison result is at least one of a comparison result between portions corresponding to the graphic area on the operation screen and a comparison result between portions corresponding to the character area on the operation screen between the first data and the second data. Including one.

  Preferably, the comparison result between the portions corresponding to the character areas on the operation screen is represented by the number of matching characters by using the character recognition process.

  Preferably, the comparison result between the portions corresponding to the graphic area on the operation screen is represented by the number of pixels with the matched information.

  Preferably, the comparison result is a comparison result between portions corresponding to specific display contents on the operation screen.

  Preferably, the specific display content is a display regarding the image processing operation on the image processing apparatus, and a start key for inputting an instruction to start the image processing operation and an instruction content of the number of copies regarding the image processing operation are displayed. At least one of a display area for resetting, a reset key for instructing reset of setting contents for the image processing operation, and a stop key for inputting an instruction to stop the image processing operation.

  Preferably, the comparison result includes a comparison result between portions corresponding to the graphic area on the operation screen and a comparison result between portions corresponding to the character area on the operation screen of the first data and the second data. When the amount of difference between the portions of the first data and the second data corresponding to the character area on the operation screen is less than a predetermined amount, the second transmission means Even if the amount of difference between portions corresponding to the region exceeds a predetermined amount, the third data is not generated.

  Preferably, the comparison result includes a comparison result between portions corresponding to the graphic area on the operation screen and a comparison result between portions corresponding to the character area on the operation screen of the first data and the second data. If the amount of difference between the portions of the first data and the second data corresponding to the graphic area on the operation screen is less than a predetermined amount, the second transmission means Even if the amount of difference between portions corresponding to the region exceeds a predetermined amount, the third data is not generated.

  Preferably, the comparison result is a part of the second operation screen when the first transmission unit transmits the first data for the second operation screen in which a part of the first operation screen is changed. It is a comparison result about the part corresponding to. The second transmission means includes a portion corresponding to a portion other than a portion in the history of the third data for the first operation screen, and a portion generated for a portion of the second operation screen based on the comparison result. By combining them, the third data for the second operation screen is generated.

Preferably, the image processing apparatus further includes third transmission means for transmitting information requesting transmission of the second data to the terminal device.

  Preferably, the terminal device further includes fourth transmission means configured to transmit information prohibiting the display of the first data and information permitting the display of the third data to the terminal device.

  According to another aspect, a method is provided in which an image processing device transmits image data to a terminal device. The method includes a step of transmitting first data, which is image data of an operation screen for combining and displaying an image and characters, to the terminal device, and the first data is transmitted from the terminal device to the terminal device. Based on the result of the comparison in the step of receiving the second data, which is the screen information data to be displayed, the step of comparing the first data and the second data, and the step of comparing, the image and the character Generating third data that is image data of an operation screen for combining and displaying the data, and transmitting the third data to the terminal device.

  According to still another aspect, a program for causing a computer to implement the above-described method is provided.

  According to the present disclosure, the image processing apparatus can transmit image data such that an image or a character is displayed in an easily viewable manner on the terminal device.

It is a figure which shows the specific example of a structure of the image processing system concerning embodiment. FIG. 3 is a diagram illustrating a specific example of a hardware configuration of an MFP (Multi-Functional Peripheral) included in the image processing system. It is a figure which shows the specific example of the hardware constitutions of a terminal device in an image processing system. FIG. 2 is a diagram schematically showing an overview of image data transmission / reception processing between an MFP and a terminal device in an image processing system. FIG. 5 is a diagram showing a flow of processing in the MFP for realizing the transmission / reception processing shown in FIG. 4. 2 is a block diagram illustrating a specific example of a functional configuration of an MFP. FIG. It is a figure which shows the specific example of the sequence of the transmission / reception process of the image data performed in an image processing system. It is a figure which shows an example of an operation screen typically. FIG. 9 is an enlarged view of a part of the operation screen of FIG. 8. It is a figure which shows an example of the object list | wrist contained in image data. It is a flowchart of the subroutine of step S40 of FIG. It is a figure for demonstrating the production | generation aspect of image data (3). It is a figure for demonstrating the production | generation aspect of image data (3). It is a figure for demonstrating the production | generation aspect of image data (3). It is a figure which shows an example of an operation screen. It is a figure which shows the other example of an operation screen. It is a figure which shows the functional block of the modification of MFP. FIG. 10 is a diagram for describing a generation mode of image data (3) in a modification of the MFP. FIG. 10 is a functional block diagram of another modification of the MFP.

  Hereinafter, an example of an image processing system including an embodiment of an image processing apparatus will be described with reference to the drawings. In the following description, parts having the same function and action are denoted by the same reference numerals throughout the drawings, and redundant description will not be repeated.

<System configuration>
FIG. 1 is a diagram illustrating a specific example of a configuration of an image processing system including an embodiment of an image processing apparatus. As shown in FIG. 1, the image processing system includes an MFP (Multi-Functional Peripheral) 100 as an example of an image processing apparatus and a terminal device 200 as an example of a control apparatus. These are connected to the network 300.

  The terminal device 200 may be any device as long as it can be connected to the network 300 and can input an instruction from a user. Preferably, a portable small terminal device is assumed. Therefore, the terminal device 200 may be, for example, a mobile phone, a tablet terminal device, a small computer, or the like. The network 300 is a LAN (Local Area Network), for example, and may be a wired connection or a wireless connection. The image processing apparatus is not limited to the MFP, and may be other apparatuses such as a printer, a facsimile transceiver, and a copier. The MFP 100 is an image processing apparatus having a combination of these functions.

<Hardware configuration of MFP>
FIG. 2 is a diagram illustrating a specific example of the hardware configuration of the MFP 100. As shown in FIG. 2, the MFP 100 includes a CPU (Central Processing Unit) 10 that is an arithmetic device for controlling the whole.

  The MFP 100 further includes a ROM (Read Only Memory) 11 for storing a program executed by the CPU 10, a RAM (Random Access Memory) 12 for functioning as a work area when the CPU 10 executes the program, and an image. An HD (hard disk) 16 for storing data, a scanner 13 for optically reading a document placed on a document table (not shown), and obtaining image data, and fixing the image data on a print sheet A printer 14, an operation panel 15 including a touch panel for displaying information and receiving an operation input to the MFP 100, and a network controller 17 for controlling communication via the network 300.

  The program executed by the CPU 10 may be stored in a storage device on the network in addition to being stored in the ROM 11, or may be downloaded from the network and installed in the HD 16, Further, it may be stored in a recording medium that is detachable from MFP 100.

<Hardware configuration of terminal device>
FIG. 3 is a diagram illustrating a specific example of the hardware configuration of the terminal device 200. As shown in FIG. 3, the terminal device 200 includes a CPU 20 that is an arithmetic device for controlling the whole.

  The terminal device 200 further includes a ROM 21 for storing a program executed by the CPU 20 and various information, a RAM 22 for functioning as a work area when the CPU 20 executes the program, and displays the information or the terminal device. 200 includes an operation panel 25 including a touch panel for receiving an operation input to 200, and a network controller 27 for controlling communication via the network 300.

  Operation panel 25 may have the same configuration as operation panel 15 of MFP 100. That is, as an example, a touch panel configured by overlapping a display device such as a liquid crystal display device and a position indicating device such as an optical touch panel or a capacitive touch panel, and an operation key are included.

  The operation panel 25 of the terminal device 200 is a specific example of a device that serves as both a display device and an input device. For example, when the control device is a mobile phone, the computer as the control device may include a display and a key group instead of the operation panel 25.

  CPU20 displays an operation screen on a touchscreen based on the data for displaying the screen memorize | stored beforehand. On the touch panel, an indicated position on the operation screen is specified, and an operation signal indicating the position is input to the CPU 20. An operation signal indicating the pressed key is input to the CPU 20. The CPU 20 specifies the operation content from the pressed key or the displayed operation screen and the indicated position, and executes processing based on the operation content.

<Outline of processing>
In the image processing system according to the present embodiment, MFP 100 transmits image data of a screen (operation screen) for inputting an operation instruction to terminal device 200. Terminal apparatus 200 displays the received image data on operation panel 25 and accepts an operation instruction input to MFP 100. Terminal apparatus 200 transmits the input operation instruction to MFP 100. Thereby, the user can cause MFP 100 to execute an operation (including an image processing operation) according to the operation instruction by inputting the operation instruction to terminal device 200.

  FIG. 4 is a diagram schematically showing an outline of image data transmission / reception processing between the MFP 100 and the terminal device 200 in the image processing system of the present embodiment. The outline of the process will be described with reference to FIG.

  Referring to FIG. 4, first, in MFP 100, the content to be displayed as the operation screen is specified according to the operation state of MFP 100. Then, MFP 100 generates image data for displaying an operation screen based on the specified content. In FIG. 4, the image data generated in this way is indicated as “image data (1)”. Then, MFP 100 transmits image data (1) to terminal device 200.

  When receiving the image data (1), the terminal device 200 performs a process (display process) for displaying on the operation panel 25 with respect to the image data (1). The display process includes, for example, a process of changing the image data so that the size of the image data (1) is adapted to the operation panel 25. In FIG. 4, the image data after the display processing is shown as “image data (2)”. Then, the terminal device 200 transmits the image data (2) to the MFP 100.

  When the image data (2) is received, the MFP 100 compares the image data (1) and the image data (2), and generates image data of the operation screen again based on the comparison result. The image data generated in this way is shown as “image data (3)” in FIG. The image data (3) may be generated by correcting part or all of the image data (1). Then, MFP 100 transmits image data (3) to terminal device 200.

  When receiving the image data (3), the terminal device 200 generates image data according to the display process described later on the image data (3), and displays an operation screen according to the image data on the operation panel 25.

  The image data (3) is generated, for example, by complementing the difference between the image data (2) and the image data (1). More specifically, in the image data (2), for example, when the image and / or character displayed clearly in the image data (1) is unclear, the image and / or character is displayed. It is generated by correcting the image data (1) so that it is clearly displayed.

<Processing flow in MFP>
FIG. 5 is a diagram showing a flow of processing in the MFP 100 for realizing the transmission / reception processing as shown in FIG. The process shown in FIG. 5 is started on the condition that a condition for updating the operation screen of MFP 100 is satisfied when an operation instruction is given to MFP 100 or the like.

  With reference to FIG. 5, CPU10 transmits the image data (1) demonstrated with reference to FIG. 4 with respect to the terminal device 200 by step S10, and advances control to step S20. The image data (1) may be prepared in advance, or may be generated when step S10 is executed.

  In step S20, the CPU 10 transmits information requesting transmission of image data to the terminal device 200, and the control proceeds to step S30.

  In step S30, the CPU 10 receives the image data (2) described with reference to FIG. 4 from the terminal device 200, and advances the control to step S40.

  In step S40, the CPU 10 compares the image data (1) and the image data (2), and advances the control to step S50.

  In step S50, the CPU 10 determines whether or not there is a difference between the two data as a result of the comparison in step S40. For example, when comparing the pixels between the two data, the CPU 10 determines that there is a difference between the two data when the number of different pixels in the data is equal to or greater than a predetermined number. When the number is less than a predetermined number, it is determined that there is no difference between the two data. If CPU 10 determines that there is a difference between the two data, control proceeds to step S60.

  On the other hand, in step S50, if the CPU 10 determines that there is no difference between the two data, the process ends. Thereby, in the terminal device 200, the operation screen according to image data (2) is displayed.

  In step S60, the CPU 10 generates the image data (3) described with reference to FIG. 4 based on the comparison result in step S40, and advances the control to step S70. The CPU 10 may generate the image data (3) from the image data for displaying the operation screen, or may generate the image data (3) by correcting the image data (1).

  In step S70, the CPU 10 transmits the image data (3) to the terminal device 200 and ends the process. Thereby, in the terminal device 200, the operation screen according to the image data produced | generated by performing the process for a display which image data (3) mentions later is displayed.

<Functional configuration of MFP>
FIG. 6 is a block diagram showing a specific example of a functional configuration of MFP 100 for performing control for the above-described image data transmission / reception processing. Each function shown in FIG. 6 is mainly realized by CPU 10 when CPU 10 of MFP 100 reads a program, which is an application for performing remote panel control, onto RAM 12 and executes it. The program is stored in, for example, the ROM 11, a storage device accessible by the CPU 10, and / or a recording medium removable from the MFP 100. Each function shown in FIG. 6 may be realized at least in part by the hardware configuration shown in FIG. 2 or other electric circuits.

  As shown in FIG. 6, the HD 16 is provided with a first operation screen storage unit 161 that is a storage area for storing screen information (image data) of the operation screen.

  Further, as shown in FIG. 6, the CPU 10 includes an instruction input unit 101 for receiving an input of instruction information transmitted from the terminal device 200 via the network controller 17, an operation screen that is already displayed with the instruction information. From the update unit 102 for executing the process specified based on the above and updating the operation screen, the transmission unit 103 for transmitting image data to the terminal device 200 via the network controller 17, and the terminal device 200 And a determination unit 109 for performing determination based on the transmitted image data (image data (2)).

  The transmission unit 103 includes a first transmission unit 104, a second transmission unit 106, and a third transmission unit 120. The first transmission unit 104 transmits the image data (1) described with reference to FIG. The second transmission unit 106 transmits the image data (3) described with reference to FIG. Specifically, the second transmission unit 106 includes a generation unit 107 for generating image data (3) and a transmission unit 108 for transmitting the image data (3) to the terminal device 200. The third transmission unit 120 transmits a request for transmission of image data (2) (step S20 in FIG. 5) to the terminal device 200.

  The determination unit 109 includes an acquisition unit 110, a comparison unit 111, an area determination unit 112, a character recognition unit 113, and an extraction unit 114. The acquisition unit 110 acquires the image data (2) transmitted from the terminal device 200 via the network controller 17. The comparison unit 111 compares the image data (1) and the image data (2). The area determination unit 112 classifies the image data (1) and / or the image data (2) into areas for each feature, and determines the type of the area corresponding to the feature. For example, the region determination unit 112 determines whether each region classified for each feature in the image data is a character region (a region including characters) or a graphic region (a region including graphics). The character recognition unit 113 uses, for example, an OCR (Optical Character Reader) function to detect characters included in an image corresponding to the image data for all or part of the image data. The extraction unit 114 extracts image data corresponding to a given display content from the image data (1) and / or the image data (2).

<Processing sequence in image processing system>
FIG. 7 is a diagram showing a specific example of a sequence of image data transmission / reception processing executed in the image processing system shown in FIG.

  Referring to FIG. 7, first, in step SA10, MFP 100 (CPU 10 thereof) creates panel display data (data for specifying contents to be displayed on the operation screen) based on the operation state of MFP 100, and the panel display. Image data (1) is generated based on the data, and the image data (1) is transmitted to the terminal device 200.

  Next, in step SB10, the terminal device 200 (the CPU 20) receives the image data (1) and displays the image data (1) on the terminal device 200 (the operation panel 25) for displaying the image data (1). To generate image data (2), and transmit the image data (2) to MFP 100.

  Next, in step SA20, MFP 100 (CPU 10 thereof) receives image data (2), compares image data (2) with image data (1), and determines that there is a difference between these data. Then, the image data (3) is generated, and the image data (3) is transmitted to the terminal device 200.

  Next, in step SB20, the terminal device 200 (the CPU 20 thereof) receives the image data (3), displays an operation screen according to the image data (3) on the operation panel 25, and ends the processing. . Thus, the user can input an operation instruction for the MFP 100 to the terminal device 200 while viewing the operation screen according to the image data (3).

<Image data>
The configuration of image data in the present embodiment will be described with reference to FIGS. FIG. 8 is a diagram schematically illustrating an example of the operation screen. FIG. 9 is an enlarged view of a part of the operation screen of FIG. FIG. 10 is a diagram illustrating an example of an object list included in image data.

  As shown in FIG. 8, the operation screen 890 includes buttons 150, 130, 140, 150 that are software buttons and a display field 170. The button 150 is a menu button for selecting a menu for document image quality. Button 130 is a start key for inputting an instruction to start image processing operation to MFP 100. The button 140 is a reset key for instructing the MFP 100 to reset the setting contents for the image processing operation. Button 150 is a stop key for inputting an instruction to stop image processing operation to MFP 100. Display column 170 is a display area for displaying instruction content of the number of copies (an example of an image processing operation) on MFP 100.

  FIG. 9 shows an enlarged view of an area (area 151) near the button 150 on the operation screen 890. Referring to FIG. 9, a button 150 includes a graphic region 152 including a pictographic image (in the example of FIG. 9, an image showing a book spread page) and a character string (in the example of FIG. 9, the character string “character / character / And a character area 153 including a “printed photograph”). The area 151 further includes a character area 154 that is arranged in the vicinity of the button 150 and includes a character string (in the example of FIG. 9, the character string “original image quality”) that can be set by the button 150.

  FIG. 10 shows an example of the contents of the portion corresponding to the area 151 in the object list included in the image data on the operation screen 890. The object list is a list of data for each object included in the image data. Contains one or more elements per object. FIG. 10 shows an example of data that can be registered in the object list for one object.

  FIG. 10 shows data of an object including four elements with IDs “0001” to “0004”. Each element has a type, a position, a size, and a code. The type is information that specifies the type of each object. The type is, for example, a graphic or a character. The position is information for specifying the position where each object is displayed, and is indicated, for example, by coordinates in the upper left xy plane of each object. The size is information for specifying the size of each object, and is represented by, for example, a dimension in the x-axis direction and a dimension in the y-axis direction on the xy plane. The code is information for specifying the contents of each object. The code of the object of type “figure” is information for specifying an image file to be displayed as an object, for example. The code of the object of type “character” is information for specifying a character string (combination thereof) such as an ASCII standard code for information interchange (ASCii) code.

  In the xy coordinates of the position and size in FIG. 10, the x-axis corresponds to the horizontal direction in FIG. 9, and the y-axis corresponds to the vertical direction in FIG. In the example shown in FIG. 10, the element with ID “0001” corresponds to the button 150 in FIG. 9. The image of the button 150 in FIG. 9 (the image of the button having a substantially square shape with rounded corners) corresponds to the graphic code of ID “0001” in FIG.

  The element of ID “0002” corresponds to the graphic area 152 of FIG. 9 corresponds to the graphic code of ID “0002” in FIG. 10.

  The element of ID “0003” corresponds to the character area 153 of FIG. The character string (“character / photo (line feed) printed photo”) in the character area 153 in FIG. 9 corresponds to the character code of ID “0003” in FIG.

  The element of ID “0004” corresponds to the area 154 in FIG. The character string (“original image quality”) in the area 154 in FIG. 9 corresponds to the character code of ID “0004” in FIG.

  The image data includes an object list that includes information about the objects constituting the operation screen. The object list includes information for specifying the position and content of one or more elements (such as the button 150) included in the object for each object included in the image data. The image data may include image data (for example, an image file corresponding to the graphic code of ID “0001” in FIG. 10) for displaying an element of the type “graphic”.

  On the other hand, the image file may not be included in the image data. In this case, the image file corresponding to the graphic code in the object list is stored in a storage device on the network, for example. A device that displays image data reads an image file from the storage device and displays an image corresponding to the graphic code.

  When the image data (2) is generated from the image data (1), for example, the object list of the image data (1) is changed according to the size of the operation panel 25 of the terminal device 200. In this specification, the object list included in the image data (n) is denoted as “object list (n)”. That is, the object list (1) is an object list included in the image data (1). The image data is displayed so that the operation screen is enlarged or reduced on the operation panel 25 while the arrangement of objects (positional relationship between the objects) on the operation screen corresponding to the image data (1) is maintained. By changing the position and size of each element in each object of the object list (1) of (1), the object list (2) is generated.

<Comparison of image data>
With reference to FIG. 11, a comparison between the image data (1) and the image data (2), which has been described as the processing in step S40 of FIG. FIG. 11 shows a flow of processing executed in the MFP 100 for the comparison, and corresponds to a flowchart of a subroutine of step S40 in FIG.

  Referring to FIG. 11, in step S402, CPU 10 obtains object list (1) from image data (1), and proceeds to step S404.

  In step S404, the CPU 10 acquires the object list (2) from the image data (2), and advances the control to step S406.

  In step S406, the CPU 10 selects the object arranged at the head in the object list (1) as the object to be processed, and advances the control to step S408.

  In step S408, the CPU 10 cuts out image data of the object to be processed in the image data (1), and advances the control to step S410. The range in which the image data of the object to be processed is cut out is a range that forms the outer edge of the region specified from the positions and sizes of all the elements belonging to the object. The CPU 10 determines the image data in the area specified by the position and size of the object to be processed in the object list (1) from the image information (for example, bitmap data) displayed by the image data (1), for example. Obtained by cutting out pixel information (for example, bitmap data).

  In step S410, based on the object list (2), the CPU 10 obtains an object area corresponding to the object selected as the processing target in step S406 in the image data (2), and advances the control to step S412.

  In step S412, the CPU 10 cuts out the image data of the area acquired in step S410 from the image data (2), and advances the control to step S414. The CPU 10 obtains the image data by, for example, cutting out image data in the range of objects to be processed in the object list (2) from image information (for example, bitmap data) displayed by the image data (2). To do.

  In step S414, the CPU 10 compares the image data cut out in step S408 (hereinafter, “cutout data (1)”) with the image data cut out in step S412 (hereinafter, “cutout data (2)”). Then, the comparison result is stored in the RAM 12, and the control is advanced to step S416. In step S414, for example, for all coordinates in the cutout data (1), the CPU 10 determines that the pixel value of each coordinate in the cutout data (1) is the pixel value of the corresponding coordinate in the cutout data (2). Determine whether they match. For the cut-out data (2), for example, the aspect ratio of the cut-out data (1) is changed according to the aspect ratio of the operation screen specified by the image data (2) and the operation screen specified by the image data (1). It corresponds to the data transformed as described. Thereby, the coordinates in the cut-out data (2) correspond to the coordinates where the coordinates are located in the cut-out data (1) before the deformation.

  The comparison result stored in step S414 may be the number of coordinates (pixels) having the same pixel value. The number of pixels corresponds to a “match rate” described later. In addition, the comparison result stored in step S414 may be the number of coordinates where the pixel values do not match, or may be information indicating whether the object as a whole matches or does not match. More specifically, for example, when the pixel values of the pixels of a specific ratio or more out of all the pixels match for the object to be processed, the CPU 10 displays the comparison result of “match” as the whole object as the pixel value. If the number of matched pixels is less than a specific ratio with respect to the total number of pixels, the comparison result of “mismatch” is stored.

  In step S416, the CPU 10 determines whether or not the object to be processed is the last object in the object list. If it is determined that the object to be processed is the last object, the control proceeds to step S418. On the other hand, if it is determined that the object to be processed is not the last object, control is returned to step S408.

  In step S418, the CPU 10 selects the object listed next to the current processing target object in the object list as the processing target object, and returns the control to FIG.

  As described above, according to the processing described with reference to FIG. 11, the pixel values of the pixels of the image data (1) and the image data (2) are compared. The result of the comparison is stored for each object in step S414. In step S50 of FIG. 4, the CPU 10 determines whether or not there is a difference between the image data (1) and the image data (2) based on the result of these comparisons.

  Note that the determination in step S50 may be performed for each object. That is, it may be determined whether there is a difference for each object. An object determined to have a difference may be a target of processing after step S60, and an object determined to have no difference may not be a target of processing after step S60. In this case, in the image data (1), image data for correction is generated for a portion corresponding to an object determined to have a difference. In the image data (1), no correction image data is generated for a portion corresponding to an object that is determined to have no difference. The image data (3) can be generated by replacing the image data (1) with the image data for correction only in the portion where the image data for correction is generated.

<Specific Example of Generation of Image Data (3)>
12-14 is a figure for demonstrating the production | generation aspect of image data (3). First, the result of comparison for each object will be described with reference to FIG. FIG. 12 shows a first example of an image 900 which is an example of an image of a button 150 (see FIG. 8 or the like) included in the image data (1) and an image of the button 150 included in the image data (2). An image 911 and an image 912 that is a second example of the image of the button 150 included in the image data (2) are shown. In this example, it is assumed that the button 150 constitutes one object.

  As shown in FIG. 12, the image 900 includes a pictographic image showing a spread page of a book and a character string (character / photo print photograph). Compared to the image 900, in the images 911 and 912, the sharpness of the character string does not change, but the pictogram image is unclear. The pictographic image of the image 912 is less sharp than the pictographic image of the image 911.

  FIG. 12 further shows an example of the result of comparison according to the process of FIG. The coincidence rate obtained as a result of comparison between the image 911 and the image 900 is “40%”, and the coincidence rate obtained as a result of comparison between the image 912 and the image 900 is “20%”. In this example, if the matching degree is 40% or more, it is determined that the objects match, and if the matching degree is less than 40%, it is determined that the objects do not match. From this, it is determined that the image 911 matches the image 900, and the image 912 is determined not to match the image 900.

  When the image data (2) includes the image 912, correction image data is generated for the portion corresponding to the image 912 of the image data (1). Each of FIGS. 13 and 14 shows a specific example of image data for correction.

  FIG. 13 shows an image 913 based on image data for correction, which is generated by enlarging only the portion (graphic area) of the pictogram image included in the object from the original image (image 900). . FIG. 14 shows an image 914 using correction image data generated by enlarging both a pictogram part (graphic area) and a character area included in an object from an original image (image 900). It is shown. In the correction image data, only the character area may be enlarged and the graphic area may not be enlarged.

  Note that “enlargement” is just one specific example of improving visibility. The image data for correction may be generated by changing the original image data so as to improve the visibility by, for example, another method. As another method, for example, a character in a character area is made thin.

<Presence / absence of transmission of image data (3)>
As described with reference to FIG. 5, in this embodiment, if the difference between the image data (1) and the image data (2) is small, the image data (3) is not generated, but the difference is large. In this case, the image data (3) is generated and transmitted to the terminal device 200. The above difference may be determined for the entire image data (1) or for each object. When it is determined for each object, when it is determined that the difference is large for at least one object, the image data (3) can be generated.

  The determination for each object may be made only for some types of elements among the elements constituting the object. For example, only the elements in the graphic area may be determined. In this case, for example, the amount of difference (for example, the number of non-matching pixels) between the image data (1) and the image data (2) between the images of the character region elements is larger than a predetermined amount. However, if the amount of difference between the images of the elements in the graphic area is less than a predetermined amount (for example, if the “matching degree” is equal to or greater than a specific value), the difference is large for the object. Is not judged. For all objects, if the amount of difference between the elements in the graphic region is smaller than a predetermined amount, the image data ( 3) is not generated.

  FIG. 15 shows an image 950 which is an example of the operation screen. The image 950 includes six objects 951 to 956. Objects 951 to 956 include graphic areas 951A to 956A and character areas 951B to 956B, respectively.

  For the image 950, even if it is determined that the amount of difference between the images of the elements of the character area for the at least one object between the image data (1) and the image data (2) is larger than a predetermined amount, If it is determined that the amount of difference between the images of the graphic region elements is less than a predetermined amount for all objects, the image data (3) is not generated. For example, when the operation screen is displayed on the terminal device 200, there may be no problem as long as only the graphic area is recognized. In such a case, even if it is difficult to see the character region on the operation screen displayed on the terminal device 200, the image data (3) is not transmitted to the terminal device 200 unless the graphic region is difficult to see. Thereby, the transmission of data from MFP 100 to terminal device 200 can be minimized while minimizing the reduction in the visibility of the operation screen on terminal device 200. In this case, data comparison may be omitted for the character area.

  For the image 950, even if it is determined that the amount of difference between the images of the elements of the graphic region between the image data (1) and the image data (2) is larger than a predetermined amount for at least one object, The image data (3) may be configured not to be generated when it is determined that the amount of difference between the images of the elements of the character area between all the objects is smaller than a predetermined amount. For example, when an operation screen is displayed on the terminal device 200, there may be no problem as long as only a character area is recognized. In such a case, even if it is difficult to see the graphic area on the operation screen displayed on the terminal device 200, the image data (3) is not transmitted to the terminal device 200 unless the character area is difficult to see. Thereby, the transmission of data from MFP 100 to terminal device 200 can be minimized while minimizing the reduction in the visibility of the operation screen on terminal device 200. In this case, data comparison may be omitted for the graphic area.

  In the present embodiment, the match / mismatch (or the degree thereof) can be determined (determined) independently between the image data (1) and the image data (2) for the character area and the graphic area. . In the present embodiment, the image data (3) can be generated only for the area determined to be inconsistent based on the result of the determination (determination). For example, if it is determined that only a character area is inconsistent for an object, the image data (3) can be generated only by correcting the image data (1) for the character area. Further, if it is determined that only a graphic area of a certain object is inconsistent, the image data (1) can be generated only by correcting the image data (1) only for the graphic area.

  Match / mismatch determination for image regions can be realized, for example, by comparing pixel values of corresponding pixels.

  The determination of matching / non-matching of the character area is performed by, for example, developing the image data of each character area of the image data (1) and the image data (2) into image information such as bitmap data and the like. It may be performed by executing a character recognition process on the information and comparing the character information obtained as a recognition result. For example, when the obtained proportion of the number of characters is equal to or greater than a predetermined proportion, it is determined that the image of the character area matches between the image data (1) and the image data (2). On the other hand, when the obtained proportion of the number of characters is lower than a predetermined proportion, it is determined that the image of the character area does not coincide between the image data (1) and the image data (2). The

<Transmission request for image data (2)>
As described with reference to step S20 and the like in FIG. 5, the CPU 10 requests the terminal device 200 to transmit the image data (2). Such a request may be omitted. When the CPU 20 of the terminal device 200 receives the image data (1), the CPU 20 may generate the image data (2) and transmit the image data (2) to the MFP 100 without receiving a request from the MFP 100. obtain.

<Modified example of comparison target (specific display contents)>
In the present embodiment, the image data (1) and the image data (2) are compared, and it is determined whether or not they match (whether there is a difference), and it is determined that they do not match or there is a difference. Then, image data (3) was generated. The comparison target may be the entire area of the image data (1), only the character area, or only the graphic area. Further, the comparison target may be only a portion corresponding to specific display contents in the image data (1). The specific display content is, for example, a part for inputting an important instruction or a part for displaying important information on the operation screen.

  FIG. 16 is a diagram illustrating an example of the operation screen. As shown in FIG. 16, the operation screen 890 includes a start key 891, a reset key 892, and a stop key 893. These three keys are displayed in area 800A. The operation screen 890 includes a display field 800B that displays the number of copies.

  When the CPU 10 compares the image data (1) and the image data (2) for the operation screen 890 in step S40 (see FIG. 5), the start key 891, the reset key 892, and the stop key in the image data (1). Only the area corresponding to 893 and the display field 800B may be compared.

  In such a modification, when the operation screen is displayed according to the image data (2) in the terminal device 200, when these four elements are considered to be displayed unclearly, these four elements are Image data (3) to be displayed clearly is generated and transmitted to the terminal device 200. However, for elements other than these four elements, transmission of the image data (3) to the terminal device 200 is not performed even when it is considered that the display according to the image data (2) is unclear. Not done. Thereby, the frequency | count of transmission to the terminal device 200 of image data (3) can be restrained to a required minimum.

  FIG. 17 is a diagram showing functional blocks of MFP 100 of such a modification. As shown in FIG. 17, a specific display content storage unit 160 is set in the HD 16. The specific display content storage unit 160 stores information for specifying the object to be compared in step S40 among the objects and / or elements included in the image data (1). Based on the information stored in the specific display content storage unit 160, the comparison unit 111 identifies a comparison target in step S40 and performs comparison.

<Modifications for comparison (changes in history)>
Still another modified example to be compared in step S40 will be described. This modification is intended for the case where the image data newly transmitted to the terminal device 200 is image data of an operation screen obtained by changing a part of the operation screen to which image data has been previously transmitted. With reference to FIG. 18, a comparison target on the operation screen in this modification will be described. For convenience, the previous operation screen is referred to as a “first operation screen”, and the current operation screen is referred to as a “second operation screen”.

  FIG. 18 shows “transmission of image data for the first operation screen” and “transmission of image data for the second operation screen”. “Transmission of image data for the first operation screen” shows image data 700, 710, and 720 for the first operation screen. The image data 700 is image data (1). The image data 710 is image data (2). The image data 720 is image data (3).

  “Transmission of image data for the second operation screen” shows image data 800, 810, and 820 for the second operation screen. The image data 800 is image data (1). The image data 810 is image data (2). The image data 820 is image data (3).

  A portion 801 indicates a changed part from the image data 700 in the image data 800. For example, the second operation screen is a screen in which the instruction content displayed as the number of copies is changed from the first operation screen.

  In this case, in step S40, only the portion 801 in the image data 800 is to be compared. A portion 811 is a portion corresponding to the portion 801 in the image data 810. In step S40, the image data of the portion 801 and the image data of the portion 811 are compared.

  In step S60 (see FIG. 5), the CPU 10 generates image data of the portion 821 as necessary. The part 821 is image data of a part corresponding to the part 801 (part 811) in the image data 820 in the terminal device 200, and is image data for displaying the image of the part 811 clearly. Then, the CPU 10 generates image data 820 by combining the image data of the portion 821 and the image data of a portion other than the portion corresponding to the portion 821 of the image data 720.

  The image data of the part 822 is image data of a part that has been determined that no change is necessary in the transmission of the image data to the terminal device 200 so far. In this modification, portions that are already determined to be unnecessary are excluded from the comparison determination. Thereby, the area to be compared in step S40 can be minimized, and the operation screen can be appropriately displayed on the terminal device 200.

<Instruction for terminal device>
The MFP 100 generates image data (1) for the operation screen of the MFP 100, transmits the image data (1) to the terminal device 200, and then generates image data (3) as necessary. (3) is transmitted to the terminal device 200. In the present embodiment, MFP 100 transmits an instruction not to display image data (1) to terminal device 200 until it is determined that it is not necessary to replace image data (1) with image data (3). can do.

  FIG. 19 is a functional block diagram of a modified example of such MFP 100. As illustrated in FIG. 19, the transmission unit 103 includes a fourth transmission unit 121 for transmitting the instruction to the terminal device 200.

  Also in this modification, the CPU 10 transmits the image data (1) to the terminal device 200 in step S10. The terminal device 200 generates image data (2). However, terminal device 200 does not display the operation screen according to the image data (2) until an instruction is received from MFP 100. Then, after comparing the image data (1) with the image data (2) in step S40 (see FIG. 5), the CPU 10 determines that there is no difference between the two image data in step S50. Information that permits display of the image data (1) is transmitted. In response to receiving the information, the terminal device 200 displays an operation screen according to the image data (2) on the operation panel 25.

  On the other hand, when determining that there is a difference between the two image data in step S50, the CPU 10 generates the image data (3) in step S60, and transmits the image data (3) to the terminal device 200 in step S70. In this modification, the CPU 10 further transmits an operation screen display instruction based on the image data (3) in step S70. When the terminal device 200 receives the instruction, the terminal device 200 displays an operation screen according to the image data obtained by performing the display process on the image data (3).

  In this modification, when the image data (3) is transmitted from the MFP 100 to the terminal device 200, the operation screen according to the image data (2) is not displayed on the terminal device 200. Thereby, it can be avoided that the terminal device 200 performs processing for displaying the operation screen unnecessarily.

  In the present embodiment described above, the image processing apparatus once generates image data of the operation screen as the first data. Then, the image processing device compares the second data indicating how the first data is displayed on the terminal device with the original first data. Then, the image processing device generates image data of the operation screen as third data based on the comparison result, and transmits the image data to the terminal device. Thereby, the image processing apparatus can transmit the image data of the operation screen generated in consideration of the result of the comparison to the terminal device.

  It should be thought that embodiment disclosed this time and its modification are illustrations in all the points, and are not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims. It is intended that the techniques disclosed in the embodiments and the modifications thereof can be implemented alone or in combination as much as possible.

  10, 20 CPU, 11, 21 ROM, 12, 22 RAM, 13 scanner, 14 printer, 15, 25 operation panel, 17, 27 network controller, 100 MFP, 101 instruction input unit, 102 update unit, 103, 108 transmission unit 104 First transmission unit 106 Second transmission unit 107 Generation unit 109 Determination unit 110 Acquisition unit 111 Comparison unit 112 Region determination unit 113 Character recognition unit 114 Extraction unit 120 Third transmission unit 121 4th transmission part, 130,140,150 button, 151,154 area | region, 152,951A-956A figure area | region, 153,154,951B-956B character area | region, 160 specific display content memory | storage part, 161 1st operation screen memory | storage part, 170, 800B display field, 200 terminal device, 300 network Work, 700,710,720,800,810,820 image data, 890 operation screen, 891 start key, 892 reset key, 893 stop key.

Claims (14)

An image processing device capable of communicating with a terminal device,
First transmission means configured to transmit first data that is image data of an operation screen for combining and displaying an image and characters to the terminal device;
Receiving means configured to receive second data, which is screen data for displaying the first data in the terminal device, from the terminal device;
Based on a comparison result between the first data and the second data, third data that is image data of an operation screen for combining and displaying the image and the character is generated, and the third data is generated. An image processing apparatus comprising: a second transmission unit configured to transmit the data to the terminal device.   When the difference between the image displayed by the first data and the image displayed by the second data is greater than or equal to a predetermined amount, the second transmitting unit transmits the third data. The image processing apparatus according to claim 1, wherein the image processing apparatus is configured to generate the image processing apparatus.   The comparison result includes a comparison result between portions corresponding to the graphic area on the operation screen and a comparison result between portions corresponding to the character area on the operation screen of the first data and the second data. The image processing apparatus according to claim 2, comprising at least one.   The image processing apparatus according to claim 3, wherein a comparison result between portions corresponding to character areas on the operation screen is represented by the number of matching characters by using character recognition processing.   The image processing apparatus according to claim 3, wherein a comparison result between portions corresponding to the graphic area on the operation screen is represented by the number of pixels having the matched information.   The image processing apparatus according to claim 1, wherein the comparison result is a comparison result between portions corresponding to specific display contents on the operation screen.   The specific display content is a display about an image processing operation on the image processing apparatus, and a start key for inputting an instruction to start the image processing operation and an instruction content of the number of copies for the image processing operation are displayed. At least one of a display area for resetting, a reset key for instructing reset of setting contents for the image processing operation, and a stop key for inputting an instruction to stop the image processing operation The image processing apparatus according to claim 6, further comprising: The comparison result includes a comparison result between portions corresponding to the graphic area on the operation screen and a comparison result between portions corresponding to the character area on the operation screen of the first data and the second data. Including
In the case where the amount of difference between the portions of the first data and the second data corresponding to the character area on the operation screen is smaller than a predetermined amount, The generation of the third data is not performed even if the amount of difference between portions corresponding to the graphic area on the operation screen exceeds a predetermined amount. The image processing apparatus according to any one of the above. The comparison result includes a comparison result between portions corresponding to the graphic area on the operation screen and a comparison result between portions corresponding to the character area on the operation screen of the first data and the second data. Including
When the amount of difference between the portions of the first data and the second data corresponding to the graphic area on the operation screen is smaller than a predetermined amount, the second transmission unit is
The generation of the third data is not performed even if the amount of difference between portions corresponding to the character area on the operation screen exceeds a predetermined amount. The image processing apparatus according to any one of the above. The comparison result is obtained when the first transmission unit transmits the first data on the second operation screen in which a part of the first operation screen is changed. It is a comparison result for the part corresponding to the part,
The second transmission means includes a portion corresponding to a portion other than the portion in the history of the third data for the first operation screen, and the portion of the second operation screen based on the comparison result. 10. The image processing apparatus according to claim 1, wherein the third data for the second operation screen is generated by combining a portion generated for. For the previous SL terminal device transmits information for requesting transmission of the second data, further comprising a third transmission means, the image processing apparatus according to any one of claims 1 to 10 .   A fourth transmission unit configured to transmit the information prohibiting the display of the first data and the information permitting the display of the third data to the terminal device; The image processing apparatus according to claim 1. A method in which an image processing device transmits image data to a terminal device,
Transmitting first data, which is image data of an operation screen for combining and displaying an image and characters, to the terminal device;
Receiving, from the terminal device, second data that is data of screen information for displaying the first data in the terminal device;
Comparing the first data and the second data;
Generating third data which is image data of an operation screen for combining and displaying the image and the character based on a result of comparison in the comparing step;
Transmitting the third data to the terminal device, wherein the image processing device transmits the image data.   A program for causing a computer to implement the method according to claim 13.