JP6766658B2 - Image processing device, input pattern confirmation method, and computer program - Google Patents

Description

The present invention relates to a technique for confirming a gesture when a user inputs a pattern such as a character by a gesture.

Conventionally, image processing devices called multifunction devices, MFPs (Multi Function Peripherals), image forming devices, and the like have become widespread. The image processing device is equipped with a touch panel display.

The image processing device displays a screen for setting job conditions on the touch panel display. Then, the condition is accepted by detecting the gesture performed by the user with the finger.

If the image processor is equipped with a mouse, the user can make gestures by moving the pointer with the mouse. The following technologies have been proposed as such technologies.

When the gesture operation operated while the pointer is dragged corresponds to a predetermined operation held in advance, the information processing system sets predetermined processing conditions according to the predetermined operation. .. For example, the user gestures the pointer on the print setting screen by using a pointing device such as a mouse. The locus of the pointer is, for example, a predetermined operation of setting a predetermined paper orientation to landscape orientation. The graphical user interface processing unit of the information processing system determines that the operation of the pointer is the landscape setting of the paper, and sets the paper orientation to be landscape under the printing processing conditions (Patent Document 1).

In addition, the following techniques have been proposed as techniques for setting conditions by gestures.

The image processing device is a recognition means that recognizes the locus of coordinate information input by the user via the operation unit as a gesture, and a setting means that sets for processing image data according to the gesture recognized by the recognition means. The setting means sets differently depending on the position of the gesture (Patent Document 2).

The display control device displays an image on a touch panel type display screen, and displays information about the image together with the image. The display control device displays a display screen when an image is displayed on the display screen and the locus of the touch input detected in response to the touch input on the display screen is a locus that can be a part of the gesture operation. In addition to hiding the information displayed in, the guidance regarding the gesture operation candidate, which is a gesture operation that can be confirmed by continuing the detected touch input among the plurality of gesture operations, is displayed on the display screen (Patent Document). 3).

Japanese Unexamined Patent Publication No. 2006-99389 Japanese Unexamined Patent Publication No. 2011-138237 Japanese Unexamined Patent Publication No. 2013-178847

According to the prior art, as described above, the user can input the processing conditions by performing a gesture of drawing a pattern such as letters or numbers on the touch panel display or the like.

However, the gesture of drawing a pattern is not as simple as the gesture of tapping or flicking a prepared button.

In other words, it is difficult to accurately perform the gesture of drawing a pattern. Therefore, according to the conventional method, if the pattern drawn by the user is similar to the model to some extent, it is considered that the pattern is input according to the model.

However, according to the conventional method, the pattern drawn by the user may be mistakenly recognized as a pattern different from the intention of the user. For example, a user may have intended to write the number "2" but mistakenly recognize it as the letter "Z".

Then, the conditions are not set as desired by the user. As a result, for example, the desired printed matter may not be obtained, a higher usage fee may be charged, or information may be transmitted to an incorrect destination.

Therefore, before starting the processing, it is conceivable to display and confirm what kind of pattern the pattern drawn by the user is recognized by the image processing apparatus.

However, according to this method, the process may be started without sufficient confirmation by the user.

In view of such problems, it is an object of the present invention to make the user confirm the conditions specified by the user by the gesture of drawing a pattern more effectively than before.

The image processing device according to another embodiment of the present invention is an image processing device that executes image-related processing, which is processing for an image, according to an input pattern input by a user of a plurality of patterns by tracing a touch panel display. The storage means and the input pattern for storing each of the plurality of conditions of the image-related processing in association with any one of the plurality of patterns and any of the plurality of areas in the touch panel display. A similar pattern similar to is included in the plurality of patterns, and among the plurality of areas, the area associated with the similar pattern and any of the plurality of conditions in the storage means When the input pattern on the touch panel display is within the input area or a predetermined range surrounding the input area, the input pattern corresponds to the similar pattern and the input pattern among the plurality of conditions. The input pattern is displayed on the touch panel display together with the conditions to be performed and the conditions corresponding to the similar pattern, and the input pattern is associated with two or more of the plurality of conditions in the storage means, and the two or more conditions are associated with each other. It has a display control means for displaying the input pattern on the touch panel display together with the two or more conditions when the areas associated with each of the conditions are different.

According to the present invention, it is possible to make the user confirm the conditions specified by the user more effectively than before by the gesture of drawing the pattern.

It is a figure which shows the example of the overall structure of the network system including the image forming apparatus. It is a figure which shows the example of the hardware composition of the image forming apparatus. It is a figure which shows the example of the functional structure of an image forming apparatus. It is a figure which shows the example of a copy job screen and an area. It is a figure which shows the example of the gesture table used. It is a figure which shows the example of the similar data. It is a figure which shows the example of the condition data. It is a figure which shows the example of the handwriting gesture. It is a figure for demonstrating an example of the method of determining a gesture position. It is a flowchart explaining the example of the flow of the erroneous recognition presence / absence determination processing. It is a flowchart explaining the example of the flow of confirmation time determination processing. It is a figure which shows the 1st example of the condition confirmation screen. It is a figure which shows the 2nd example of the condition confirmation screen. It is a flowchart explaining an example of the flow of anomaly detection processing. It is a flowchart explaining an example of the overall processing flow of an image forming apparatus. It is a flowchart explaining an example of the overall processing flow of an image forming apparatus.

FIG. 1 is a diagram showing an example of an overall configuration of a network system including an image forming apparatus 1. FIG. 2 is a diagram showing an example of the hardware configuration of the image forming apparatus 1.

The image forming apparatus 1 shown in FIG. 1 is an image processing apparatus generally called a “multifunction device” or an “MFP (Multi Function Peripherals)”, and integrates functions such as copying, PC printing, fax, scanner, and box. It is a device that has been used.

The PC print function is a function of printing an image on paper based on image data received from the terminal device 2 described later.

The box function is a function in which a storage area called a "box" or a "personal box" is provided to each user, and each user saves and manages image data or the like according to his / her own storage area. A box corresponds to a "folder" or "directory" on a personal computer.

The image forming apparatus 1 communicates with the terminal apparatus 2 and the like via the communication line 3. As the communication line 3, an Internet, a LAN (Local Area Network) line, a public line, or the like is used. As the terminal device 2, a personal computer, a tablet computer, a smartphone, or the like is used.

As shown in FIG. 2, the image forming apparatus 1 includes a CPU (Central Processing Unit) 10a, a RAM (Random Access Memory) 10b, a ROM (Read Only Memory) 10c, an auxiliary storage device 10d, a touch panel display 10e, and an operation key panel 10f. , NIC (Network Interface Card) 10g, modem 10h, scan unit 10i, print unit 10j, finisher 10k and the like.

The touch panel display 10e displays a screen showing a message to the user, a screen for the user to input a command or information, a screen showing the result of the process executed by the CPU 10a, and the like. Further, the touch panel display 10e sends a signal indicating the touched position to the CPU 10a.

The operation key panel 10f is a so-called hardware keyboard, and is composed of a numeric keypad, a start key, a stop key, a function key, and the like.

The NIC 10g communicates with other devices by a protocol such as TCP / IP (Transmission Control Protocol / Internet Protocol).

The modem 10h exchanges image data with a fax terminal by a protocol such as G3.

The scan unit 10i reads the image written on the sheet set on the platen glass and generates image data.

In addition to the image read by the scan unit 10i, the print unit 10j prints an image received from another device by the NIC 10g or the modem 10h on paper.

The finisher 10k performs post-processing on the printed matter obtained by the print unit 10j, if necessary. Post-treatment includes staple binding, punching, and bending.

The ROM 10c or the auxiliary storage device 10d stores programs for realizing the above-mentioned functions such as copying, PC printing, and faxing. These programs are loaded into the RAM 10b as needed and executed by the CPU 10a. As the auxiliary storage device 10d, a hard disk drive, SSD (Solid State Drive), or the like is used.

Hereinafter, a case where the copy job program 10P is stored as one of these programs will be described as an example.

The user can make the image forming apparatus 1 execute the copy job by the copy job program 10P. The copy job mainly consists of a phase of reading an image from the original paper (scan phase), a phase of correcting the read image (correction phase), and a phase of printing the corrected image on a blank sheet of paper and performing post-processing as necessary. It consists of (printing phase).

Further, according to the copy job program 10P, the user can specify the conditions of various items of the copy job by performing a gesture.

The “gesture” is an action performed by touching the touch surface (display surface) of the touch panel display 10e with a finger and moving the finger as appropriate. Examples of gestures include tap, double tap, flick, pinch in, pinch out, and hold (hold).

In addition, the action of writing a pattern on the touch surface with a finger also corresponds to a gesture. Hereinafter, this gesture will be referred to as a "handwritten gesture". In addition, the case of writing symbols such as numbers, letters, or symbols as a pattern will be described as an example.

According to the copy job program 10P, it is possible to reduce the number of unintended conditions specified by the user when using the handwritten gesture. This mechanism will be described below.

FIG. 3 is a diagram showing an example of the functional configuration of the image forming apparatus 1. FIG. 4 is a diagram showing an example of the copy job screen 6A and the area 6B. FIG. 5 is a diagram showing an example of the gesture table 5A used. FIG. 6 is a diagram showing an example of similar data 5C. FIG. 7 is a diagram showing an example of the condition data 5E. FIG. 8 is a diagram showing an example of a handwritten gesture. FIG. 9 is a diagram for explaining an example of a method of determining the gesture position. FIG. 10 is a flowchart illustrating an example of the flow of the erroneous recognition presence / absence determination process. FIG. 11 is a flowchart illustrating an example of the flow of the confirmation time determination process. FIG. 12 is a diagram showing a first example of the condition confirmation screen 6C. FIG. 13 is a diagram showing a second example of the condition confirmation screen 6C. FIG. 14 is a flowchart illustrating an example of the flow of the abnormality detection process.

According to the copy job program 10P, the gesture storage unit 101 used, the similarity symbol storage unit 102, the flag storage unit 103, the condition storage unit 104, the copy job screen display unit 121, the gesture determination unit 122, and the position determination unit 123, as shown in FIG. False recognition gesture presence / absence determination unit 124, confirmation time determination unit 125, first condition change unit 126, confirmation screen display unit 127, second condition change unit 128, scan control unit 131, image correction unit 132, print control unit 133. , And the abnormal image detection unit 134 and the like are realized.

In the used gesture storage unit 101, the used gesture table 5A of the screen to be displayed during the execution of the copy job program 10P is stored in advance. When there are a plurality of screens to be displayed, one gesture table 5A is stored for each screen.

Hereinafter, as one of such screens, a case where the copy job screen 6A shown in FIG. 4A is displayed will be described as an example. The copy job screen 6A is a screen for the user to specify the conditions of the copy job.

The used gesture table 5A1 as shown in FIG. 5 is stored in the used gesture storage unit 101 as the used gesture table 5A of the copy job screen 6A.

The gesture table 5A1 used is applied to each handwritten gesture used on the copy job screen 6A, the symbol represented by the handwritten gesture, the identifier of the area 6B in which the handwritten gesture is valid, and when the handwritten gesture is performed. Use gesture data 5B indicating a condition (item and its value) is stored.

The area 6B is an area in which the copy job screen 6A is divided into a plurality of areas. In the present embodiment, the copy job screen 6A is divided into 3 × 3 areas 6B as shown in FIG. 4B, and is given a two-dimensional address. This address is used as an identifier for area 6B.

Hereinafter, the area 6B of (x, y) may be described as "area 6Bxy" in order to distinguish it from other areas 6B. For example, the area 6B of (2, 3) may be described as "area 6B23".

The same handwritten gesture may be used in multiple areas 6B. In this case, a plurality of used gesture data 5B are stored for the handwritten gesture. Also, the handwritten gesture may be used in these areas 6B for the same conditions, or in different areas 6B for different conditions.

For example, a handwritten gesture of writing the symbol "-" is used in areas 6B23 and 6B22 to specify different conditions.

Handwritten gestures that write the symbol (number) "2" are used to specify the same conditions in areas 6B12, 6B22, and 6B32, but in these three areas 6B12, 6B22, 6B32 and area 6B31. Used to specify different conditions.

As shown in FIG. 6, the similarity symbol storage unit 102 stores in advance similar data 5C indicating another symbol (similar symbol) similar to each symbol.

The flag storage unit 103 stores the existence flag 5D1, the direct confirmation flag 5D2, the start flag 5D3, the pre-printing flag 5D4, and the abnormality flag 5D5 in advance. The initial value of these flags is "0". The usage of these flags will be described later in sequence.

As shown in FIG. 7, the condition storage unit 104 stores the item, the default condition, the specified condition, and the condition data 5E indicating suitability for each item of the copy job.

The "default condition" is a default (initial) condition and is a fixed value in principle, but can be changed as appropriate by the administrator or the user.

The "specified condition" is a condition specified by the user by a gesture or the like on the copy job screen 6A.

"Appropriateness" is whether or not the specified condition is applied instead of the default in the case of a copy job. "Appropriate" means that it applies, and "No" means that it does not apply.

Immediately after the copy job program 10P is started, nothing is shown in the "specified condition", and "no" is shown as "appropriate". The usage of the condition data 5E and the update of the specified conditions and suitability will be described later.

When the copy job screen 6A is displayed on the touch panel display 10e, the user can specify the copy job conditions by tapping a button arranged on the copy job screen 6A. Hereinafter, the processing when the user specifies the condition by the handwritten gesture will be described in particular.

The copy job screen display unit 121 displays the copy job screen 6A on the touch panel display 10e when a predetermined command is input or the copy job program 10P is started.

When the user makes a handwritten gesture on the touch panel display 10e while the copy job screen 6A is displayed, the gesture determination unit 122 and the position determination unit 123 perform the following processing.

The gesture discrimination unit 122 discriminates the symbol represented by the handwritten gesture performed by the user (hereinafter, referred to as “target gesture”) by a known method. For example, the symbol represented by the locus of the user tracing the touch panel display 10e with his / her finger is identified by the technique of OCR (Optical Character Recognition).

For example, when the user moves his / her finger from the left to the right on the touch surface of the touch panel display 10e as shown in FIG. 8, the gesture determination unit 122 is represented by the handwritten gesture performed by the user because it matches the symbol “-”. It is determined that the symbol has a code of "-".

The position determination unit 123 determines the area 6B where the target gesture is performed (hereinafter, referred to as “gesture position”). The target gesture may span multiple areas 6B. In such a case, the area 6B having the longest touched distance (distance to which the locus of the target gesture belongs) in the target gesture is determined to be the gesture position.

For example, the locus of the target gesture shown in FIG. 9 straddles the four areas 6B21, 6B31, 6B22, and 6B32, but the distance of the portion belonging to the area 6B22 is the longest. Therefore, the position determination unit 123 determines that the gesture position is the area 6B22.

When the symbol represented by the target gesture is determined by the gesture determination unit 122 and the gesture position is determined by the position determination unit 123, the misrecognition gesture presence / absence determination unit 124 determines whether or not the target gesture has a gesture that is easily misrecognized. The gesture is discriminated by the procedure shown in FIG.

The erroneous recognition gesture presence / absence determination unit 124 searches the use gesture table 5A on the current screen for the use gesture data 5B indicating the symbol determined by the gesture determination unit 122 (# 701 in FIG. 10). The "current screen" is a screen displayed on the touch panel display 10e when the target gesture is performed, and is the copy job screen 6A in this example.

When a plurality of used gesture data 5Bs are found (Yes in # 702), the erroneous recognition gesture presence / absence determination unit 124 indicates the identifier of the gesture position determined by the position determination unit 123 from the used gesture data 5B. Gesture data 5B (hereinafter referred to as "target area data") and used gesture data 5B (hereinafter referred to as "adjacent area data") indicating an identifier of an adjacent area are extracted (# 703). The "adjacent area" is an area 6B adjacent to the gesture position. In the present embodiment, eight adjacent areas 6B so as to surround the gesture position are adjacent areas.

Then, when the condition shown in the extracted target area data and the condition shown in any of the extracted adjacent area data are different (Yes in # 704), the misrecognition gesture presence / absence determination unit 124 is likely to be misrecognized. It is determined that there is, and the existence flag 5D1 is updated to "ON" (# 709).

That is, the misrecognition gesture presence / absence determination unit 124 determines whether or not the target gesture can be used in any adjacent area to specify a condition different from the condition at the gesture position, and when it can be used, The existence flag 5D1 is updated to "ON".

If all the conditions shown in the extracted adjacent area data are the same as the conditions shown in the extracted target area data (No in # 704), the misrecognition gesture presence / absence determination unit 124 is similar to the symbol represented by the target gesture. Similar data 5C indicating the symbol to be used as a similar symbol is searched from the similar symbol storage unit 102 (# 705).

When such similar data 5C is found (Yes in # 706), the misrecognition gesture presence / absence determination unit 124 determines whether or not the handwritten gesture of the similar symbol shown in the found similar data 5C is used in the current screen. , Check area 6B where the handwritten gesture is used (# 707). If the used gesture data 5B whose similar symbol is shown as a "symbol" is stored in the used gesture table 5A (in this example, the used gesture table 5A1) on the current screen, it is determined that the handwritten gesture is used. be able to. Further, the area 6B in which the handwritten gesture is used can be specified by the identifier shown in the used gesture data 5B.

If the handwritten gesture is used on the current screen and the area 6B in which the handwritten gesture is used is a gesture position or an adjacent area (Yes in # 708), the misrecognition gesture presence / absence determination unit 124 is misrecognized. It is determined that there is an easy gesture, and the existence flag 5D1 is updated to "ON" (# 709). In addition, it is determined that the handwritten gesture is likely to be erroneously recognized as the target gesture.

If this is not the case (No in # 708), the misrecognition gesture presence / absence determination unit 124 determines that there is no gesture that is likely to be misrecognized (# 710). In this case, the existence flag 5D1 is left as "OFF". The same applies when none of the gesture data 5B used in step # 701 is found and when only one gesture data 5B is found (No in # 702).

Returning to FIG. 3, the confirmation time determination unit 125 determines the time when the user should confirm the target gesture (hereinafter, referred to as “confirmation time”) by the procedure shown in FIG.

The existence flag 5D1 is updated to "ON" by the processing of the erroneous recognition gesture presence / absence determination unit 124 accompanying the target gesture (Yes in # 721 of FIG. 11), and the condition corresponding to the target gesture (shown in the target area data). If (condition) is a condition related to a predetermined item (Yes in # 722), the confirmation time determination unit 125 determines the gesture confirmation time as "immediately" and updates the direct confirmation flag 5D2 to "ON" (Yes). # 726).

The predetermined items can be arbitrarily set by the administrator of the image forming apparatus 1. For example, when the image forming apparatus 1 is used in an environment in which the usage fee is charged to the user, items that affect the usage fee (aggregate printing, double-sided printing, color mode, etc.) are set as predetermined items. ..

The existence flag 5D1 was updated to "ON" along with the target gesture (Yes in # 721), but if the condition corresponding to the target gesture is not a condition related to a predetermined item (No in # 722), the confirmation time determination unit 125 is determined as follows according to the presence or absence of other items (hereinafter, referred to as “related items”) related to the item of the condition.

If there is a related item (Yes at # 723), if the user has already specified a condition for the related item (Yes at # 725), the confirmation time determination unit 125 determines that the gesture confirmation time is "immediately". Then, the direct confirmation flag 5D2 is updated to "ON" (# 726). If the user has not yet specified the conditions for any of the related items (No in # 725), it is determined that the gesture confirmation time is just before the copy job is executed, and the start flag 5D3 is updated to "ON". (# 729).

If there is no related item (No in # 723), the confirmation time determination unit 125 prints the gesture confirmation time if the item of the condition is related to a phase after the scan phase (Yes in # 727). It is determined that it is immediately before the printing, and the pre-printing flag 5D4 is updated to "ON" (# 729). If it is related to the scan phase (No at # 727), the start flag 5D3 is updated to "ON" (# 728).

If the existence flag 5D1 is not updated to "ON" (No in # 721) and there is a related item (Yes in # 724), the confirmation time determination unit 125 sets the condition for any related item by the user. If it has already been specified (Yes at # 725), the gesture confirmation time is determined to be "immediately", and the direct confirmation flag 5D2 is updated to "ON" (# 729). If not (No in # 725), it is determined that the gesture confirmation time is just before the copy job is executed, and the start flag 5D3 is updated to "ON" (# 728).

The confirmation time determination unit 125 resets the existence flag 5D1 to "OFF" after the above determination process.

Which item is related to which item may be defined in the table in advance. For example, a plurality of items related to the same phase may be set as items related to each other.

Alternatively, associate specific items with each other. For example, associate the upper left single staple with the enlargement. Correspondence between enlargement and aggregate printing. Correspondence between aggregate printing and image shift. Correspond the paper size and enlargement. Associate image shift with paper size.

The first condition changing unit 126 updates the condition data 5E as follows so that the condition corresponding to the target gesture is reflected.

The first condition changing unit 126 searches the condition data 5E of the condition item shown in the target area data from the condition storage unit 104 (see FIG. 7). Then, the value of the "designated condition" of the condition data 5E is rewritten to the condition corresponding to the target gesture (condition shown in the target area data).

When it is determined by the erroneous recognition gesture presence / absence determination unit 124 that the target gesture has a gesture that is easily erroneously recognized, the first condition change unit 126 keeps the value of "appropriate" of the condition data 5E as "negative". Leave it to. Even when any of the direct confirmation flag 5D2, the start time flag 5D3, and the pre-printing flag 5D4 is turned “ON” by the processing of the confirmation time determination unit 125, it is left as “No”. In cases other than these, rewrite as "appropriate".

The confirmation screen display unit 127 displays the condition confirmation screen 6C as shown in FIG. 12 on the touch panel display 10e instead of the copy job screen 6A. The timing for displaying the condition confirmation screen 6C depends on the result of processing by the confirmation time determination unit 125, and is as follows.

When the direct confirmation flag 5D2 is updated to "ON", the confirmation screen display unit 127 immediately displays the condition confirmation screen 6C.

If the start flag 5D3 is "ON" when the command to start the copy job is given, the confirmation screen display unit 127 displays the condition confirmation screen 6C immediately before starting the copy job.

When the pre-print flag 5D4 is "ON", the confirmation screen display unit 127 displays the condition confirmation screen 6C immediately after the correction phase is completed and immediately before the start of the print phase.

The condition confirmation screen 6C is provided with a target gesture area 6C1, an same and similar gesture area 6C2, an apply button 6C3, a cancel button 6C4, and the like.

In the target gesture area 6C1, a locus representing the target gesture, that is, the user's handwriting 6C11 with respect to the touch panel display 10e is arranged. Further, a character string 6C12 representing the conditions shown in the target area data is arranged.

The same similarity symbol 6C21 and the character string 6C22 are arranged in the same similarity gesture region 6C2 as follows.

If the erroneous recognition gesture presence / absence determination unit 124 determines that the target gesture can be used in any adjacent area to specify a condition different from the condition at the gesture position, the symbol represented by the target gesture is the same similar symbol. It is arranged as 6C21. Further, a character string representing the conditions shown in the target area data is arranged as the character string 6C22.

If the handwritten gesture of a similar symbol is used on the current screen and the area 6B in which the handwritten gesture is used is determined by the misrecognition gesture presence / absence determination unit 124 to be the gesture position or the adjacent area, the similar symbol is used. It is arranged as the same similarity symbol 6C21. A character string representing the handwritten gesture and the condition corresponding to the area 6B is arranged as the character string 6C22.

By looking at the condition confirmation screen 6C, the user confirms whether or not the condition desired by the user can be specified by the target gesture. Then, if it can be specified, tap the apply button 6C3. If it cannot be specified, tap the cancel button 6C4.

When the apply button 6C3 or the cancel button 6C4 is tapped, the condition data 5E is updated as follows by the second condition change unit 128.

The second condition changing unit 128 searches the condition data 5E of the condition item shown in the target area data from the condition storage unit 104 (see FIG. 7).

Then, when the apply button 6C3 is tapped, the second condition changing unit 128 rewrites the value of "appropriate" of the condition data 5E to "appropriate". As a result, the specified conditions are applied in the copy job.

On the other hand, when the cancel button 6C4 is tapped, the second condition changing unit 128 clears the "designated condition" of the condition data 5E and returns "appropriate" to "negative".

Further, when either the apply button 6C3 or the cancel button 6C4 is tapped, the second condition change unit 128 sets the existence flag 5D1, the direct confirmation flag 5D2, the start flag 5D3, and the pre-print flag 5D4 to "OFF". Reset.

If the command to start the copy job has not been given yet, after the apply button 6C3 or the cancel button 6C4 is tapped, the copy job screen 6A is displayed by the copy job screen display unit 121 instead of the condition confirmation screen 6C. It is displayed on the touch panel display 10e again. Then, the user can continue to specify conditions for other items as appropriate.

At the timing of displaying the condition confirmation screen 6C, a plurality of condition data 5E in which the designated condition is indicated but the suitability is “No” may be stored in the condition storage unit 104. That is, there may be a plurality of target gestures that have been determined by the erroneous recognition gesture presence / absence determination unit 124 that the user needs to confirm, but have not yet been confirmed.

In such a case, the confirmation screen display unit 127 performs the above-described processing using each of these condition data 5E as the target area data. Then, the target gesture region 6C1 and the same similar gesture region 6C2 are arranged one by one on the condition confirmation screen 6C for each such target gesture. For example, when there are two condition data 5E as such condition data 5E, that is, when two unconfirmed target gestures remain, as shown in FIG. 13, the condition confirmation screen 6C has the same similarity as the target gesture area 6C1. Two gesture regions 6C2 are arranged.

Then, when the apply button 6C3 or the cancel button 6C4 is tapped, the second condition change unit 128 performs the above-mentioned processing for each of these condition data 5E.

The user can specify the conditions for each item by tapping or double-tapping an object such as a button arranged on the copy job screen 6A as in the conventional case.

In this case, the first condition changing unit 126 specifies the item and the condition according to the operation of the user, and rewrites the designated condition of the condition data 5E of the item to the specified condition. Furthermore, the suitability is rewritten as "suitable".

After specifying the conditions of the copy job, the user gives a command to start the copy job to the image forming apparatus 1 by performing a predetermined operation. For example, press the start key in the operation key panel 10f.

When the start flag 5D3 is "ON", the confirmation screen display unit 127 causes the condition confirmation screen 6C to be displayed on the touch panel display 10e before starting the copy job. As described above, the condition confirmation screen 6C is displayed based on the condition data 5E in which the designated condition is shown at this time but the suitability is “No”.

Then, when the apply button 6C3 or the cancel button 6C4 is tapped, these condition data 5E are updated by the second condition change unit 128 as described above.

If the cancel button 6C4 is tapped, the copy job is canceled. When the apply button 6C3 is tapped, the scan control unit 131 and the print control unit 133 execute the copy job as follows.

The scan control unit 131 causes the scan unit 10i to execute the scan phase process, that is, the process of reading the image written on the loaded paper. At this time, the condition data 5E of each item related to the scan is read from the condition storage unit 104. Then, the specified condition is applied to the item whose suitability is "appropriate", and the default condition is applied to the item whose suitability is "appropriate" to execute the scan phase process. Hereinafter, the scanned image will be referred to as “scanned image 5F1”.

The image correction unit 132 performs correction phase processing, that is, processing for correcting the scanned image 5F1 for printing. For example, predetermined processing such as tilt correction and top-bottom correction is appropriately applied to the scanned image 5F1.

Further, the image correction unit 132 reads the condition data 5E of each item related to the image correction from the condition storage unit 104. Then, regardless of whether the suitability is "suitable" or "bad", if the specified condition is set, it is applied, and if it is not set, the default condition is applied, and the correction phase is started. Let the process be executed. Even if it is "No", the designated condition is applied for the abnormality detection process described later by the abnormality image detection unit 134.

For example, the image correction unit 132 appropriately performs processing such as resizing (enlargement or reduction), aggregation of a plurality of pages, or color adjustment. Hereinafter, the corrected image will be referred to as "corrected image 5F2".

If the pre-print flag 5D4 is "OFF" when the correction phase processing is completed, the print control unit 133 skips the abnormality detection process described later and prints the print phase process, that is, the correction image 5F2 on paper. The print unit 10j is made to execute the processing to be performed.

On the other hand, when the pre-print flag 5D4 is "ON", the following processing is performed by the abnormal image detection unit 134, the confirmation screen display unit 127, and the like before the processing of the printing phase.

The abnormality image detection unit 134 detects whether or not there is a possibility that an abnormality has occurred in the corrected image 5F2 by the procedure shown in FIG.

The abnormal image detection unit 134 specifies a printable area (printable area) on the paper used in the printing phase (# 731 in FIG. 14). The printable area may be specified by the size of the paper and the specifications of the print unit 10j. The size of this paper may be specified by the condition data 5E regarding the printing destination paper.

Further, the abnormal image detection unit 134 specifies a rectangular area (print area) inscribed in the portion of the paper on which the corrected image 5F2 is printed (# 732). The print area may be specified by the condition data 5E regarding the shift (movement) of the image.

Then, when even a part of the specified print area extends beyond the specified printable area (Yes at # 733), the abnormality image detection unit 134 updates the abnormality flag 5D5 to “ON” (# 736).

If it does not protrude at all (No in # 734), the abnormal image detection unit 134 calculates the ratio of the diagonal length of the print area to the diagonal length of the printable area. Then, if the ratio is less than a predetermined value (for example, 0.87) (Yes at # 735), the abnormality flag 5D5 is updated to ON (# 736).

When the abnormality flag 5D5 is updated to "ON", the confirmation screen display unit 127 displays the condition confirmation screen 6C on the touch panel display 10e instead of the copy job screen 6A.

The confirmation screen display unit 127 may display the condition confirmation screen 6C regardless of whether the abnormality flag 5D5 is “ON” or “OFF”. However, in this case, if the abnormality flag 5D5 is "ON", it is desirable to arrange a message notifying that an abnormality may have occurred on the condition confirmation screen 6C.

Then, when the apply button 6C3 is tapped, the print control unit 133 controls the print unit 10j so that the corrected image 5F2 is printed. When the cancel button 6C4 is tapped, the copy job is canceled.

Further, when the apply button 6C3 or the cancel button 6C4 is tapped, the condition data 5E is updated by the second condition change unit 128 as described above.

After the printing phase is completed, the existence flag 5D1, the direct confirmation flag 5D2, the start flag 5D3, the pre-printing flag 5D4, and the abnormality flag 5D5 are reset to "OFF". Further, the scanned image 5F1 and the corrected image 5F2 are deleted.

Further, after the start or completion of the print phase, the copy job screen display unit 121 may display the copy job screen 6A on the touch panel display 10e.

15 to 16 are flowcharts illustrating an example of the overall processing flow of the image forming apparatus 1.

Next, the overall flow of processing during the copy job will be described with reference to the flowchart. The image forming apparatus 1 executes the process according to the copy job program 10P in the procedure shown in FIGS. 15 to 16.

The image forming apparatus 1 displays the copy job screen 6A after starting the copy job program 10P or after inputting a predetermined command (# 11 in FIG. 15).

Until the start key is pressed (No in # 12), the image forming apparatus 1 appropriately performs the processes of steps # 14 to # 25 as follows each time the user makes a gesture on the touch panel display 10e. Execute.

When the user makes a gesture (Yes at # 13), the image forming apparatus 1 determines the gesture position, the type of gesture, and the like (# 14, # 15). In particular, when the gesture is a handwritten gesture, the symbol represented by the gesture is also determined.

If the gesture is not a handwritten gesture (No in # 16), the image forming apparatus 1 writes the condition corresponding to the gesture to the corresponding condition data 5E as a designated condition, and updates the suitability to "suitable" (# 17). ).

When the gesture is a handwritten gesture (Yes in # 16), the image forming apparatus 1 determines whether or not the gesture (target gesture) has a gesture that is easily misrecognized (# 18). The procedure for discrimination is as described with reference to FIG. Furthermore, the gesture confirmation time is determined (# 19). The procedure for discrimination is as described with reference to FIG.

Further, the image forming apparatus 1 updates the condition data 5E so that the condition corresponding to the target gesture is reflected (# 20). If the target gesture is a gesture that is easily misrecognized, the suitability value is left as "No". Even if any of the direct confirmation flag 5D2, the start flag 5D3, and the pre-printing flag 5D4 is set to "ON" as a result of the process of step # 19, it is left as "No". In other cases, update to "appropriate".

When the direct confirmation flag 5D2 is turned “ON” (Yes at # 21), the image forming apparatus 1 displays the condition confirmation screen 6C instead of the copy job screen 6A (# 22). Further, the image forming apparatus 1 resets the existence flag 5D1, the direct confirmation flag 5D2, the start flag 5D3, and the pre-print flag 5D4 (# 23).

When the apply button 6C3 is tapped (Yes in # 24), the image forming apparatus 1 searches for the condition data 5E whose suitability is "No", and sets the value of the suitability of the found condition data 5E to "Appropriate". Rewrite (# 25). On the other hand, when the cancel button 6C4 is tapped, the specified condition of the condition data 5E is reset (erased) (# 26).

Then, the image forming apparatus 1 displays the copy job screen 6A again instead of the condition confirmation screen 6C (# 11).

When the user presses the start key (Yes at # 12), the image forming apparatus 1 replaces the copy job screen 6A with the condition when the start flag 5D3 is "ON" (Yes at # 27 in FIG. 16). The confirmation screen 6C is displayed (# 28). Further, the existence flag 5D1, the direct confirmation flag 5D2, the start flag 5D3, and the pre-print flag 5D4 are reset (# 29). Then, when the apply button 6C3 or the cancel button 6C4 is tapped, the condition data 5E is updated in the same manner as in steps # 24 to # 26 (# 30 to # 32).

When the cancel button 6C4 is tapped (No in # 30), the image forming apparatus 1 displays the copy job screen 6A instead of the condition confirmation screen 6C (# 11).

On the other hand, when the apply button 6C3 is tapped (Yes at # 30), the image forming apparatus 1 executes the scan phase based on the condition data 5E of each item related to scanning (# 33). As a result, the scanned image 5F1 is obtained. The same applies when the start flag 5D3 is "OFF" (No in # 27).

The image forming apparatus 1 performs the correction phase processing on the scanned image 5F1 based on the condition data 5E of each item related to the correction (# 34). As a result, the corrected image 5F2 is obtained.

When the pre-print flag 5D4 is "ON" (Yes at # 35), the image forming apparatus 1 detects whether or not there is a possibility that an abnormality has occurred in the corrected image 5F2 (# 36). The detection procedure is as previously described with reference to FIG.

As a result of the process of step # 37, when the abnormality flag 5D5 becomes “ON” (Yes at # 37), the image forming apparatus 1 displays the condition confirmation screen 6C instead of the copy job screen 6A (# 38). Further, the existence flag 5D1, the direct confirmation flag 5D2, the start flag 5D3, the pre-print flag 5D4, and the abnormality flag 5D5 are reset (# 39). Then, when the apply button 6C3 or the cancel button 6C4 is tapped, the condition data 5E is updated in the same manner as in steps # 24 to # 26 (# 40 to # 42).

When the cancel button 6C4 is tapped (No in # 40), the image forming apparatus 1 displays the copy job screen 6A instead of the condition confirmation screen 6C (# 11).

On the other hand, when the apply button 6C3 is tapped (Yes at # 40), the image forming apparatus 1 executes the printing phase based on the condition data 5E of each item related to printing (# 43). As a result, the corrected image 5F2 is printed on the paper. The same applies when the pre-print flag 5D4 is “OFF” (No in # 35).

When the print phase is started or completed, the image forming apparatus 1 may reset all the condition data 5E and the flags (existence flag 5D1 to error flag 5D5), return to step # 11, and display the copy job screen 6A. Good.

According to the present embodiment, it is possible to make the user confirm the conditions specified by the user more effectively than before by the gesture of drawing the pattern.

In the present embodiment, the case where the adjacent area is eight adjacent areas 6B so as to surround the gesture position has been described as an example, but the adjacent areas are four areas 6B on the top, bottom, left, and right adjacent to the gesture position. May be good.

In the present embodiment, the area 6B is an area obtained by dividing the copy job screen 6A into 3 × 3, but it may be an area other than the 3 × 3. For example, the region may be divided into 4 × 4 or 5 × 3.

Alternatively, if the used gesture data 5B in which the similar symbol is indicated as a "symbol" is stored in the used gesture table 5A on the current screen, the erroneous recognition gesture presence / absence determination unit 124 may determine the presence / absence of the similar symbol regardless of the handwritten gesture area 6B of the similar symbol. , The existence flag 5D1 may be updated to "ON".

In the present embodiment, the confirmation time determination unit 125 has set an item that affects the usage fee as a predetermined item. Then, when the item of the condition corresponding to the target gesture corresponds to a predetermined item, the direct confirmation flag 5D2 is updated to "ON".

However, the confirmation time determination unit 125 does not determine whether or not the item of the condition corresponding to the target gesture corresponds to the predetermined item, and calculates the usage fee before and after applying the condition. When the two are different, the direct confirmation flag 5D2 may be updated to "ON". The usage fee may be calculated from a table prepared in advance.

In the present embodiment, the user performs a handwritten gesture by tracing the touch panel display, but when the image forming apparatus 1 is provided with a pointing device such as a mouse or a trackball, the user operates the pointing device to move the mouse pointer. You may make a handwritten gesture by moving it.

In addition, the configuration of the entire image forming apparatus 1 or each part, the content of processing, the order of processing, the configuration of the screen, and the like can be appropriately changed according to the gist of the present invention.

1 Image forming device (image processing device)
101 Gesture storage unit used (memory means)
124 Misrecognition gesture presence / absence determination unit (selection means)
127 Confirmation screen display unit (display control means)
128 Second condition change department (reception means)
133 Print control unit (image-related processing means)
6C condition confirmation screen

Claims (12)

An image processing device that executes image-related processing, which is processing for an image, according to an input pattern input by a user among a plurality of patterns by tracing the touch panel display.
A storage means for storing each of the plurality of conditions of the image-related processing in association with any one of the plurality of patterns and one of the plurality of areas in the touch panel display.
A similar pattern similar to the input pattern is included in the plurality of patterns, and is associated with any of the plurality of conditions together with the similar pattern in the storage means in the plurality of areas. When the area is within the input area where the input pattern is input on the touch panel display or a predetermined range surrounding the input area, the input pattern is input to the similar pattern and the input among the plurality of conditions. The input pattern is displayed on the touch panel display together with the conditions corresponding to the pattern and the conditions corresponding to the similar pattern, and the input pattern is associated with two or more of the plurality of conditions in the storage means, and the 2 is A display control means for displaying the input pattern on the touch panel display together with the two or more conditions when the area associated with each of the two or more conditions is different.
An image processing device characterized by having. When the user performs a predetermined operation after the input pattern is displayed on the touch panel display, the input pattern and the corresponding condition corresponding to the input area among the plurality of conditions are accepted as designated conditions. Means and
An image-related processing means that executes the image-related processing based on the designated conditions, and
Have,
The image processing apparatus according to claim 1. The display control means displays the input pattern immediately after the input pattern is input.
The image processing apparatus according to claim 1 or 2. When the condition related to the corresponding condition among the plurality of conditions has already been accepted as the designated condition, the display control means inputs the input pattern by a command to start the image-related processing. After that, it is displayed before the image-related processing is started.
The image processing apparatus according to claim 2. The display control means displays the input pattern when the condition related to the corresponding condition among the plurality of conditions has already been accepted as the designated condition.
The image processing apparatus according to claim 2. The image-related process is a process of reading the image from a sheet of paper and copying the image onto another sheet of paper.
The display control means displays the input pattern after the image is read from the paper and before the image is copied to the other paper.
The image processing apparatus according to claim 1 or 2. The image-related process is a process of reading the image from a sheet of paper, correcting the image, and copying the image onto another sheet of paper.
When it is determined that at least a part of the image protrudes from the printable range on the other paper, the display control means displays the input pattern before copying the image.
The image processing apparatus according to claim 1 or 2. The image-related process is a process of reading the image from a sheet of paper, correcting the image, and copying the image onto another sheet of paper.
When the size of the rectangle inscribed in the image is smaller than a predetermined size with respect to the printable range on the other paper, the display control means displays the input pattern before copying the image. Let,
The image processing apparatus according to claim 1 or 2. The display control means displays the input pattern when the amount charged to the user changes depending on whether the corresponding condition is applied or not.
The image processing apparatus according to claim 2. Each of the plurality of areas is an area in which the screen displayed on the touch panel display is divided into a plurality of areas.
The input area is an area to which the locus representing the input pattern belongs the longest among the plurality of areas.
The image processing apparatus according to any one of claims 1 to 9. This is an input pattern confirmation method in an image processing device that executes image-related processing, which is processing for an image according to an input pattern input by a user among a plurality of patterns by tracing the touch panel display.
Each of the plurality of conditions of the image-related processing is stored in the storage means in association with any one of the plurality of patterns and one of the plurality of areas in the touch panel display.
A similar pattern similar to the input pattern is included in the plurality of patterns, and is associated with any of the plurality of conditions together with the similar pattern in the storage means in the plurality of areas. When the area is within the input area where the input pattern is input on the touch panel display or a predetermined range surrounding the input area, the input pattern is input to the similar pattern and the input among the plurality of conditions. The input pattern is displayed on the touch panel display together with the conditions corresponding to the pattern and the conditions corresponding to the similar pattern, and the input pattern is associated with two or more of the plurality of conditions in the storage means, and the 2 is When the area associated with each of the two or more conditions is different, the input pattern is displayed on the touch panel display together with the two or more conditions.
An input pattern confirmation method characterized by this. Image-related processing, which is processing for an image, is executed according to an input pattern input by the user by tracing the touch panel display among the plurality of patterns, and the plurality of conditions of the image-related processing are set to the plurality of patterns. A computer program used in an image processing device having a storage means for storing in association with any one of the above and any of a plurality of areas in the touch panel display.
In the image processing device
A similar pattern similar to the input pattern is included in the plurality of patterns, and is associated with any of the plurality of conditions together with the similar pattern in the storage means in the plurality of areas. When the area is within the input area where the input pattern is input on the touch panel display or a predetermined range surrounding the input area, the input pattern is input to the similar pattern and the input among the plurality of conditions. The input pattern is displayed on the touch panel display together with the conditions corresponding to the pattern and the conditions corresponding to the similar pattern, and the input pattern is associated with two or more of the plurality of conditions in the storage means, and the 2 is When the area associated with each of the two or more conditions is different, the input pattern is displayed on the touch panel display together with the two or more conditions, and the display process is executed.
A computer program characterized by that.

Images