JP7521319B2 - Image forming apparatus, image processing system, image processing method, and program - Google Patents

Description

The present invention relates to an image forming device, an image processing system, an image processing method, and a program.

Even if the color information of a document is important, when a user makes a monochrome copy using an image forming device, the color information is printed in grayscale, making the color information less noticeable. Also, when a student or other person makes a monochrome copy of a reference book or notebook in which important information is written in color for rote learning using an image forming device, a monochrome image is output. For this reason, it was not possible for students or other people to use a colored sheet to cover the printed material to hide important information, making it easier to remember the important information, like a flashcard.

Technology has been devised for printing monochrome copies of color documents (see, for example, Patent Document 1). Patent Document 1 discloses a technology that separates a single page into two parts: the part where text is emphasized using color or the like, and the other part, and prints in monochrome on the front and back of the page, or combines each part onto a single sheet.

However, conventional technology has the problem that it is not possible to copy defined areas such as color without printing multiple pages or printing in a consolidated manner. In other words, conventional technology has the problem that one sheet of paper needs to be separated into two sheets, doubling the paper and printing costs. Another problem is that when the pages are consolidated, the text becomes smaller, making it difficult to read.

In view of the above problems, the present invention aims to provide an image forming device that can copy a defined area without printing multiple pages or performing aggregate printing.

In view of the above problems, the present invention provides an image forming apparatus comprising: a reading unit that reads an original document to generate image data; a defined area identification unit that identifies a defined area consisting of a character string of less than one line from the image data generated by the reading unit; a storage location addition unit that identifies the character string in the defined area identified by the defined area identification unit through OCR processing, deletes the defined area leaving only the first character or the last character of the character string, and forms storage location information of the defined area after the first character or before the last character of the character string ; a storage unit that stores an image of the defined area identified by the defined area identification unit in a storage location indicated by the storage location information; and a printing unit that prints the image data for which the storage location addition unit has formed the storage location information onto a sheet-like member.

It is possible to provide an image forming device that can copy a defined area without printing multiple pages or performing aggregate printing.

FIG. 1 is a diagram illustrating an example of an outline of an operation of an image processing system. FIG. 1 is a diagram illustrating an example of a schematic configuration of an image processing system. FIG. 2 is a diagram illustrating an example of a hardware configuration of an image forming apparatus. FIG. 2 is a diagram illustrating an example of a hardware configuration of an information processing device. FIG. 2 is a diagram illustrating an example of a hardware configuration of a terminal device. FIG. 2 is an example of a functional block diagram illustrating functions of an information processing apparatus, a terminal apparatus, and an image forming apparatus, divided into blocks. 1 is an example of a diagram illustrating coordinates of a definition region. 1 is a diagram showing an example of an original that a user causes an image forming apparatus to read; 13 is a diagram showing an example of a definition area confirmation screen displayed on a panel display unit after scanning by the image forming apparatus. FIG. FIG. 13 is a diagram showing an example of a code image printout in which a defined region is replaced with a code image. FIG. 13 is a diagram illustrating an example of a composite image screen displayed by the terminal device. 11 is a flowchart illustrating an example of a procedure in which an image forming apparatus prints a code image printout in which a defined region is replaced with a code image. 11 is a flowchart illustrating an example of a procedure in which a terminal device captures an image of a code image printed by an image forming apparatus with a camera and combines an image of a defined region with the code image print.

Below, an image processing system and an image processing method performed by the image processing system will be described with reference to the drawings as an example of an embodiment of the present invention.

<Outline of operation>
1 is a diagram for explaining an outline of the operation of an image processing system 100 according to this embodiment. This image processing system 100 replaces a defined area, such as a color area, of an original with a code image and prints it. An image of the defined area is stored in an image forming apparatus 20 or an information processing apparatus 10 on a network, and the storage location (e.g., a URL) is coded in the code image. When a user reads the code image with a terminal device 40, the terminal device 40 can access the storage location and display the original image data.

(1) The user selects a specific operating mode and copies an original document using the image forming device 20. Assume that this original document has one or more defined areas. The defined area is, for example, an area in which text or the like is highlighted in color.

(2) The image forming device 20 identifies a defined area from the image data generated by scanning the document, and acquires the coordinates and an image of the defined area for each defined area. The image of the defined area is transmitted to the information processing device 10. The information processing device 10 stores the image of the defined area.

(3) The image forming device 20 converts the storage location of the image of the defined area into a code image. Therefore, the URL where the image of the defined area is stored is encoded in the code image. The image forming device 20 prints the image in which the defined area has been replaced with the code image on paper (hereinafter, referred to as a code image printout).

(4) When the user wants to check the information written in the definition area, the user captures the code image printout with the camera of the terminal device 40. This allows the terminal device 40 to decode the code image and obtain the storage location. The terminal device 40 obtains the image of the definition area from the URL where the image of the definition area is stored. The terminal device 40 displays the image of the definition area obtained from the information processing device 10 on the display by combining (superimposing) it with the location of the code image in the image captured by the camera. Thus, the user can read the original document that was copied.

In this way, the image processing system 100 of this embodiment can provide learning assistance even in monochrome printing. Because the image forming device 20 prints only one sheet of paper, the cost of providing learning assistance can be reduced. In addition, because there is no need to print multiple documents on one sheet of paper, it is possible to prevent information from becoming difficult to read.

<Terminology>
A defined region is an area that is predefined in the image data, such as a color region, although other examples are shown in Table 1. A defined region can also be referred to as a highlighted region, for example.

The storage location information is information that indicates the storage location of the image in the defined area. Any form that indicates the storage location is acceptable. For example, it could be a URL, a file path, etc.

The sheet-like member may be any member that is thin enough to allow an image to be formed on it. For example, it may be paper, but is not limited to paper.

<System configuration example>
2 is an example of a schematic configuration diagram of the image processing system 100 of this embodiment. The image processing system 100 of this embodiment has an image forming apparatus 20 and an information processing apparatus 10, which are capable of communicating via a network N. In addition, although not included in the image processing system 100, it is preferable that the image processing system 100 has a terminal device 40 used by a user.

The network N includes the LAN of the facility where the image forming device 20 is located, and includes the Internet when the information processing device 10 is located on the Internet. The network N may also include telephone lines such as a mobile phone network. Note that when the information processing device 10 is located on-premise, the network N may not include the Internet.

The image forming device 20 reads an original document with a scanner and prints it on paper. The printing method of the image forming device 20 can be electrophotographic or inkjet, but the printing method is not limited in this embodiment. The image forming device 20 is a copier, but may also be called a printer device, printing device, laser printer, or inkjet printer. It may also be called a multifunction device (multifunction printer) or MFP (Multifunction Peripheral/Product/Printer). These have multiple functions such as a scanner, printer, and fax in one device. However, the scanner and printing device may be configured as separate devices.

The image forming device 20 of this embodiment is a terminal that allows a user to use a service that codes and prints a defined area of a document. The user can log in to the information processing device 10 from the image forming device 20. In this case, the information processing device 10 can identify the user, so that the user can be charged cashlessly using a pre-registered credit card or the like. In addition, after printing the code image printout, the information processing device 10 can improve security by allowing only the user to download the image of the defined area.

When the user does not log in to the information processing device 10, the user can insert cash into a coin vendor or pay with a prepaid card. In addition, any user who has a printed code image can download the image of the defined area, which is convenient when multiple people share a printed code image.

The information processing device 10 serves as a storage means for storing the image of the defined area. It may further have a function (print server) for assisting the processing of the image forming device 20. As described above, the information processing device 10 transmits the image of the defined area to the terminal device 40 in response to a request from the terminal device 40. Therefore, it is sufficient for the information processing device 10 to have the functionality of a web server.

Note that if the image forming device 20 stores the image of the defined area within its own device, the information processing device 10 may not be necessary. In this case, the image forming device 20 has the functionality of a web server.

In addition, multiple information processing devices 10 may be connected to the network. In this case, the functions of one information processing device 10 may be realized by multiple information processing devices 10, or the multiple information processing devices 10 may each have the same functions described above.

The information processing device 10 may be compatible with cloud computing. Cloud computing refers to a form of usage in which resources on a network are used without being aware of specific hardware resources. An information processing device 10 compatible with cloud computing may be called a cloud system. The cloud system may be located on the Internet or on-premise.

The terminal device 40 is a general-purpose information processing device 10 carried by a user. For example, it is a smartphone, a mobile phone, a tablet PC, a notebook PC, a PDA (Personal Digital Assistant), etc. An app dedicated to the image processing system 100 of this embodiment or a general-purpose web browser is running on the terminal device 40. The app or web browser requests an image of the defined area from the information processing device 10, using the URL included in the code image as the destination.

Even if the terminal device 40 is not present, when the user has the image forming device 20 read the code image printout, the image forming device 20 can download the image of the definition area from the information processing device 10 and combine and print the image of the definition area on the code image printout.

<Hardware configuration example>
<<Image forming apparatus>>
3 is a diagram showing the hardware configuration of the image forming apparatus 20. The image forming apparatus 20 includes a controller 910, a short-range communication circuit 920, an engine control unit 930, an operation panel 940, and a network I/F 950.

Of these, the controller 910 has a CPU 901, which is the main part of the computer, a system memory (MEM-P) 902, a north bridge (NB) 903, a south bridge (SB) 904, an ASIC (Application Specific Integrated Circuit) 906, a local memory (MEM-C) 907, which is a storage unit, a HDD controller 908, and a HD 909, which is also a storage unit, and is configured such that the NB 903 and the ASIC 906 are connected by an AGP (Accelerated Graphics Port) bus 921.

Of these, the CPU 901 is a control unit that performs overall control of the image forming apparatus. The NB 903 is a bridge that connects the CPU 901 with the MEM-P 902, the SB 904, and the AGP bus 921, and includes a memory controller that controls reading and writing to the MEM-P 902, a PCI (Peripheral Component Interconnect) master, and an AGP target.

The MEM-P 902 is composed of a ROM 902a, which is a memory for storing programs and data that realize the various functions of the controller 910, and a RAM 902b, which is used for expanding the programs and data, and as a drawing memory during memory printing. The programs stored in the RAM 902b may be provided by recording them in an installable or executable format on a computer-readable recording medium such as a CD-ROM, CD-R, or DVD.

SB904 is a bridge for connecting NB903 with PCI devices and peripheral devices. ASIC906 is an IC (Integrated Circuit) for image processing applications that has hardware elements for image processing, and acts as a bridge connecting AGP bus921, PCI bus922, HDD controller908, and MEM-C907. This ASIC906 is composed of a PCI target and AGP master, an arbiter (ARB) that is the core of ASIC906, a memory controller that controls MEM-C907, multiple DMACs (Direct Memory Access Controllers) that rotate image data using hardware logic, and a PCI unit that transfers data between scanner unit931 and printer unit932 via PCI bus922. In addition, a USB (Universal Serial Bus) interface or an IEEE 1394 (Institute of Electrical and Electronics Engineers 1394) interface may be connected to the ASIC 906.

MEM-C907 is a local memory used as an image buffer for copying and a code buffer. HD909 is a storage for storing image data, font data used during printing, and forms. HD909 controls the reading and writing of data from and to HD909 under the control of CPU901. AGP bus921 is a bus interface for a graphics accelerator card proposed to speed up graphic processing, and by directly accessing MEM-P902 with high throughput, the graphics accelerator card can be made faster.

The short-range communication circuit 920 also includes a short-range communication circuit antenna 920a. The short-range communication circuit 920 is a communication circuit such as NFC or Bluetooth (registered trademark).

The engine control unit 930 is further made up of a scanner unit 931 and a printer unit 932. The operation panel 940 is equipped with a panel display unit 940a, such as a touch panel, which displays the current settings and selection screens and receives input from the operator, and an operation panel 940b consisting of a numeric keypad that receives settings for image formation conditions such as density settings and a start key that receives a copy start command. The controller 910 controls the entire image forming apparatus, and controls, for example, drawing, communication, and input from the operation panel 940. The scanner unit 931 or the printer unit 932 includes an image processing unit such as error diffusion and gamma conversion.

The image forming device 20 can sequentially switch between the document box function, copy function, printer function, and facsimile function using the application switching key on the operation panel 940. When the document box function is selected, the document box mode is selected; when the copy function is selected, the copy mode is selected; when the printer function is selected, the printer mode is selected; and when the facsimile mode is selected, the facsimile mode is selected.

The network I/F 950 is an interface for data communication using a communication network. The short-range communication circuit 920 and the network I/F 950 are electrically connected to the ASIC 906 via the PCI bus 922.

<<Information processing device>>
Fig. 4 is a hardware configuration diagram of the information processing device 10. As shown in Fig. 5, the information processing device 10 is constructed by a computer, and includes a CPU 501, a ROM 502, a RAM 503, a HD 504, a HDD (Hard Disk Drive) controller 505, a display 506, an external device connection I/F (Interface) 508, a network I/F 509, a bus line 510, a keyboard 511, a pointing device 512, a DVD-RW (Digital Versatile Disk Rewritable) drive 514, and a media I/F 516.

Of these, the CPU 501 controls the operation of the entire information processing device 10. The ROM 502 stores programs used to drive the CPU 501, such as IPL. The RAM 503 is used as a work area for the CPU 501. The HD 504 stores various data such as programs. The HDD controller 505 controls the reading or writing of various data from the HD 504 according to the control of the CPU 501. The display 506 displays various information such as a cursor, menu, window, character, or image. The external device connection I/F 508 is an interface for connecting various external devices. In this case, the external device is, for example, a USB (Universal Serial Bus) memory or a printer. The network I/F 509 is an interface for data communication using a communication network. The bus line 510 is an address bus, a data bus, or the like for electrically connecting each component such as the CPU 501 shown in FIG. 4.

The keyboard 511 is a type of input means equipped with multiple keys for inputting characters, numbers, various instructions, etc. The pointing device 512 is a type of input means for selecting and executing various instructions, selecting a processing target, moving the cursor, etc. The DVD-RW drive 514 controls the reading and writing of various data from the DVD-RW 513, which is an example of a removable recording medium. Note that this is not limited to a DVD-RW, and may be a DVD-R, etc. The media I/F 516 controls the reading and writing (storing) of data from the recording medium 515, such as a flash memory.

<<Terminal Device>>
Fig. 5 is a hardware configuration diagram of the terminal device 40. As shown in Fig. 5, the terminal device 40 includes a CPU 401, a ROM 402, a RAM 403, an EEPROM 404, a CMOS sensor 405, an image sensor I/F 406, an acceleration/direction sensor 407, a media I/F 409, and a GPS receiving unit 411.

Of these, the CPU 401 controls the operation of the entire terminal device 40. The ROM 402 stores the CPU 401 and programs used to drive the CPU 401, such as the IPL. The RAM 403 is used as a work area for the CPU 401. The EEPROM 404 reads or writes various data, such as programs for the terminal device 40, under the control of the CPU 401. The CMOS (Complementary Metal Oxide Semiconductor) sensor 405 is a type of built-in imaging means that captures an image of a subject (mainly a self-portrait) under the control of the CPU 401 to obtain image data. Note that instead of a CMOS sensor, an imaging means such as a CCD (Charge Coupled Device) sensor may also be used. The imaging element I/F 406 is a circuit that controls the operation of the CMOS sensor 405. The acceleration/direction sensor 407 is various sensors such as an electronic magnetic compass or gyrocompass that detects geomagnetism, and an acceleration sensor. The media I/F 409 controls the reading and writing (storage) of data from and to a recording medium 408 such as a flash memory. The GPS receiver 411 receives GPS signals from GPS satellites.

The terminal device 40 also includes a long-distance communication circuit 412, a CMOS sensor 413, an image sensor I/F 414, a microphone 415, a speaker 416, an audio input/output I/F 417, a display 418, an external device connection I/F (Interface) 419, a short-distance communication circuit 420, an antenna 420a of the short-distance communication circuit 420, and a touch panel 421.

Among these, the long-distance communication circuit 412 is a circuit that communicates with other devices via a communication network. The CMOS sensor 413 is a type of built-in imaging means that captures an image of a subject and obtains image data according to the control of the CPU 401. The image sensor I/F 414 is a circuit that controls the drive of the CMOS sensor 413. The microphone 415 is a built-in circuit that converts sound into an electrical signal. The speaker 416 is a built-in circuit that converts an electrical signal into physical vibrations to generate sounds such as music and voice. The sound input/output I/F 417 is a circuit that processes the input and output of sound signals between the microphone 415 and the speaker 416 according to the control of the CPU 401. The display 418 is a type of display means such as a liquid crystal or organic EL (Electro Luminescence) that displays an image of a subject, various icons, etc. The external device connection I/F 419 is an interface for connecting various external devices. The short-distance communication circuit 420 is a communication circuit such as NFC (Near Field Communication) or Bluetooth (registered trademark). The touch panel 421 is a type of input means that allows the user to operate the terminal device 40 by pressing the display 418.

The terminal device 40 also includes a bus line 410. The bus line 410 is an address bus, a data bus, etc., for electrically connecting the components such as the CPU 401 shown in FIG. 5.

<About the function>
FIG. 6 is an example of a functional block diagram for explaining functions of the information processing device 10, the terminal device 40, and the image forming device 20 by dividing them into blocks.

<<Information processing device>>
The information processing device 10 has a communication unit 11 and a data storage unit 12. Each function of the information processing device 10 is a function or means realized by the CPU 501 shown in FIG.

First, the communication unit 11 can communicate with the image forming device 20 and the terminal device 40. The communication unit 11 receives an image of the defined area from the image forming device 20, and also transmits the image of the defined area to the terminal device 40 in response to a request from the terminal device 40.

The data storage unit 12 stores the image of the defined area in the storage location transmitted from the image forming device 20. The storage location of the image of the defined area in the data storage unit 12 is coded into a code image.

<<Terminal Device>>
The terminal device 40 has a communication unit 41, an image analysis unit 42, a display unit 43, an image acquisition unit 44, an image synthesis unit 45, and an operation reception unit 46. Each function of the terminal device 40 is a function or means realized by the CPU 401 shown in FIG. 5 executing a program expanded from the EEPROM 404 to the RAM 403.

The image acquisition unit 44 has the functionality of a camera realized by the CMOS sensor 413, and captures an image of the code image printout in which the defined area has been replaced with the code image.

The image analysis unit 42 determines whether the image data acquired by the image acquisition unit 44 includes a code image. For example, in the case of a QR code (registered trademark), the image data is searched for three position detection patterns located at three corners. In a QR code (registered trademark), the ratio of white cells to black cells is 1:1:3:1:1 in all directions, including horizontal, vertical, and diagonal directions, so if the three position detection patterns can be detected, the position of the code image can be identified.

If a code image is present, the image analysis unit 42 decodes the embedded information. In other words, it obtains the storage location information (URL, file path, etc.) embedded in the code image.

The communication unit 41 communicates with the information processing device 10 and receives an image of the defined area from the storage destination information contained in the code image (an example of a second communication unit).

The image synthesis unit 45 synthesizes the image of the defined area received by the communication unit 41 into the defined area (where the code image is located) of the code image printout acquired by the image acquisition unit 44.

The display unit 43 has a function of displaying images to the user, such as a liquid crystal display, and displays the image data synthesized by the image synthesis unit 45. In other words, it displays the image data before the definition area is replaced with the code image. The operation acceptance unit 46 accepts various operations on the terminal device 40.

<<Image forming apparatus>>
The image forming apparatus 20 includes a reading unit 21, a definition area specifying unit 22, a definition area coordinate holding unit 23, a display unit 24, an operation receiving unit 25, a storage location adding unit 26, a definition area extracting unit 27, a printing unit 28, and , and a communication unit 29. Each function of the image forming apparatus 20 is realized by the CPU 901 shown in FIG.

The reading unit 21 optically reads an original placed on the contact glass or an original transported to the contact glass by an ADF (Auto Document Feeder) to generate image data. The image data may be color or monochrome, but it is preferable to read in color so that the image forming device 20 can identify the color definition area. However, reading in monochrome is also acceptable.

The defined area identification unit 22 judges whether the image data generated by the reading unit 21 includes a defined area. For example, it judges the circumscribing rectangle of the color pixels in the image data to be the defined area. Since the RGB of black is (0,0,0) and the RGB of white is (255,255,255), the defined area identification unit 22 detects, for example, colored areas with RGB of (1,1,1) to (254,254,254).

Alternatively, the defined area identification unit 22 performs character recognition using an OCR (Optical Character Reader) and detects an area surrounded by "( ) (parentheses)" or angle brackets as the defined area. In addition, since OCR can detect bold and italic characters, it is also possible to determine an area written in bold or italic characters as the defined area. Images, figures, etc. may also be used as the defined area.

表１は、定義領域の一例である。ユーザは画像形成装置２０を操作してメニューなどから自分がコード化したい定義領域を設定できる。このように、ユーザは所望の定義領域を登録できる。ユーザは任意の文字列を登録することもできる。 The defined region is not limited to these, and may be a region defined by the user. Table 1 shows an example of the defined region.

Table 1 shows an example of a definition area. A user can operate the image forming apparatus 20 to set a definition area that the user wishes to code from a menu or the like. In this way, the user can register a desired definition area. The user can also register an arbitrary character string.

The definition area coordinate storage unit 23 stores the coordinates of the four corners of the definition area (defined area coordinates). The display unit 24 displays the frame of the detected definition area on the panel display unit 940a, superimposed on the image data. In other words, a preview is displayed to indicate that the area indicated by the frame will be coded as the definition area.

The operation reception unit 25 receives editing or addition of the definition area. When the operation reception unit 25 receives a change, the definition area coordinate storage unit 23 updates the coordinates of the definition area. When an addition is received, the definition area coordinate storage unit 23 additionally stores the coordinates of the definition area.

When the change or addition is complete, the definition area extraction unit 27 extracts the image of the definition area specified by the coordinates of the definition area from the image data. In other words, it trims or cuts out. Therefore, images of the definition areas are extracted from one image data as many as the number of definition areas.

The storage location adding unit 26 forms a code image at the coordinates of the definition area of the image data. Since the code image is overwritten onto the definition area, the information written in the definition area is lost. The code image contains encoded storage location information for saving the image of the definition area.

The communication unit 29 communicates with the information processing device 10, specifies storage location information, and transmits a request to save the image of the defined area to the information processing device 10. The communication unit 29 is also a storage unit that saves the image of the defined area in the storage location.

The printing unit 28 prints the image data, on which the storage location adding unit 26 has formed a code image in the defined area, in monochrome on paper (outputs a code image printout). Because a monochrome code image has been formed, the terminal device 40 can capture the code image, obtain an image of the defined area from the information processing device 10, and reproduce the information that was written in the defined area.

<About the coordinates of the defined area>
The coordinates of the defined area will be described with reference to Fig. 7. Fig. 7(a) is a diagram for explaining the coordinates of the defined area. The coordinates of the defined area are determined, for example, with the upper left vertex of the document or the image data generated by reading the document by the image forming apparatus 20 as the origin. The width direction is the X axis, and the height direction is the Y axis. The unit is, for example, the number of pixels.

Figure 7(b) is an example of the coordinates of a defined area. In Figure 7, the coordinates of each defined area on one page of a manuscript are shown. The coordinates of the defined area in Figure 7(b) are the coordinates of the four vertices 200 of the defined area rectangle. It is preferable that the coordinates of the defined area specified by color are the circumscribing rectangle of the defined area, and the coordinates of the defined area specified by parentheses are the inscribing rectangle.

The coordinates of the definition area may be specified by the top left and bottom right vertices, the top right and bottom left vertices, or one vertex, width, and height. Such coordinates of the definition area are stored in the definition area coordinate storage unit 23. The coordinates of the definition area stored in the definition area coordinate storage unit 23 may be discarded after the code image printout is printed.

<Manuscripts, printed code images, image data displayed on terminal devices, etc.>
The following describes the process from reading the image of the original document to the terminal device 40 capturing and displaying the code image printout.

8 shows an example of an original that a user causes the image forming apparatus 20 to read. The original has four defined areas 301, 302, 303, and 304.
The definition area 301 is an area colored with a color marker.
・Definition areas 302, 303, and 304 are areas enclosed in parentheses.

Figure 9 shows an example of a defined area confirmation screen 310 that is scanned by the image forming device 20 and displayed on the panel display unit 940a. The defined area confirmation screen 310 displays the defined areas detected by the defined area identification unit 22 of the image forming device 20 surrounded by square frames 311, 312, 313, and 314. In Figure 9, the four defined areas 301, 302, 303, and 304 that existed on the original have been correctly detected.

The definition area confirmation screen 310 has an add selection range button 315 and an OK button 316. The add selection range button 315 is a button that allows the user to add a definition area. When the user presses the add selection range button 315, a rectangular frame is displayed on the panel display section 940a. The user can move this rectangular frame to the definition area and enlarge or reduce it as appropriate.

When a box is displayed in an area that should not be highlighted, the user can remove the box by long pressing the box to reveal a delete button. Alternatively, there may be a delete button that is always visible.

When the user presses the OK button 316, an image of the definition area specified by the square frames 311, 312, 313, and 314 is sent to the information processing device 10. In addition, a code image printout is printed in which the definition area specified by the square frames 311, 312, 313, and 314 is replaced with a code image of the storage location information.

Figure 10 shows an example of a code image printout 340 in which the definition area has been replaced with a code image. The beginnings of the square frames 311, 312, 313, and 314 identified as the definition area on the definition area confirmation screen 310 in Figure 9 have been replaced with code images 321, 322, 323, and 324. These code images 321, 322, 323, and 324 contain coded URLs and other information that stores the images of the definition areas.

In addition, the definition area in FIG. 10 is formed with diagonal lines 330. The diagonal lines 330 are not necessary, but they make it easier for the user to understand the areas that have been left blank by the coding, and to estimate the number of characters, etc.

The code images 321, 322, 323, and 324 may be printed in an inconspicuous color such as yellow (in this case, two-color printing is used instead of monochrome). This makes it easier for the user to grasp the entire area that has been left blank by the coding, making it easier to estimate the number of characters, etc.

In FIG. 10, none of the information in the original definition area is readable. However, a code image may be formed leaving part of the definition area. That is, the storage location adding unit 26 may delete at least part of the definition area. For example, when the image forming device performs OCR, it is possible to leave the first or last character. For color information, it is possible to leave a certain amount of the image from the left edge. This allows the user to get a hint as to the answer.

The location where the code image is formed does not have to overlap with the defined area. For example, the code image may be formed around the defined area. Preferably, the user is able to understand the correspondence between the code image and the defined area, but the code image may be formed in the margins of the code image printout 340 in the order in which the defined areas appear.

Even if the user looks at the paper in FIG. 10, the information written in the definition area cannot be seen, so the user can learn to guess the information written in the definition area. When the answer is needed, the user captures the code image with the camera of the terminal device 40.

Figure 11 is an example of a composite image screen 360 displayed by the terminal device 40. In Figure 11, the range of the code image printout 340 that falls within the camera's angle of view is indicated by a dotted line 350. The range of the code image printout within the dotted line 350 is displayed on the composite image screen 360. An image of the definition area acquired by the terminal device 40 from the information processing device 10 based on the coded URL is composited at the location of the code images 321, 322, 323, 324. Therefore, on the composite image screen 360, the definition areas 302, 303, 304 are displayed instead of the code images 321, 322, 323, 324. Therefore, the user can confirm the answer.

<Operation or processing procedure>
FIG. 12 is a flow chart showing an example of a procedure in which the image forming apparatus 20 prints a code image printout in which the defined area has been replaced with a code image.

The user places an original on the image forming device 20 and inputs a copy operation to the image forming device 20. The operation reception unit 25 of the image forming device 20 receives the operation, and the reading unit 21 reads the original and generates image data. This original has one or more defined areas.

When image data is generated, the definition area identification unit 22 searches for the definition area (color area) from the top left corner of the image data (S1). Alternatively, OCR is performed to search for the defined character string. The definition area coordinate storage unit 23 stores (stores) the coordinates of the identified definition area (S2).

The definition area identification unit 22 determines whether or not the entire image data has been searched (S3). When the entire image data has been searched, the display unit 24 displays a frame indicating the definition area on the operation panel 940b together with the image data (S4). The definition area is displayed in its original position on the image data, highlighted with a rectangular frame or in a different color.

The user checks whether the position of the definition area is incorrect, and if it is incorrect, changes the position of the definition area. Also, if there is an unspecified definition area, it is added. The operation reception unit 25 receives these operations.

Once the user's confirmation is complete, the definition area extraction unit 27 extracts (trims) the image of the definition area, and the storage location addition unit 26 forms a code image in which the storage location information is coded in the definition area (S5). The definition area extraction unit 27 may delete the image of the definition area, or may overwrite the information written in the definition area with the code image. The storage location information (URL, etc.) coded in the code image may be "a predetermined folder + date and time + branch number", etc. The folder name may be determined in this manner when the storage location information is a file path.

The printing unit 28 prints the image data in which the defined area has been replaced with the code image on paper (S6). This completes the process of outputting the code image printout.

Then, the communication unit 29 specifies a storage location and transmits the image of the defined area to the information processing device 10. The information processing device 10 stores the image of the defined area in the specified storage location (S7). The storage may be performed before printing.

Next, the restoration of the image of the defined area will be described with reference to FIG. 13. FIG. 13 is an example of a flowchart showing the procedure in which the terminal device 40 captures an image of the code image printed by the image forming device 20 with a camera and combines the image of the defined area with the code image printout.

When the user uses the printed paper for study and thinks that the answer is necessary, the terminal device 40 captures an image of the code image printout (S11). The image acquisition unit 44 of the terminal device 40 acquires image data of the code image printout. Note that there are two methods for capturing an image: capturing a still image and capturing a video. Either method is acceptable in this embodiment.

The image of the code image printout is sent to the image analysis unit 42, which analyzes the image data to detect the code image (S12). If a code image is detected, the image analysis unit 42 decodes the embedded storage location information.

The communication unit 41 acquires the image of the defined area from a storage location such as the information processing device 10 based on the embedded storage location information (S13). The communication unit 41 transmits the image of the defined area to the image synthesis unit 45.

The image synthesis unit 45 synthesizes the image of the definition area at the location of the code image on the code image printout acquired by the image acquisition unit 44, and sends the result to the display unit 43. This synthesis can be achieved by overlaying the image of the definition area in front of the code image printout. The display unit 43 displays the image data with the image of the definition area synthesized (S14). Therefore, the user can confirm (see) the information before the definition area was masked.

During the synthesis, the image synthesis unit 45 aligns the left edge of the image of the definition area with the left edge of the code image on the code image printout. The image synthesis unit 45 also aligns the center of the height direction of the image of the definition area with the center of the height direction of the code image.

The size of the image of the defined area depends on the resolution (e.g., 200 dpi) used when reading by the reading unit 21 of the image forming device 20. Meanwhile, the size of the defined area in the image data obtained by the camera of the terminal device 40 capturing an image of the code image printout depends on the focal length, the number of pixels of the camera, and the distance between the camera and the code image printout. For this reason, the image synthesis unit 45 finds the ratio at which the height of the code image matches the height of the image of the defined area, and scales the vertical and horizontal sizes of the image of the defined area at this ratio. In this way, an image of the defined area that is approximately the same size as the code image on the code image printout can be synthesized onto the code image printout.

<Major Effects>
As described above, the image processing system 100 of the present embodiment can provide learning assistance even in monochrome printing. Since the image forming device 20 prints only one sheet of paper, the cost of providing learning assistance can be reduced. In addition, since there is no need to print multiple documents on one sheet of paper, it is possible to prevent information from becoming difficult to read.

<Other application examples>
The above describes the best mode for carrying out the present invention using examples, but the present invention is not limited to these examples in any way, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention.

For example, while this embodiment has been described as an application example of learning assistance, this embodiment can also be applied for purposes such as improving security when a user carries around a printed code image. For example, if an individual's name is written on an internal company document, the image forming device 20 can replace the individual's name with a code image. Alternatively, the image forming device 20 can replace a term registered in a dictionary with a code image. Therefore, the original document is not necessarily in color.

In addition, in this embodiment, the image forming device 20 reads the document, but the document may be read by a separate scanner or camera. In this case, the image forming device obtains the image data and prints the code image printout.

In addition, in this embodiment, the information processing device 10 is used as a storage location for the image of the defined area, but the information processing device 10 may perform image processing instead of the image forming device. In this case, the image forming device scans an original and transmits image data to the information processing device 10. The information processing device 10 detects the defined area and transmits image data in which a code image has been formed to the image forming device 20. The image forming device 20 prints a printout of the code image. In this way, functions related to image processing can be consolidated in the information processing device 10, and the image forming device 20 only needs to have its original functions (scanner and printer), thereby suppressing cost increases. Note that this type of embodiment is called a Web app.

The configuration example in FIG. 6 is divided according to main functions to facilitate understanding of the processing by the terminal device 40, image forming device 20, and information processing device 10. The present invention is not limited by the manner in which the processing units are divided or the names of the processing units. The processing by the terminal device 40, image forming device 20, and information processing device 10 can also be divided into even more processing units depending on the processing content. Also, it can be divided so that one processing unit includes even more processes.

Furthermore, each function of the embodiments described above can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to execute each function by software, such as a processor implemented by an electronic circuit, and devices such as an ASIC (Application Specific Integrated Circuit), DSP (digital signal processor), FPGA (field programmable gate array), and conventional circuit modules designed to execute each function described above.

10 Information processing device 20 Image forming device 40 Terminal device 100 Image processing system

Patent No. 6598665

Claims (9)

A reading unit that reads an original and generates image data;
a defined area specifying unit that specifies a defined area consisting of a character string of less than one line in the image data generated by the reading unit;
a storage location adding unit that identifies the character string in the definition area identified by the definition area identifying unit through OCR processing, deletes the definition area leaving only the first character or the last character of the character string, and forms storage location information of the definition area after the first character or before the last character of the character string ;
a storage unit that stores an image of the definition area specified by the definition area specifying unit in a storage location indicated by the storage location information;
a printing unit that prints the image data on which the storage location information has been formed by the storage location adding unit onto a sheet-like member;
An image forming apparatus comprising: The image forming apparatus according to claim 1, characterized in that the storage location adding unit either deletes the information of the definition area identified by the definition area identification unit and then forms the storage location information of the definition area in the definition area, or overwrites the storage location information of the definition area in the definition area identified by the definition area identification unit. a display unit that highlights the defined area specified by the defined area specifying unit and displays it on the image data generated by the reading unit;
an operation receiving unit that receives a change to the definition area displayed by the display unit or an addition of a definition area that was not specified by the definition area specifying unit;
3. The image forming apparatus according to claim 1, further comprising: The image forming device according to any one of claims 1 to 3, characterized in that the defined area is an area having color pixels. The image forming device according to any one of claims 1 to 3, characterized in that the defined area is an area enclosed in parentheses. A reading unit that reads an original and generates image data;
a defined area specifying unit that specifies a defined area consisting of a character string of less than one line from the image data generated by the reading unit;
a storage location adding unit that deletes at least a part of the definition area specified by the definition area specifying unit and forms storage location information of the definition area in the image data;
a storage unit that stores an image of the definition area specified by the definition area specifying unit in a storage location indicated by the storage location information;
an image forming apparatus having a printing unit that prints the image data on which the storage location adding unit has formed the storage location information on a sheet-like material; and a terminal device,
The terminal device
an image acquisition unit that captures an image of the sheet-like member printed by the image forming apparatus;
an image analysis unit that analyzes the image data of the sheet-like member acquired by the image acquisition unit to acquire the storage location information;
a second communication unit that acquires an image of the defined area from a storage location indicated by the storage location information acquired by the image analysis unit;
an image synthesis unit that synthesizes an image of the defined area acquired by the second communication unit into an area in which the storage location information is formed in the image data of the sheet-like member acquired by the image acquisition unit ,
the image synthesis unit determines a ratio at which the height of the storage location information on the sheet-like member coincides with the height of the image of the definition area, scales the vertical and horizontal sizes of the image of the definition area at the ratio, and synthesizes the image of the definition area having the same size as the size of the storage location information on the sheet-like member into the area in which the storage location information is formed.
Image processing system. A reading unit reads an original document and generates image data;
a step of a definition area specifying unit specifying a definition area consisting of a character string of less than one line in the image data generated by the reading unit;
a storage location adding unit, by OCR processing, identifying the character string in the definition area identified by the definition area identifying unit, deleting the definition area while leaving only the first character or the last character of the character string, and forming storage location information of the definition area after the first character or before the last character of the character string ;
A storage unit stores an image of the definition region specified by the definition region specifying unit in a storage location indicated by the storage location information;
a printing unit printing the image data on which the storage location information has been formed by the storage location adding unit onto a sheet-like member;
13. An image processing method comprising the steps of: An image forming apparatus,
A reading unit that reads an original and generates image data;
a defined area specifying unit that specifies a defined area consisting of a character string of less than one line in the image data generated by the reading unit;
a storage location adding unit that identifies the character string in the definition area identified by the definition area identifying unit through OCR processing, deletes the definition area leaving only the first character or the last character of the character string, and forms storage location information of the definition area after the first character or before the last character of the character string ;
a storage unit that stores an image of the definition area specified by the definition area specifying unit in a storage location indicated by the storage location information;
a printing unit that prints the image data on which the storage location information has been formed by the storage location adding unit onto a sheet-like member;
A program to function as a A reading unit that reads an original and generates image data;
a defined area specifying unit that specifies a defined area consisting of a character string of less than one line from the image data generated by the reading unit;
a storage location adding unit that deletes at least a part of the definition area specified by the definition area specifying unit and forms storage location information of the definition area in the image data;
a storage unit that stores an image of the definition area specified by the definition area specifying unit in a storage location indicated by the storage location information;
a printing unit that prints the image data on which the storage location information is formed by the storage location adding unit onto a sheet-like member; and an image acquiring unit that captures an image of the sheet-like member on which the image forming apparatus has printed;
an image analysis unit that analyzes the image data of the sheet-like member acquired by the image acquisition unit to acquire the storage location information;
a second communication unit that acquires an image of the defined area from a storage location indicated by the storage location information acquired by the image analysis unit;
making the terminal device function as an image synthesis unit that synthesizes an image of the defined area acquired by the second communication unit into an area in which the storage location information is formed in the image data of the sheet-like member acquired by the image acquisition unit;
The image synthesis unit is a program for calculating the ratio at which the height of the storage location information on the sheet-like member matches the height of the image of the defined area, scaling the vertical and horizontal sizes of the image of the defined area by said ratio, and synthesizing the image of the defined area of the same size as the size of the storage location information on the sheet-like member into the area in which the storage location information is formed .

Images