JP2006238289A - Method of magnifying display data - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To facilitate reading of a sentence, even within a limited image size. <P>SOLUTION: When displaying on a two-dimensional display screen an image represented by paintings and calligraphic works information, image data are separated into a character region and background by character/background separation or layout analysis processing; further, the size of one character is measured by performing OCR processing or character rectangle analysis processing on the character region, a display magnification power is automatically computed from the character size; and at least one direction in the two-dimensional directions of an image data picture is varied. For MRC, PDF data stored, while separating background/photograph portions and character portions, a character portion/photographic portion are discriminated; further, the size of one letter is measured by performing OCR processing or character rectangle analysis processing on the character portion, display magnification power is automatically computed from the letter size; and at least one direction in the indication of two-dimensional directions is varied with the calculated magnification power. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a method for scaling document information in a screen display of a document, photograph, drawing or the like by a two-dimensional display device such as a CRT, a liquid crystal display, a plasma display, an LED surface display, or an EL display. The present invention can be implemented in a device that displays a document on a two-dimensional display device that displays a document such as a document and an image on a screen, such as a personal computer, an operation board, an operation terminal, a portable terminal, and a mobile phone with a two-dimensional display.

JP 2001-230937 A Japanese Patent Laid-Open No. 2003-60901 Special table 2003-531429 gazette.

  Patent Document 1 describes a facsimile apparatus that captures color image data from a digital camera, reads an original image with a scanner, synthesizes the read image with a color image of the digital camera by MRC processing, and sends the synthesized image to a communication line. ing.

  In Patent Document 2, when the document size does not fit in the paper size, a reduction ratio that enters the paper size in both the vertical and horizontal directions is calculated, and the copy is made by scaling the vertical and horizontal directions at the reduction ratio. The copying machine to take is described.

  Patent Document 3 describes digital document processing for converting content in a predetermined data format into an internal representation data format and converting it into output representation data suitable for image display or printer output. There is a reference to a process for identifying whether the input data format is HTML, XML, PDF, DOC, RM, VRML, or SGML.

  By the way, in the two-dimensional display of a document, when the screen size is small or the displayed document (or content) is too large to fit on one screen, the document is scrolled and viewed, but both vertical and horizontal scrolling is performed. Operation becomes complicated. Another problem is that it is difficult to understand where you are reading. Therefore, when the document is reduced and displayed so that scrolling does not occur, the characters become too small and difficult to read.

  Therefore, for example, in the case of a vertically long document, for example, a method of scrolling and reading only in the vertical direction is displayed in such a manner that the horizontal size matches (fits) the screen size. In this method, the scrollability is only in the vertical direction while improving the operability while leaving the readability of characters at a moderate reduction ratio. If the reduction ratio in the vertical direction is slightly increased, it becomes easier to read the characters, and since the scroll is only in the vertical direction, the operation becomes easy and it is easy to determine where the current display is. However, even such a method has a limit when the screen size is extremely small, such as a portable terminal, or when the document size itself is large. In addition, since the screen size is extremely small in a mobile terminal or the like, a dedicated page for the mobile terminal is separately prepared for displaying a Web homepage or the like.

  In order to facilitate scrolling operations, for example, when displaying at a magnification that fits in the horizontal direction and scrolling only in the vertical direction, sufficient display size is not obtained for large documents, complex documents, etc., and characters are difficult to decipher There is a case. If the magnification is increased in order to eliminate this, a scroll operation is required in the horizontal direction, and the operability is deteriorated. When preparing another homepage for a portable terminal, two managements are required even if the same contents are displayed. Or, the burden of managing received documents and sending documents is large, such as having to prepare pages with simplified contents for mobile terminals.

  An object of the present invention is to realize readability of a sentence even with a limited screen size.

  (1) When an image represented by document information is displayed on a two-dimensional display surface, the image data is separated into a character area and other background by character / background separation or layout analysis processing, In the area, the size of one character is measured by performing OCR (Optical Character Reader) processing or character rectangle analysis processing, the display magnification is automatically calculated from the character size, and the two-dimensional display surface of the image represented by the image data A scaling method for display data, wherein scaling processing is performed on at least one of the two-dimensional directions at the display magnification.

  As a result, it is possible to realize a character display that is easy to read even at a reduction magnification that is difficult to read by reduction with a fixed aspect ratio. For example, when the input data is image data ((c) in FIG. 6), in (g) character / background separation in FIG. 6, (m) foreground (character) portion image in FIG. (N) background image). Separation is performed by (o) extracting lines by dot distribution analysis by the layout analysis unit in FIG. 6 or (p) OCR processing unit in FIG. When the character area is determined, the rectangular size of one character is measured in the (s) character size detector in FIG. Thereafter, the lowest magnification at which characters can be read is calculated in the (u) optimum size determination unit in FIG. The determination is made based on information that, for example, at least 24 × 48 dots (dots) are necessary to express Japanese kanji. This minimum size is rewritten with an empirical value depending on the user's visual acuity and the characteristics of the display device. By this process, the optimum display magnification is automatically calculated.

  (2) When displaying the image represented by the document information on the two-dimensional display surface, the image file (MRC, PDF) in which the background / photo part and the character part are separated and stored is discriminated from the character part / photo part, Further, the character portion is subjected to OCR processing or character rectangle analysis processing to measure the size of one character, the display magnification is automatically calculated from the character size, and the two-dimensional display surface of the image represented by the image file data is displayed. A scaling method for display data, wherein scaling processing is performed on at least one of the two-dimensional directions at the display magnification.

  The input / background / photo / text part was separated and saved in advance, like MRC (Mixed Raster Contect: ITU-T T.44 or ISO / IEC 16485: 2000) and high compression PDF (Portable Document Form). In an image file, even if it does not have a character / background separation or layout analysis function uniquely, it distinguishes a character portion / photograph portion and performs effective document display. For example, when the input data is MRC ((b) in FIG. 6), the MRC itself is separated and stored in the character part and the background part, so this information is used as it is. There is no need for character / background separation processing, a simple configuration is required, and the processing time is reduced.

(3) In displaying the image represented by the document information on the two-dimensional display surface,
The image data is separated into a character area and other background by character / background separation or layout analysis processing, and the character data is subjected to OCR processing or character rectangle analysis processing for the size of one character. , And automatically calculating a display magnification from the character size, scaling at least one direction of the two-dimensional direction of the two-dimensional display surface of the image represented by the image data with the display magnification,
An image file (MRC, PDF) in which the background / photo part and character part are separated and stored is discriminated from the character part / photo part, and the character part is subjected to OCR processing or character rectangle analysis processing for one character. The display magnification is automatically calculated from the character size, and at least one of the two-dimensional directions of the two-dimensional display surface of the image represented by the data of the image file is scaled by the display magnification. The scaling method of display data.

  (4) The document information that does not conform to the scaling process controls an application that processes the document information, converts the document information into a document that conforms to the scaling process, and performs the scaling process. 1) A method for scaling display data according to any one of (3).

  When displaying data of another application, the application itself needs to have the display function according to the present invention. Even if the application side does not have the function of the present invention, the same display can be performed using the display function of this embodiment. It can be carried out. For example, (h) the external application control unit in FIG. 6 controls the (i) external application in FIG. 6, and (j) the print driver or the paste control unit in FIG. 6 performs printing, via the clipboard, or OLE ( Object Linking and Embedding) Once converted to image data by automation control (ActiveX Control). After that, scaling processing is performed.

  (5) Analyzing document layout information to detect photographs and diagrams in the document, scaling the photos and diagrams so that the aspect ratio is constant, and scaling other document information by the scaling process. The method for scaling display data according to any one of (1) to (4) above. As a result, it is possible to correctly display a photograph, a figure, or the like whose contents are difficult to convey accurately when the aspect ratio changes. For example, it is possible to display only a photograph part at a fixed aspect ratio as shown in FIG. 8B without extending as shown in FIG. In the case of data in which characters / photos and the like are structured from the beginning as in (a) HTML of FIG. 6, an image portion that does not overlap with characters is extracted in (f) structure analysis unit of FIG. 6. In the case of image data, the position of a photograph and a figure is specified by (o) layout analysis unit in FIG. 6 and (q) photo extraction unit in FIG. At the time of display, the (w) photo display position control unit in FIG. 6 calculates an appropriate display position and magnification to realize the display in FIG.

  (6) The display data scaling method according to (5) above, wherein the photograph and figure are displayed separately in a field (layer) separate from other document information. Thereby, a user can change to arbitrary magnitude | sizes or can move a display position freely. If the photo tends to be displayed smaller, the photo portion is displayed in a separate layer, so that the user can display it at any size and at any display position. For example, as shown in FIG. 9, a move / resize (size change) knob is displayed at the time of selecting a photo portion, and movement to an arbitrary position and resize are made possible. Also, after moving, the original photo position is displayed in monochrome so that it can be easily understood.

  (7) The display data scaling described in any one of (1) to (5) above, wherein the document is divided into blocks by extracting areas such as characters, photographs, and tables and displayed for each block. Method. Thereby, the operability is improved by scrolling only one side, and the reduction magnification is also increased, so that the legibility of characters can be improved. When the screen size is small or the document itself is fine and characters cannot be read sufficiently unless the magnification is larger, for example, the document content is divided into blocks as shown in FIG. Display in order. 6 is divided into blocks by (o) layout analysis unit and (q) photo extraction unit of FIG. 6, and is re-laid out in an appropriate order by (x) block order inference unit of FIG. As a result, the contents of the document can be browsed efficiently even by scrolling in only one direction. If there is an error in the block display order, the user can arbitrarily change the playback order.

  (8) The display data scaling method described in (7) above, wherein information that clearly indicates the connection between blocks is added to the display data under magnification. The document layout may be changed, and there is a possibility that the user may lose sight of what is displayed. In this embodiment, for example, a mark representing the relationship between each block on the display screen in method 1 in FIG. Is displayed. For example, if the next arrow points to the right, it indicates that the next scrolled and displayed part is a block originally located on the right side. The same applies to the forward arrow. When the screen is directed to the left side, it means that the left block is displayed when scrolling back. Further, as in method 2 in FIG. 11, an entire display window is displayed separately from the main screen to display the block reproduction order and the current display position.

  (9) The display data scaling method according to (8), wherein character information in the character area is analyzed to determine the reproduction order of the character blocks, and information that clearly indicates the connection between the blocks is generated. For example, when the reproduction order of the character blocks is unknown as shown in FIG. 12, (r) sentence syntax shown in FIG. 6 based on the TEXT information extracted by performing character recognition in advance in (p) OCR processing section of FIG. The analysis unit determines the probability of the document connection between the blocks. Alternatively, (x) block order estimation unit in FIG. 6 determines the block order from the character layout, the change in character size, and the chapter and item title extracted by the chapter / item title extraction unit in FIG. Guess at. For example, if the end of a block ends with “.”, The next block is inferred from the possibility that the beginning of a sentence (indent) or the beginning of a chapter or item is highly likely.

  (10) When the character area does not fit in the setting area in the two-dimensional direction of the two-dimensional display surface, the character area in the character area is extracted and a line break is performed for each line to generate a character use area image. The display data scaling method according to any one of (1) to (9), wherein rearrangement is performed. In other words, it is easy to read the text so that it fits within the screen, and a display with good operability is provided by one-way scrolling. This is a method for avoiding a case where the size of one block of the character area is large and a sufficient display magnification cannot be obtained. As shown in FIG. 13, lines are extracted and divided into lines. Then, one line is divided into two at an arbitrary position (a place where characters are not divided), and the latter half is folded to the next line. Thereby, display with sufficient character size is realizable.

  (11) When the document information to be displayed is JPEG (Joint Photographic Experts Group), by expanding only the DC component and the low frequency component, the horizontal n / 8 times, the vertical m / 8 times, and n and m are integers. The method for scaling display data according to any one of (1) to (3), wherein scaling of an arbitrary aspect ratio is performed. This is a method for acquiring a reduced image having an arbitrary aspect ratio, and is effective when the input image is JPEG or JPEG2000. In JPEG compression, compression processing is performed in units of 8 × 8 pixel (pixel) blocks. In the processing, this is decomposed into frequency components and processed. At this time, the rough feature of the image is separated into a low frequency component, and the fine feature of the image is separated into a high frequency component. For example, it is known that if only the lowest frequency component (= DC component) is expanded and extracted, an image in which features of 1/8 times are condensed can be acquired. This embodiment applies this, and obtains and displays a high-quality image at a high speed as a reduced image having an arbitrary aspect ratio (for example, an image having a width of 1/8 times and a height of 2/8 times). One of the features of this method is that the target image can be acquired at high speed because the number of operations for JPEG decompression can be greatly reduced. Normally, smoothing reduction must be performed after JPEG is expanded to its original size, but if it is processed during JPEG expansion, not only can a reduced image be obtained directly, but also the calculation time for JPEG expansion can be greatly reduced. Another feature of this method is that a high-quality reduced image can be obtained. Since the features decomposed into frequency components by DCT (Discrete Cosine Transform) conversion of JPEG are used, the features of the image are captured, and a high-quality image that is smoothed and reduced can be obtained. Taking advantage of these features, the easy-to-read character display that is the object of the present invention can be realized more easily.

  The same applies to the case of JPEG2000. In JPEG2000, a tiling process is originally performed so that an image of an arbitrary resolution can be easily acquired. By utilizing this, the number of processing tiles in the horizontal direction and the number of process tiles in the vertical direction are controlled to obtain a high-quality reduced image having an arbitrary aspect ratio at high speed.

  Other objects and features of the present invention will become apparent from the following description of embodiments with reference to the drawings.

  FIG. 1 shows the external appearance of a full-color digital multi-function copier MF1 that receives a print instruction from the personal computer PC according to the first embodiment of the present invention. The full-color copying machine MF1 is roughly constituted by units of an automatic document feeder (ADF) 120, an operation board 10, a color scanner 100, and a color printer 200. The operation board 10 and the color scanner 100 with the ADF 120 are units that can be separated from the printer 200. The color scanner 100 has a control board having a power device driver, sensor input, and controller, and an engine controller ( CPU 301: Communicates directly or indirectly with FIG. 2) to read the document image under timing control.

  A controller board (400: FIG. 2) to which a scanner 100, a printer 200, and an engine (300: FIG. 2) including an image input / output device (302: FIG. 2) are connected is connected to a LAN (Local Area Network) connected to a personal computer PC. Is connected, and a facsimile control unit (FCU 417: FIG. 2) is connected to a switch PBX connected to a telephone line PN (facsimile communication line).

  FIG. 2 shows the configuration of the image processing system of the multifunction copying machine MF1 shown in FIG. The multi-function copier MF1 includes an engine 300 that performs document image reading and color printing, a controller board 400, and an operation board 10. The engine 300 includes a CPU 301 that controls image reading and printing processes, the above-described color scanner 100, the above-described printer 200, and an image input / output processing 302 that includes an ASIC (Application Specific IC).

  The reading unit 110 of the scanner 100 includes a CPU, a ROM, and a RAM, and the CPU 100 controls the entire scanner 100 by writing a program stored in the ROM and executing the program. Further, it is connected to the process control CPU 301 via a communication line, and performs an operation instructed by transmission / reception of commands and data. The CPU in the reading unit 110 performs detection and ON / OFF control of a filler sensor (document detection sensor), a base point sensor, a pressure plate switch, a cooling fan, and the like. In the reading unit 110, a scanner motor driver is driven by a PWM output from the CPU, generates an excitation pulse sequence, and drives a pulse motor for scanning a document.

  The document image is illuminated by the light output of the halogen lamp 102 (FIG. 2) energized by the lamp regulator, and the reflected light of the document, that is, the optical signal, passes through the plurality of mirrors 103 to 105 and the lens 106 to read R, G, and B. The image is formed on a CCD 107 including three line sensors. The 3-line CCD 107 outputs an analog image signal of each RGB pixel to the AFE 111. The AFE 111 is an image signal processing unit that amplifies an image signal, digitally converts it into image data, and performs shading correction.

  The controller board 400 includes a CPU 402, a document storage control 403 constituted by an ASIC, a hard disk device (hereinafter referred to as HDD) 401, a local memory (MEM-C) 406, a system memory (MEM-P) 409, North Bridge (hereinafter referred to as NB) 408, South Bridge (hereinafter referred to as SB) 415, NIC 410 (Network Interface Card), USB device 411, IEEE 1394 device 412, Centronics device 413 and others. The operation board 10 is connected to the document accumulation control 403 of the controller board 400. A facsimile control unit (FCU) 417 is also connected to the document storage control 403 by a PCI bus.

  The CPU 402 can send / receive document information to / from a personal computer connected to the LAN via the NIC 410 or another personal computer PC via the Internet. In addition, it is possible to communicate with a personal computer PC, a printer, a digital camera, or the like using the USB 411, IEEE 1394 412, and Centronics 413.

  The SB 415, the NIC 410, the USB device 411, the IEEE 1394 device 412, the Centronics device 413, and the MLB 414 are connected to the NB 408 via a PCI bus. As described above, the MLB 414 is a board connected to the engine 300 via the PCI bus. The MLB 414 converts the document data input from the outside into image data (image data), and outputs the converted image data to the engine 300.

  The local memory 406, HDD 401, and the like are connected to the document storage control 403 of the controller board 400, and the CPU 402 and the document storage control 403 are connected via the NB 408 of the CPU chipset. The document accumulation control 403 and the NB 408 are connected via an AGP (Accelerated Graphics Port).

  The CPU 402 performs overall control of the multifunction function copying machine MF1. The NB 408 is a bridge for connecting the CPU 402, system memory 409, SB 415, and document storage control 403. A system memory 409 is a memory used as a drawing memory or the like of the multifunction copying machine MF1. The SB 415 is a bridge for connecting the NB 408 to the PCI bus and peripheral devices. The SB 415 is connected to a card IF 418 for reading and writing an external ROM and an SD memory card (hereinafter referred to as an SD card). The card IF 418 is connected to an SD card read / write device (card reader), and can read data from an SD card attached to the card reader, and can write data to the SD card.

  The local memory 406 is a memory used as a copy image buffer and a code buffer. The HDD 401 is a memory for storing image data, document data, programs, font data, forms, LUT (Look Up Table), and the like. The operation board 10 is an operation unit that receives an input operation from the user and performs display for the user.

  FIG. 2 shows a flow of image data exchanged between the scanner 100 and the printer 200 and the image input / output processing 302. The image input / output processing 302 includes scanner image processing 303 that performs reading γ correction, MTF correction, and the like for each of R, G, and B image data generated when the color original scanner 100 reads an original image. Printer image processing 304 that converts R, G, B image data into c, m, y, k recording color data (print data) that matches the image expression characteristics of the C, M, Y, K color writing of the printer 200. In addition, there is an image processing I / F (Interface circuit) 305 that outputs document read image data RGB to the document storage control 403 and supplies the image data RGB output from the document storage control 403 to the printer image processing 304.

  In the case of monochrome copying, the G image data is output from the scanner image processing 303 to the image processing I / F 305, the image processing I / F 305 outputs the G image data to the printer image processing 304, and the printer image processing 304 is the G image data. Is converted into k recording color data, and if necessary, scaling, image processing, printer γ conversion and gradation processing are performed, and the result is output to the C writing unit 212 of the printer 200. The writing unit 212 modulates or turns on / off the current supplied to the laser light emitting diode of the optical scanning unit for forming the electrostatic latent image of the printer 200 according to the k recording color data output from the image processing 304.

  For color copying, RGB image data output by the scanner image processing 303 is temporarily stored in the local memory 406 or the HDD 401 or registered in the HDD 401 and read out via the image processing I / F 305 and the image storage control 403. Used for copying or printing, or sent to the outside.

  When printing registered image data by the printer 200 or image data received from the outside, the image data is given to the printer image processing 304 via the image accumulation control 403 and the image processing I / F 305. The printer image processing 304 converts the image data into cmyk recording color data, performs scaling, image processing, printer γ conversion and gradation processing as necessary, and outputs the result to the writing unit 212.

  As shown in FIG. 3, in addition to the liquid crystal touch panel 11, the operation board 10 includes a numeric keypad 15, a clear / stop key 16, a start key 17, an initial setting key 18, a mode switching key 19, a test print key 20, and a power key. There are 21. Although not shown, on the left side of the liquid crystal touch panel 11, there is an alphabet keyboard with hiragana added for inputting, setting, and abbreviated registration for URL, mail text, file name, folder name, and the like.

  The power key 21 is an operation key for instructing switching from the energy saving mode (the sleep mode or the low power mode) to the standby mode in which image printing is possible and vice versa. If the power key 21 is pressed once when the energy saving mode is set, the energy saving mode is switched to the standby mode. When the power key 21 is pressed once in the standby mode, the standby mode is switched to the sleep mode. The test print key 20 is a key for printing only one copy regardless of the set number of print copies and confirming the print result.

  By pressing the initial setting key 18, the initial state of the machine can be arbitrarily customized. It is possible to arbitrarily set the state set when the transition time to the energy saving mode is set, the paper size stored in the machine is set, or the copy function reset key is pressed. When the initial setting key 18 is operated, an "initial value setting" function for setting various initial values, an "ID setting" function, a "copyright registration / setting" function, an "usage record output" function, etc. A selection button to specify is displayed.

  On the liquid crystal touch panel 11, various function keys and messages indicating the operation states of the engine 300 and the controller board 400 are displayed. The LCD touch panel 11 is used for selecting and executing the “copy” function, “scanner” function, “print” function, “facsimile” function, “store” function, “edit” function, “register” function, and other functions. A function selection key 14 representing is displayed. An input / output screen determined for the function designated by the function selection key 14 is displayed. For example, when the “copy” function is designated, the function keys, the number of copies, and the state of the image forming apparatus are shown as shown in FIG. Messages 12 and 13 are displayed. Among the function keys 12, print color designation keys “black (BK)”, “full color”, “automatic color selection”, “blue (C)”, “red (M)” and “yellow (Y)” designation There is a key. When the operator touches a key displayed on the liquid crystal touch panel 11, the operation board 10 reads it as an operator input, and highlights the key indicating the selected function in gray indicating that it is being designated. Further, when it is necessary to specify the details of a function (for example, the type of page printing), a detailed function setting screen is popped up by touching a key. Thus, since the liquid crystal touch panel 11 uses a dot display, it is possible to graphically perform an optimal display at that time.

  FIG. 4 shows a circuit block of the operation board 10. The main body of the electric control system of the operation board 10 communicates with the CPU 402 of the controller board 400, reads the input of the operation board 10, and controls the display on the operation board 10, and stores the control program of the CPU 1. ROM 2, RAM for temporarily storing data during control, VRAM 7 for storing drawing data of the liquid crystal touch panel 11, drawing timing control of the liquid crystal touch panel 11 connected to the VRAM 7, touch input detection, etc. There are a liquid crystal display controller (LCDC) 6, a clock IC 5 for generating time data, and the like. A liquid crystal touch panel 11 having a CFL light source as a backlight 9 is connected to the LCDC 6. The CPU 1 is further connected to an inverter 8 that drives the CFL backlight 9, a key matrix of operation key groups 15 to 21, an LED matrix of display LEDs, an LED driver that drives these LEDs, and the like. Further, a nonvolatile RAM (NVRAM) 4 for storing an image processing mode is connected to the data bus to which the CPU 1 is connected.

  The CPU 1 of the operation board 10 corresponds to the user's operation on the operation board 10, reads the numeric key press, generates the input numeric data, reads the start key press, and sends a start instruction to the controller board 400. It performs normal operation reading and display output control of copiers such as transfer and reading of paper size switching input.

  In the ROM 2, in addition to the operation program of the CPU 1, the distribution of on / off signals instructing light emission / non-light emission of each pixel and the display gradation of the pixel block of a pixel block composed of a plurality of pixels in the horizontal and vertical directions are displayed. A pattern group composed of a plurality of light emission pattern data expressing different gradations is stored. The pattern group stored in the ROM 2 is written to the pattern register allocated to the memory area of the RAM 3 by the CPU 1 and is used for subsequent gradation display when the operation voltage is applied to the operation board 10. The display data (gradation data) is written into the VRAM 7 and is read out from the VRAM 7 by the LCDC 6 in synchronization with the raster display synchronization signal of the LCD 11 and used as a read address for the pixel block data of the pattern register. The pixel block data read from the pattern register is output from the LCDC 6 to the LCD 11 in accordance with the raster display synchronization signal of the LCD 11.

  FIG. 5 shows an outline of data processing of the “display application” (DCA) of the personal computer PC according to the application program installed in the personal computer PC connected to the copying machine MF1 and implementing the present invention. The data processing function of the “display application” (DCA) is shown in block sections, and corresponding processing steps and processing blocks in FIGS. 5 and 6 are given the same alphabetic symbols. In the “display application” (DCA), when the image to be displayed is wider than the set area on the display surface and the image is reduced, the character area is set to a magnification that can identify the character, and information loss in the reduced display is minimized. This keeps the character readable. As a result, it is possible to realize a character display that is easy to read even at a reduction magnification that is difficult to read by reduction with a fixed aspect ratio. Furthermore, a sufficient display size is secured by performing display in units of blocks. If the size of the character block is still large and the display area is insufficient, line extraction is performed and folding is performed on a line-by-line basis, thereby increasing the display character size while keeping the scroll direction in one direction to make the text easier to read.

  In the document memory of the personal computer PC connected to the copier MF1, images (image data) read by the scanner 100 of the MF1, document information received via LAN or Internet communication, document information received by facsimile, generated or edited by the personal computer PC The printed document information is saved, the document file designated by the user is opened, displayed on the display of the personal computer PC by being developed into display data by the display application (DCA). The user can instruct the multifunction copying machine MF1 to print out the image file in the document memory from the operation board of the personal computer PC. Further, it is possible to instruct reading of an original image using the original scanner 100 and the ADF, and further, it is possible to instruct facsimile transmission.

  The personal computer PC opens the document file designated by the user with the display data input (e) in FIG. 5 ((e) input control unit in FIG. 6), and refers to the data format of the image data in the file. Proceed to processing corresponding to the data format.

  When the data of the document file is image data, it is separated into a character portion and other background by character / background separation or layout analysis processing. Further, the size of one character is measured by performing OCR processing or character rectangle analysis processing on the character area, and a magnification that does not impair readability is automatically calculated from the character size, and the display magnification is automatically determined. That is, when the input data is image data ((c) in FIG. 6), in (g) character / background separation in FIG. 6, (m) foreground (character) portion image in FIG. (N) background image). Separation is performed by (o) extracting lines by dot distribution analysis by the layout analysis unit in FIG. 6 or (p) OCR processing unit in FIG. When the character area is determined, the rectangular size of one character is measured in the (s) character size detector in FIG. Thereafter, the lowest magnification at which characters can be read is calculated in the (u) optimum size determination unit in FIG. The determination is made based on information that, for example, at least 24 × 48 dots are necessary to express Japanese kanji. This minimum size is rewritten with an empirical value depending on the user's visual acuity and the characteristics of the display device. By this process, the optimum display magnification is automatically calculated.

  If the document file data is an image file in which the background / photo part / character part is separated and saved in advance, such as MRC and high-compression PDF, the character part / photo part is distinguished, and the display magnification is effective. Calculate automatically. That is, when the input data is MRC ((b) of FIG. 5), since the MRC itself is separated and stored in the character part and the background part, this information is used as it is, and the character part is As in the case where the input data is an image, the (p) OCR processing unit in FIG. 6 recognizes characters, and the (s) character size detection unit in FIG. 6 measures the rectangular size of one character. Thereafter, the lowest magnification at which characters can be read is calculated in the (u) optimum size determination unit in FIG. The determination is made based on information that, for example, at least 24 × 48 dots are necessary to express Japanese kanji. This minimum size is rewritten with an empirical value depending on the user's visual acuity and the characteristics of the display device. By this process, the optimum display magnification is automatically calculated.

  When the document file data is displayed by another application or created by another application, the external application control unit shown in FIG. 6 (h) controls the external application shown in FIG. 6 (j) The image data is once converted into image data via the print driver, via the clipboard, or via OLE automation control by the print driver or paste control unit. Thereafter, similarly to the case where the user-specified document file is image data, the display magnification is calculated after the separation of the character portion and the other background.

  In the case of data in which characters / photos and the like are structured from the beginning as in (a) HTML of FIG. 6, an image portion that does not overlap with characters is extracted in (f) structure analysis unit of FIG. 6. In the case of image data, the position of a photograph and a figure is specified by (o) layout analysis unit in FIG. 6 and (q) photo extraction unit in FIG. At the time of display, the (w) photo display position control unit in FIG. 6 calculates an appropriate display position and magnification so that the photographs and figures are displayed in a constant aspect ratio, thereby realizing the display in FIG. . That is, in a document in which photographs and drawings are mixed, if the display is performed at a magnification with different aspect ratios such as a character area, the photographs are also undesirably elongated as shown in FIG. Therefore, as shown in FIG. 8B, only the photo portion is displayed at a fixed aspect ratio. As a result, it is possible to correctly display a photograph, a figure, or the like whose contents are difficult to convey accurately when the aspect ratio changes.

  The (w) photo display position control unit in FIG. 6 further displays the photos / drawings in another field (layer). As shown in FIG. 9, when a photograph part is selected, a move / resize knob is displayed to enable movement and resize to an arbitrary position. Also, after moving, the original photo position is displayed in monochrome so that it can be easily understood.

  When the screen size is small or the document itself is fine and characters cannot be read sufficiently unless the magnification is larger, if the user instructs block division by the operation board or mouse input, (o) layout analysis in FIG. 6 and (q) the photo extraction unit in FIG. 6 divides the image into blocks as shown in FIG. 10 by extracting areas such as characters, photos, and tables, and the (x) block order inference unit in FIG. As shown in FIG. 11, the images are displayed in order in an appropriate order. As a result, the contents of the document can be browsed efficiently even by scrolling in only one direction. If there is an error in the display order of the blocks, the user can arbitrarily change the playback order by an operation board or mouse input. Since the blocks are displayed continuously according to the display order of the blocks, the display can be confirmed by scrolling only in one direction (x or y) on the two-dimensional (x, y) display surface, so the display operability is good and the reduction magnification is also high. Since it becomes higher (lower reduction ratio), it is possible to improve the legibility of characters.

  The (x) block order inference unit in FIG. 6 adds an arrow mark display and a direction display to the order display so that the connection between the blocks can be understood. As shown in FIG. 11, in the method 1 of FIG. 11, a mark representing the relationship between before and after is displayed for each block on the display screen. For example, if the next arrow points to the right, it indicates that the next scrolled and displayed part is a block originally located on the right side. The same applies to the forward arrow. When the screen is directed to the left side, it means that the left block is displayed when scrolling back. In the method 2 in FIG. 11, the entire display window is displayed separately from the main screen so that the block reproduction order is displayed and the current display position is known.

  As shown in FIG. 12, when the reproduction order of the character blocks is unknown, (r) the sentence syntax analysis unit in FIG. 6 based on the character information extracted by performing character recognition in advance in the (p) OCR processing unit in FIG. Thus, the probability of the connection of the documents between the blocks is determined. 6 (x) block order estimation unit in FIG. 6 shows the order of blocks based on the character layout and the change in character size and the chapters and item titles extracted by the chapter (t) and item title extraction unit in FIG. Guess at. For example, if the end of a block ends with “.”, The next block is inferred from the possibility that the beginning of a sentence (indent) or the beginning of a chapter or item is highly likely.

  If the size of one block is large and sufficient display magnification cannot be obtained as shown in FIG. 13, the (x) block order number theory unit and (y) character display position control unit in FIG. Extract lines as shown and divide into lines. Then, one line is divided into two at an arbitrary position (a place where characters are not divided), and the latter half is folded to the next line. Thereby, a sufficient character size can be realized. In other words, by extracting lines, making line breaks in units of lines, and re-laying out the image, the text fits within the screen and is easy to read, and the display is easy to operate with one-way scrolling.

  When the data of the document file is JPEG, only the DC component and the low frequency component are expanded, so that two reduced reduced images that are easy to read characters having an arbitrary aspect ratio of horizontal n / 8 times and vertical m / 8 times are read. In JPEG compression, compression processing is performed in units of 8 × 8 pixel blocks. Internally, this is decomposed into frequency components and processed. At this time, the rough features of the image are separated into low-frequency components, and the fine features of the image are separated into high-frequency components. For example, it is known that if only the lowest frequency component (= DC component) is expanded and extracted, an image in which features of 1/8 times are condensed can be acquired.

  By applying this, the character / background separation unit in FIG. 6 determines the display size in the case of JPEG data as shown in FIG. 5 in some detail, and reduces the image with an arbitrary aspect ratio. The image is developed and displayed as (for example, an image of 1/8 horizontal and 2/8 vertical). As a result, the number of operations for JPEG decompression can be greatly reduced, so that an image with a desired reduction magnification can be developed at high speed. Normally, smoothing reduction must be performed after JPEG is expanded to its original size, but if it is processed at the time of JPEG image expansion, it is possible not only to obtain a reduced image directly but also to greatly reduce the calculation time of JPEG expansion. In addition, since the feature decomposed into frequency components by JPEG DCT conversion is used, the feature of the image is captured, and a high-quality image that is smoothed and reduced can be obtained. Easy-to-read characters can be displayed more easily. One display example is shown in FIG.

  The same applies to the case of JPEG 2000. In JPEG 2000, tiling processing is originally performed so that an image with an arbitrary resolution can be easily acquired. By utilizing this, the number of processing tiles in the horizontal direction and the number of process tiles in the vertical direction are controlled to obtain a high-quality reduced image having an arbitrary aspect ratio at high speed.

  The maximum display size is obtained in the display control unit (z) in FIG. 6, and the magnification processing unit (v) in FIG. FIG. 7 shows an example, and it can be seen that, with fixed aspect ratio reduction, characters become too small and difficult to read, but by reducing in only one direction, the characters can be reduced according to the screen size while maintaining readability.

  Although the above-described embodiment is implemented by a personal computer PC and controls the image size displayed on the display of the personal computer, the present invention can be applied to other devices such as operation boards, operation terminals, portable terminals, 2 The present invention can be similarly applied to a device that displays a document on a two-dimensional display device such as a CRT, a liquid crystal display, a plasma display, an LED surface array display, and an EL display, such as a cellular phone with a three-dimensional display.

1 is a longitudinal sectional view showing an outline of the mechanism of a multifunctional copying machine MF1 that receives a print instruction from a personal computer PC according to a first embodiment of the present invention; FIG. 2 is a block diagram showing a configuration of an image processing system in the copying machine MF1 shown in FIG. FIG. 2 is an enlarged plan view showing a part of an upper surface of an operation board 10 of the copying machine MF1 shown in FIG. 3 is a block diagram showing an outline of a configuration of an electric system of the operation board 10. FIG. 4 is a flowchart showing an outline of data processing of a “display application” (DCA) of a personal computer PC in accordance with an application program installed in the personal computer PC connected to the copying machine MF1 and implementing the present invention. It is a block diagram which shows the data processing function of the "display application" (DCA) shown in FIG. 5 by a block division. (A) is a plan view showing an example of a display on the display of a personal computer PC before the present invention is applied, and (b) is a reduction in the vertical direction according to the present invention to facilitate user character recognition. It is a top view which shows the displayed. (A) is a plan view showing an example of display with reduced reduction in the vertical direction in order to facilitate user's character recognition, and (b) is a display in which only the photographic block has the same vertical and horizontal scaling to eliminate distortion of the photographic image. FIG. It is a top view which shows the image block division of the original image before applying this invention, and the scaling process by the scaling process of this invention. It is a top view which shows the original image before applying this invention, the image block division of the scaling process process by the scaling process of this invention, and the arrangement processing of a block unit. It is a top view which shows the original image before applying this invention, the image block division of the scaling process by the scaling process of this invention, and an operation assistance mark display. It is a flowchart which shows the block order determination process applied to the image which cannot specify the connection order of a character block, and this image. It is a flowchart which shows the character line edit process of a character block with a wide block width. It is a top view which shows two examples of the scaled JPEG image.

Claims (13)

  When displaying the image represented by the document information on the two-dimensional display surface, the image data is separated into the character area and the other background by character / background separation or layout analysis processing. The size of one character is measured by performing OCR processing or character rectangle analysis processing, the display magnification is automatically calculated from the character size, and at least one of the two-dimensional directions of the two-dimensional display surface of the image represented by the image data is displayed. A display data scaling method for scaling a direction at the display magnification.   In displaying an image represented by document information on a two-dimensional display surface, an image file in which a background / photo part and a character part are separated and stored is discriminated from a character part / photo part. A character rectangle analysis process is performed to measure the size of one character, the display magnification is automatically calculated from the character size, and at least one direction of the two-dimensional direction of the two-dimensional display surface of the image represented by the data of the image file A scaling method for display data, wherein scaling processing is performed at the display magnification. In displaying the image represented by the document information on the two-dimensional display surface,
The image data is separated into a character area and other background by character / background separation or layout analysis processing, and the character data is subjected to OCR processing or character rectangle analysis processing for the size of one character. , And automatically calculating a display magnification from the character size, scaling at least one direction of the two-dimensional direction of the two-dimensional display surface of the image represented by the image data with the display magnification,
The image file in which the background / photo part and the character part are separated and stored is distinguished from the character part / photo part, and the character part is subjected to OCR processing or character rectangle analysis processing to measure the size of one character. The display magnification is automatically calculated from the character size, and at least one direction of the two-dimensional direction of the two-dimensional display surface of the image represented by the data of the image file is scaled at the display magnification.
A scaling method for display data.   4. The document information that does not conform to the scaling process is controlled by converting an application that processes the document information into a document that conforms to the scaling process, and performs the scaling process. A scaling method for display data according to any one of the above.   The layout information of the document is analyzed to detect the area of the photograph or figure in the document, the photograph or figure is scaled to have a constant aspect ratio, and the other document information is scaled by the scaling process. 5. A method for scaling display data according to any one of 1 to 4.   The display data scaling method according to claim 5, wherein the photograph and figure are displayed separately in a separate field from other document information.   The display data scaling method according to any one of claims 1 to 5, wherein the document is divided into blocks by extracting regions such as characters, photographs, and tables, and each block is displayed.   The display data scaling method according to claim 7, wherein information that clearly indicates connection between blocks is added to the display data under scaling.   9. The method for scaling display data according to claim 8, wherein character information in the character area is analyzed to determine the reproduction order of the character blocks, and information that clearly indicates the connection between the blocks is generated.   When the character area does not fit in the set area in the two-dimensional direction of the two-dimensional display surface, the character line in the character area is extracted, the line is broken in line units, and the character use area image is rearranged. A method for scaling display data according to any one of claims 1 to 9.   When the document information to be displayed is JPEG, only the direct current component and the low frequency component are expanded to change the aspect ratio n / 8 times, m / 8 times, and n and m are integers. A method for scaling display data according to any one of claims 1 to 3.   A program for executing the scaling method according to any one of claims 1 to 11 on a computer. A storage medium on which the program according to claim 12 is recorded.

Images