JP2006185336A - Image processor, method and computer program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reduce misoperation such as copying of an erroneous data kind to reduce a work burden on a user. <P>SOLUTION: This image processor has: a display means drawing and displaying graphic data; a range designation means designating a range on the display means; a means specifying only data present inside the designated range of the graphic data; a data kind designation means designating the kind of the graphic data; and an output means outputting the graphic data specified by the range and the kind. When designating the range, different visual effect is generated according to the kind designation of the data. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image processing apparatus, method, and computer program for converting image data read by an image processing apparatus such as a copying machine into vector data that can be reused by so-called document creation application software such as a word.

  In recent years, paperless offices are rapidly becoming paperless as environmental problems are screamed. In other words, a document management system has been built in which a paper document accumulated with a binder or the like is read by a scanner, converted into image data, and accumulated as a database in an image storage device.

  Furthermore, in order to reuse a part of a document such as a table or figure written on paper in a document creation application, the contents of the image data read by the scanner are analyzed and elements such as characters, line drawings, natural images, tables and backgrounds are analyzed. The technology to convert into graphic data with suitable expressions has been developed. Examples of suitable expressions here include expressions based on character code and font information for characters, image expressions or vector representations of character outlines, images or vector expressions for line drawings, image expressions for natural images, and tables. If there is, it changes according to the type of document element such as vector expression or character code + table structure expression, etc. In addition, when reusing on a document creation application, the appropriate expression changes from time to time depending on the user's arbitrary intention . For example, if the character recognition rate is poor, a vector representation may be desired rather than a character code representation.

  That is, when image data is converted into graphic data, the creation and holding of graphic data that expresses a plurality of document elements provides a desired data immediately in response to various user requests. It is desirable to be possible.

  In order to use a part of such graphic data, it is necessary to consider the type of data in addition to the range selection by position. With regard to such a selection method, the document processing apparatus described in Patent Document 1 uses graphic data, image data, character data, and the like to be displayed in a certain document frame when editing a document using a display screen. Was stored and the user could read at least one of the data and copy it to another part of the document for reuse in editing the document. In particular, when reading out data within a frame, if there are multiple types of data (for example, graphic data, image data, etc.), a selection screen for the data to be extracted is displayed, and one or more of them can be specified. , Allowing the user to select the data type.

Japanese Patent Publication No. 5-47860

  In the case of the document processing apparatus described in Patent Document 1, a data selection screen is displayed every time an instruction to read out data in a frame is given for copying, and the user has to select a desired data format each time. there were.

  As a countermeasure, it is possible to avoid the selection every time by setting the data format in advance, but in that case, it is difficult to understand that the existing setting matches the desired selection, and there is a risk that erroneous data will be extracted. It was.

  The present invention has been made in view of the above points, and it is an object of the present invention to reduce erroneous operations such as copying an incorrect data type and to reduce a user's work load.

An image processing apparatus according to the present invention includes a display means for drawing and displaying graphic data, a range specifying means for specifying a range on the display means, and means for specifying only data within the specified range of the graphic data; A data type designating unit for designating the type of the graphic data, and an output unit for outputting the graphic data specified by the range and the type. When the range is designated, the visual type varies depending on the data type designation. It is characterized in that it produces an effect.
An image processing method according to the present invention includes a step of drawing graphic data and displaying it on a display means, a step of designating a range on the display means, and a step of specifying only data within a designated range of the graphic data; And a step of designating a type of the graphic data and a step of outputting graphic data specified by the range and type, and when the range is designated, a different visual effect is produced depending on the type designation of the data. Characterized by points.
The computer program according to the present invention includes a process for drawing graphic data and displaying it on a display means, a process for specifying a range on the display means, a process for specifying only data within a specified range among the graphic data, A process of designating the type of graphic data and a process of outputting graphic data specified by the range and type are executed by a computer, and when the range is designated, a different visual effect is obtained depending on the type of data. It is characterized by the points that occur.

  According to the present invention, when a range of graphic data is specified and the data in the range is copied to a means for use in another application, even if multiple types of data exist in the same range, Since the user can immediately know whether the data type is selected and copied, erroneous operations such as copying the wrong data type can be reduced, and the work burden on the user can be reduced.

  Preferred embodiments of the present invention will be described below with reference to the accompanying drawings. FIG. 1 shows the configuration of a system to which the present invention is applied. 101 is a computer device, 102 is a display device that displays image information created by the computer device on a screen such as a cathode ray tube or a liquid crystal display, 103 is a printer device that prints image information created by the computer device 101 on paper, and 104 is a computer An operation device such as a mouse or a keyboard for sending instructions to the device 101, and a scanner device 105 for photoelectrically converting the content of the paper into image data and inputting the image data to the computer device 101.

  FIG. 2 shows an example of the internal hardware configuration of the computer apparatus 101. 201 is a CPU that controls most of the apparatus by executing software, 202 is a memory that temporarily stores software and data executed by the CPU 201, 203 is a hard disk that stores software and data, 204 is a keyboard, mouse, scanner, etc. This is an I / O unit that receives input information and outputs information to a display or a printer.

  FIG. 3 shows a module configuration example of software executed by the computer apparatus 101. Reference numeral 301 denotes computer basic software (OS). The OS 301 has a so-called window system, and the application software that operates on the OS 301 can perform so-called GUI operation that expresses image information on a display and has a graphical operation function using a mouse or the like. Examples of such OS include Microsoft Windows (registered trademark) (TM) and MacOS X (TM).

  Reference numerals 302 to 305 denote applications that operate on the OS 301. Reference numeral 302 denotes a scan application, 303 denotes a graphic data generation application, 304 denotes a graphic editing application, and 305 denotes a clipboard.

  The flowchart in FIG. 4 describes the operation when processing is performed using the applications 302 to 305. In step S401, image data of the page is generated from one page of the paper document input to the scanner by the user using the scan application 302. The image data is appropriately compressed and stored as a file in the hard disk 203 of FIG.

  In step S402, the graphic data generating application 303 is used to convert the image data of the document page into editable graphic data. The graphic data mentioned here is data for composing visual information of one page, and in particular includes not only image data but also vector drawing commands such as paths and figures, or rendering information of character fonts. Point to the data. In addition, editable means a state in which an image, a vector drawing command, a character code, and the like can be changed. In this case, in particular, the document data originally provided as one image is a character, It is divided into objects such as line drawings, natural images, tables, and backgrounds, and each of them is represented by graphic data in an optimal expression format, so that the layout can be changed as a document, and characters, line drawings, etc. can be changed as document objects. It means a state where it is easy to deform.

  In step S <b> 403, the user selects a part or all of the graphic data generated by the graphic data generation application 303 with a mouse or the like, and copies the selected data to the clipboard 305.

  In step S <b> 404, the user instructs the graphic editing application 304 to import the graphic data held on the clipboard 305 into the graphic editing application 304.

  In step S405, the graphic editing application 304 is used to output graphic data processed and edited by the user as paper from the printer. Processing / editing means, for example, changing a part of graphic data representing a line drawing or adding a part.

  Of the software used in the processing described in FIG. 4, the scan application 302 and the graphic editing application 304 are not related to the essence of the present invention, and an existing application based on a known technique can be used, so that the description thereof is omitted. To do.

  Similarly, the clipboard 305 is not related to the essence of the present invention, but its mechanism will be briefly described below. The clipboard 305 is a kind of buffer used for exchanging data between applications running on the OS 301. An application that can communicate with the clipboard 305 can copy its own data to the clipboard. At this time, the clipboard 305 holds data and an ID for identifying the data type. Another application can read the data held in the clipboard 305 and use it as its own data. At this time, whether or not the data type can be read by the user is identified by an ID held by the clipboard 305.

  Hereinafter, the operation of the graphic data generation application 303 that implements the essential part of the present invention will be described in detail. The graphic data generation application (hereinafter referred to as application) 303 starts its operation by an activation method unique to the OS. The application immediately after startup displays one unique window and waits for a user instruction operation.

  FIG. 5 is an example of a window that makes the appearance of the application 303. 501 is a title bar for displaying the application name and processing file name, 502 is a menu bar for the user to instruct the application 303 to operate using a hierarchical menu, and 503 is a tool bar for the user to instruct the application 303 to perform an operation by clicking a button. It is. Reference numeral 504 denotes a display area in which graphic data to be processed is rasterized into two-dimensional image data for display, and reference numerals 505 and 506 denote scroll bars for scrolling the range displayed in the display area 504 vertically and horizontally by a mouse operation.

  On the menu bar 502, character strings 507 to 509 for roughly classifying the operation content are written. Clicking each of the character strings 507 to 509 with the mouse will display a menu with items for instructing the operation classification in more detail. By clicking the item, processing is actually performed, or more detailed items are displayed. Or On the toolbar 503, frequently used functions among the items in the hierarchical menu that can be selected from the menu bar 502 can be arranged. In addition, since the instruction from the hierarchical menu and the instruction with the corresponding button can be regarded as equivalent, the description of the instruction example with the button is omitted.

Operations that can be instructed by the user to the application 303 are classified into the following six types.
(1) File read operation (2) Graphic display operation (3) Graphic selection operation (4) Copy operation (5) File write operation (6) End operation

  Each operation of (1) to (6) has a transition relationship as shown in FIG. For example, a file reading operation is required prior to all operations, followed by a graphic display operation. The copy operation can be executed only after the graphic selection operation, and the file write operation and end operation can be executed at an arbitrary time after the file is read.

Hereinafter, the operations (1) to (6) will be described in detail.
(1) File Read Operation The internal operation of the application 303 performed during the file read operation will be described with reference to the flowchart of FIG. In step S701, the user specifies a file to be read. As an actual operation in the embodiment, when the character string “file” 507 is clicked on the menu bar 502 with the mouse, items 901 to 904 are expanded as shown in FIG. 9, and then the “open” item 901 is clicked. By doing so, a file selection dialog (not shown) is displayed. A file selected with the mouse or the keyboard on the file dialog is read into the application 303. The above operation may be an operation in which a file selected by another file system management software or the like is read by so-called drag-and-dop using a mouse.

  In step S702, it is determined whether the type of file to be read is an image file or a graphic data file. Specifically, the type may be determined by associating part of the file name with the file type, such as a predetermined extension, or by reading the first few bytes of the file and identifying characters or codes The type may be determined by obtaining the above. If the file is a graphic data file, the process proceeds to step S703. If the file is an image file, the process proceeds to step S704.

  If the process proceeds to step S703, the graphic data file is read to obtain graphic data, and then the process proceeds to step S706. In step S706, the graphic data is drawn on the screen, and the read operation process is terminated.

  If the processing proceeds to step S704, the image file is read and developed as raster image data in an area in the memory 202 that can be used by the application 303. If the image is in a compressed state, an expansion process is performed as appropriate. An example of the compression format is the JPEG format.

  In step S705, the raster image data is converted into graphic data. The processing content of this step will be described later. If the graphic data is obtained, the process proceeds to step S706, where the graphic data is drawn on the screen, and the reading operation process is terminated.

  Here, the contents of the conversion processing from raster image data to graphic data in step S705 will be described in detail with reference to the flowchart of FIG. Conversion from raster image data to graphic data means that the content of a single page image is separated into foreground objects such as text, line drawings, photographs, and tables, and the remaining background objects, and the foreground object It is converted into a graphic representation suitable for each. The graphic representation suitable for each object will be described in various parts of the processing according to the subsequent flowchart.

  In step S801, raster image data is input for conversion to graphic data.

  In step S802, if the raster image data is a full color or gray image, it is converted into a black and white binary image. In binarization, the luminance distribution of the entire image is examined to find the binarization threshold, and binarization is performed so that the character portion at the low luminance peak is black and the background portion at the high luminance peak is white. Do. Alternatively, binarization may be performed by dividing an image into several blocks and obtaining an optimal binarization threshold value for each block.

  In step S803, block selection processing is performed on the binary image. The block selection process is a process for recognizing blocks of document objects such as characters, line drawings, natural images, tables, and the like described in raster image data representing a page and classifying each block as a rectangular area.

  An example of the block selection process will be described below. First, contour tracing is performed on a binary image to extract a block of pixels surrounded by a black pixel contour. For a black pixel block with a large area, contour tracking is also performed for white pixels inside, and a white pixel block is extracted. Further, a black pixel block is recursively extracted from a white pixel block with a certain area or more. Extract lumps. The black pixel blocks thus obtained are classified by size and shape, and are classified into regions having different attributes. For example, if the aspect ratio is close to 1 and the size is within a certain range, the pixel block corresponding to the character is set, the portion where the adjacent characters can be grouped in a well-aligned group is the character region, and the flat pixel block is the line drawing region. The area occupied by the black pixel block that is larger than the size and contains the square white pixel block in a well-aligned area is the surface area, the area where the irregular pixel block is scattered is the natural image area, and the other pixel block of any other shape Is a line drawing area.

  FIG. 15 shows an example of block selection processing. When the block selection processing is performed on the raster image data 1501, the raster image data 1501 is divided into character areas 1502 and 1507, a natural image area 1503, line drawing areas 1504 and 1505, a table area 1506, and a background area 1508. Although omitted in FIG. 15, six white pixel block regions are extracted from the inside of the table region 1506, and a character region is extracted from each of the insides. As for the table region 1506, the alignment state of the above-described rectangular white pixel block included is analyzed as a matrix structure. More specifically, the horizontal boundary of white pixel blocks is regarded as the boundary of the column, and the vertical boundary is regarded as the boundary of the row, and a grid composed of the obtained number of boundaries (excluding both ends) + 1 row and column is assumed. The position information of (i, j) is given to the grid of the column position i and the row position j, respectively, and the position information where each white pixel block exists is recorded accordingly. In addition, in the case of straddling a white pixel block straddling a plurality of grids, for example, in the horizontal direction k and the vertical direction m, spread information (k, m) is added to the white pixel block.

  FIG. 16 shows an example of table structure analysis. The table shown on the left in FIG. 16 is determined to have a 4 × 3 matrix structure, and position information as shown on the right in FIG. 16 is given to each white pixel block.

  FIG. 17 shows an example of data that the application 303 has as block selection processing result information. The block selection processing results are binary data lists 1701 and 1702 representing information of one area. Information included in one area includes area coordinates 1703, area type 1704, link 1705 to individual information that varies depending on the area type, link 1708 to graphic data in the area, link 1709 to the next area, link 1701 to the child area. It consists of. Reference numerals 1709 to 1711 denote graphic data of the area 1701, and when there are data with a plurality of graphic expressions for one area, they are held as a list as shown in the figure. The link 1708 to the child area is used as information indicating the area to be included when there is a parent-child relationship as shown in the table. Note that FIG. 17 is merely an example, and the data format may take any form. For example, even if it is described in an XML format using tagged text, the same effect can be obtained in the present invention.

  Thereafter, each area obtained by block selection is converted into graphic data by graphic expression suitable for each area type. In step S804, graphic data of a character code representation including a character code string and font information is generated for the character area. The character code string can be obtained by performing OCR processing on the image in the character area. Also, font information including size and font can be obtained by recognition processing similar to OCR processing. Hereinafter, specific examples will be described. The OCR process is a process for obtaining a corresponding character code by recognizing an image cut out in character units using a pattern matching method. More specifically, the observed feature vector obtained by converting features obtained from a character image into a tens of dimensional numerical sequence is compared with the dictionary feature vector obtained for each character type in advance, and the character type with the closest distance is recognized. As a result. There are various known methods for extracting feature vectors. For example, there is a method characterized by dividing a character into meshes, and using a mesh number-dimensional vector obtained by counting character lines in each mesh as line elements according to directions.

  When character recognition is performed for a character area, the pre-recognition process first determines whether the area is written horizontally or vertically, cuts out lines in the corresponding directions, and cuts out characters from the lines. Get the image. Here, horizontal / vertical writing is determined by taking a horizontal / vertical projection of the pixel values in the corresponding area. If the horizontal projection variance is large, the horizontal writing area is determined. If the vertical projection variance is large, the vertical writing area is determined. That's fine. For horizontal writing, character strings and characters are decomposed by cutting out lines using horizontal projection, and cutting out characters from the projection in the vertical direction with respect to the extracted lines. For vertically written character areas, horizontal and vertical may be reversed. At the same time, the character size can be detected.

  As a method of recognizing fonts, a plurality of dictionary feature vectors for the number of character types used for character recognition are prepared for character shape types, that is, font types, and characters are used for matching in the previous recognition process. There is a way to get the font type along with the code.

  In step S805, vector representation graphic data including path drawing data for drawing and painting the outline of the character is generated as second graphic data different from the character code representation for the character region. Vectorization of the character area is obtained by dividing the outline of the black pixel block corresponding to each character into a plurality of line segments and approximating each with a straight line and a curve in the binary image of the character area. A specific example of line segment approximation processing will be described below. The outline of the black pixel block is traced with respect to the binary image of the character area to obtain a point sequence of the outer outline. This point sequence is divided by points regarded as corners, and each section is approximated by a partial straight line or curve. An angle is a point where the curvature is maximum, and the point where the curvature is maximum is that a string is drawn between points Pi-k and Pi + k that are k left and right apart from an arbitrary point Pi as shown in FIG. Then, the distance between the chord and the arbitrary point Pi is obtained as the maximum point. Furthermore, let R be the chord length / arc length between the points Pi-k and Pi + k, and the point where the value of R is equal to or less than the threshold value can be regarded as a corner. Each section after being divided by the corners can be approximated using a least square method or the like for a straight line and a curve using a cubic spline function or the like for a straight line. Similarly, the inner contour formed by the white pixel block existing inside the black pixel block is also approximated by a partial straight line or curve.

  Further, character extraction equivalent to the processing in step S804 is performed, and an average pixel value inside each character line is obtained from the original image to obtain the color of each character, and this is color information for drawing a path corresponding to each character. can do.

  In step S806, as third graphic data for the character area, graphic data expressed by binary or N-value image data that can easily distinguish pixels inside the character line from pixels other than the character is generated. The binary image data representing the character area is obtained by cutting out the corresponding area from the image obtained by binarizing the entire page. It is also possible to obtain a binary image of a more optimal character area by obtaining again the binarization threshold value in the corresponding area from the multi-valued original image.

  In addition, the character extraction is performed using the processing in step S804, and the average pixel value inside each character line is obtained from the original image to obtain the color of each character, and the binary image of the character region is determined according to the color for each character. By dividing into N planes, graphic data based on N binary images having color information is generated. Alternatively, graphic data may be generated by an N-value image that can take N values for each pixel using color information as a palette code without being divided into N planes as described above.

  In step S807, vector representation graphic data including path drawing data of the inner and outer contours is created for the line drawing area. The vector data consisting of the path drawing of the inner and outer contours is obtained by approximating the contour line of the black pixel block from the binary image of the line drawing region, as in the character vector processing in step S805 described above. Can do.

  Further, for the pixels forming the line portion on the binary image, the pixel values obtained from the multi-valued original image are averaged to obtain pass drawing color information. When the line drawing is composed of a plurality of colors, one pixel value that is the main color is selected as color information.

  In step S808, vector representation graphic data including drawing data of shape elements is created as second graphic data for the line drawing area. Rendering data for shape elements is not a description of a line part as a set of outer contour and inner contour paths, but is expressed as a path of a line segment with a single thickness, or a representative such as a square or a circle. A simple shape is not a set of paths, but vector expression data that is expressed by data for drawing various shapes.

  In order to obtain element drawing data expressed by a line segment path, a known thinning method is applied to a binary image in a line drawing area in advance to set all line widths to 1, and then a point sequence of each line is traced. As in step S807, vertices that are corners that divide the point sequence are detected, and fitting of approximated straight lines or curves is performed.

  For representative shapes such as squares and circles, a replaceable image pattern is prepared in advance, and corresponding line elements are replaced by pattern matching or the like.

  Furthermore, for each of the elements that subdivide the line image such as each line segment and shape, the pixel values obtained from the multi-valued original image are averaged for the pixels forming the corresponding line portion on the binary image. The color information of each element drawing is obtained.

  In step S809, as third graphic data for the line drawing area, graphic data represented by binary or N-value image data that can easily separate the line drawing portion and the background is generated. Specifically, the multi-valued original image in the line drawing area is divided into background color pixels and other pixels, the background pixels are values that virtually represent transparency, and otherwise the number of colors in the line drawing area is appropriately quantized. An N-value image with a palette assigned to a value corresponding to one of the N palette colors is generated.

  In step S810, graphic data composed of images is created for a natural image area. Specifically, image data in a state in which the corresponding area portion is cut out from the original image is created.

  In step S811, vector-represented graphic data composed of drawing data of the table frame line is created for the table area. As a specific processing method, pixels in a character part are deleted from a binary image in a table area, and an image having only a table frame is created. Thereafter, the same line segment extraction and approximation processing as in the case of the line drawing is performed, and if necessary, it is converted into vector data of rectangular drawing corresponding to the table cell.

  In step S812, graphic data composed of data representing the character code of the character in the table and its matrix arrangement structure is created for the table region. Regarding the characters in the table, it is assumed that in step S804, the character code and font information for the internal character region are extracted for each region surrounded by a frame in the table, that is, for each cell. The matrix arrangement structure has already been obtained in the process of block selection processing for the table. Here, the cells in the table correspond to white pixel blocks in the table, and each has position information and spread information when the table is arranged in a matrix grid. The above character code, cell position, and spread information are combined to form graphic data for the table area.

  In step S813, the foreground information converted into the various graphic data in steps S804 to S812 is removed from the original image, that is, graphic data including a background image is created. The concrete background is created as follows. In the character, table, and line drawing area, the binary image in the area is referred to, and the pixel corresponding to the black pixel is changed to the background color in the original image. The background color is the color of a pixel in the vicinity of the pixel and corresponding to a white pixel in the binary image. In the actual scan image, the pixel edge near the line changes somewhat gently. Therefore, the pixel to be changed to the background color is better in the range where the black pixel on the actual binary image is thickened by about 1 dot. When obtaining the color, it is better to obtain the average color of several pixels outside the fattened range. For the photo portion, the area after removing the foreground is large, and when the foreground is overwritten, it is completely overwritten, so there is no need to complement it with the background color. However, in order to improve the compression ratio as data, it may be painted with a background color or simply white.

  As described above, the graphic data generated in steps S804 to S812 is held as data referenced from the link to the graphic data in each area as indicated by 1709 in FIG. Here, any format of graphic data may be used. It may be a set of binary drawing data corresponding to a drawing command on the application one-to-one, or may be text data described in a graphic language such as SVG.

(2) Graphic Display Operation Graphic display is to render a part or all of graphic data into two-dimensional image data and draw it in the display area 504. At this time, which part of the page area that can be represented by the graphic data held by the application 303 is displayed, or which kind of graphic is held when holding plural types of graphic data representing a certain same area This graphic display operation provides a function of selecting whether to draw and display data based on the user's intention. As a specific operation, each process is executed by clicking each item of a menu (FIG. 10) that is expanded when the character string “display” 508 is clicked on the menu bar 502. Details will be described below.

(2-1) Constant Magnification Display Operation Graphic data can be freely changed in size after rasterization by drawing a certain ratio with respect to the coordinate values of each drawing element. Here, when the physical size of the original page is equal to the apparent size on the display, the magnification is set to 100%. The ratio value multiplied by the coordinate value at a magnification of 100% is uniquely determined from the ratio between the resolution of the graphic data and the display resolution.

Reference numerals 1001 to 1007 in FIG. 10 are examples of menus for instructing a constant magnification display operation. There are seven items of “25%” 1001, “50%” 1002, “100%” 1003, “200%” 1004, “400%” 1005, “800%” 1006, and “magnification setting” 1007. 1006 is rasterized at the selected magnification and displayed on the display. When an item of “magnification setting” 1007 is selected, a dialog for inputting a numerical value of the magnification is opened, and is rasterized at a magnification according to the input value and displayed on the display.
If the page rasterized at the specified magnification does not fit in the display area 504, only the displayable range is drawn. In addition to moving the scroll bars 505 and 506 at the right and bottom of the display area where the user can change the drawing range, the drawing range can also be changed by a pan operation described later.

(2-2) Window Dependent Display Operation This is an operation for rasterizing graphic data with a size depending on the current application window display size and displaying it on the display. 1008 and 1009 in FIG. 10 are examples of menus for instructing window-dependent display operations. “Display entire page” 1008 is a size that fits within the display area in the window. The “display” 1009 is rasterized and displayed in such a size that the display area width is equal to the rasterized page width. If you change the size of the application window, it will be re-rasterized and displayed in the same size.

(2-3) Loupe operation An operation for enlarging and displaying a part of the graphic data currently displayed in the application window. After selecting the “Loupe” item 1010 in FIG. 10, if a mouse button is pressed at one point on the display area and then dragged as it is to release the button, a rectangular range consisting of two points, the pressed point and the released point. Magnify and display. At this time, graphic data is rasterized and displayed at a magnification at which the width of the enlarged region is equal to the width of the display region, a magnification at which the height of the enlarged region is equal to the height of the display region, and the smaller of the two. If the mouse button is pressed and then released without dragging, it is displayed at twice the magnification currently displayed.

(2-4) Pan operation This operation changes the display range of rasterized graphic data partially displayed in the application window by dragging the mouse. When the “pan” item 1011 in FIG. 10 is selected and then the mouse button is pressed and dragged at one point on the display area, the displayed range moves in conjunction with the drag. When the page width is within the width of the display area, it does not move to the left or right. Similarly, when the page height is within the display area, it does not move up and down.

(2-5) Display selection The graphic data constituting the page is held as graphic data in each area for each area divided into document objects as described above. Document objects are roughly divided into foreground and background.

  1012, 1013, and 1014 are items for selectively displaying the foreground and the background. When “foreground” 1012 is selected, only the foreground graphic data is displayed, and when “background” 1013 is selected, the background graphic is displayed. Only the data is displayed. When “foreground + background” 1014 is selected, the foreground graphic data is displayed on the background graphic. FIG. 18 shows an example of display selection. When “foreground” 1012 is selected, 1801 is displayed, when “background” 1013 is selected, 1802 is displayed, and when “foreground + background” 1014 is selected, 1803 is displayed.

  When “Foreground Details” 1015 in FIG. 10 is selected, a dialog is opened that allows the user to select which expression type is displayed as drawing data when the same area holds graphic data of a plurality of expression types. As an example of the dialog for selecting the foreground details, reference numeral 1901 in FIG. 19 shows either a character outline path data expression or an image data expression for a character area, and an inner / outer outline path for a line drawing area. A choice of data representation, shape element data representation, or image data representation is provided. In the present embodiment, the character code representation in the character region, the in-table character code and the matrix arrangement structure representation in the table region are not handled as graphics drawn as the foreground, and are also included in the options in FIG. I can't.

(3) Graphic selection operation The user can selectively instruct a partial range or the whole of graphic data by graphic selection operation. This operation is performed for the purpose of using part or all of graphic data held by the application 303 on another application such as the application 304 in combination with (4) copy operation described later.

  The menus 1101 to 1103 shown in FIG. 11 are menus for designating graphic types to be selected. After clicking one of the three items “Select Drawing Object” 1101, “Character Code Selection” 1102, and “Table Code Selection” 1103, drag the mouse over the display area to indicate the selection range. In addition, the graphic data to be selected is selected. The graphic data in the selected state is a target of a copy operation (4) described later.

  Hereinafter, each of the above three types of selection will be described. First, when “drawing object selection” 1101 is instructed, the range designation on the display area by the mouse, that is, among the graphic data existing in the rectangular range whose diagonal is the point separated from the point where the mouse button is pressed is present. The graphic data being drawn is selected. Here, the currently drawn graphic data coincides with the graphic data selected by the display selection operations 1012 to 1014 and 1015 described above. That is, when the “foreground” 1012 display is selected, only the graphic data corresponding to the foreground is selected, only the background graphic data is selected when the “background” 1013 display is selected, and both the foreground and background graphic data are selected when the “foreground + background” 1014 display is selected. It becomes. When the foreground graphic data including a plurality of types of expressions is included in the target area, the type of graphic data selected by the “foreground details” 1015 is selected.

  If there is a graphic drawing element that straddles the boundary of the selected rectangular range, the selection range is changed or the graphic data drawing element itself to match the correspondence between the selected range and the selected graphic drawing element It is necessary to perform a process such as cutting a part of the image so that it falls within the selection range.

  Specifically, when the graphic drawing element to be selected is image representation data, new graphic data obtained by cutting only the portion within the selection range from the original data is selected. However, it is better to actually perform the cutting process when the copy operation described later is executed, not at the time of selection.

  On the other hand, when the graphic drawing element to be selected is vector representation data, it is not easy to cut out the data within the selection range, so one of the following two measures is taken. (Countermeasure 1) The graphic drawing elements that protrude from the selection range are excluded from the selection target. (Countermeasure 2) A rectangle that becomes the selection range is included so that all graphic drawing elements that overlap the selection range that the user first specified are included. Enlarge automatically.

  FIG. 12 shows an example of (Countermeasure 1). When the mouse button dragged in the state 1201 is released, the graphic elements 1202 and 1203 that protrude from the range are excluded from the selection target. FIG. 13 shows an example of (Countermeasure 2). When the mouse button dragged in the state 1301 is released, the selection range is automatically expanded as shown by 1304 so that the graphic elements 1302 and 1303 protrude from the range are included. Is done.

  Which of (Countermeasure 1) and (Countermeasure 2) is to be performed may be fixed or selected by the user. The operation may be changed depending on whether image data such as a background is included, or a small element such as a character and a large element such as a table frame or a line drawing.

  At the “drawing object selection” 1101, a square frame corresponding to the selected range is displayed on the display area so as to visually indicate the selected rectangular range. In addition, in order to visually indicate to the user which graphic drawing elements are selected, some of the graphic drawing elements that are selected symmetrically are displayed in the display area with a modification such as color or shape. Drawn on. For example, for data expressed in vectors such as characters, line drawings, and table frames, the graphic drawing element in the selected state is drawn in red instead of the original color. As a result, the user can easily distinguish the data in the selected state from the data in the non-selected state.

  In addition to the above example, other colors may be used, for example, colors obtained by inverting the R, G, and B values of the original color. In addition, if it is easy to visually distinguish between the graphic drawing elements in the selected state and the non-selected state, the fill pattern in the outline instead of the color can be changed, or the thickness can be changed. May be.

  Furthermore, the graphic drawing elements selected by specifying the range are limited to those selected by the above-described (2-5) display selection, that is, the drawing data that is actually displayed. In some cases, the type of the graphic drawing element becomes difficult to understand simply by looking at the graphic drawn in the display area. For example, for a white background portion, it is not possible to visually distinguish between the case where only “foreground” is displayed and the case where “foreground + background” is displayed. Alternatively, in the “foreground details” 1015 setting in FIG. 10, whether the path data representation of the inner and outer contours is drawn as the data representing the same line drawing, or whether the shape element data is drawn, etc. The difference may be small and difficult to distinguish.

  Therefore, by further changing the expression method that clearly indicates the previous selection state according to the type of data in the selection state, the user clearly shows which data is copied in the copy operation described later, Prevent copying different types of data that is not intended. For example, when “foreground” is displayed in the display selection, the display color of the square frame corresponding to the selected area is blue, and when “background” is displayed and “foreground + background” is displayed, red and green are used.

  In addition, for the setting of the drawing type for an object having a plurality of expression types in the “foreground details” setting in FIG. 10, a double frame in which a color frame synchronized with the drawing type is drawn slightly inside the above frame. Display. For example, when the inner / outer contour path data is selected, the inner frame is displayed in a one-to-one color for the drawing type, such as a yellow frame when a shape element data is selected. . Of course, it is not limited to the above-described double frame with color, but may be combined with colors, line types, blinking, and the like. Alternatively, not only the frame but also the color and shape of the graphic drawing element in the selected state may be changed according to the data type.

  Returning to FIG. 11, when “character code selection” 1102 is instructed, data of character code expression in the range specified by dragging the mouse is selected. Since the character code expression is not drawn as a graphic, the character code at that position is in the selected state by, for example, blacking out the range corresponding to each selected character or arithmetically inverting the pixel value. Present to the user.

  FIG. 20 shows a display example of the character code selection state. When an area 2001 is designated with the mouse, portions corresponding to characters in the area are blackened as in 2002. In FIG. 11, when “select table code” 1103 is instructed, data representing the internal character code and matrix arrangement structure is selected for a table in the range specified by dragging the mouse. . In order to clearly indicate to the user that the table has been selected, a frame of a color different from the selection of other drawn objects is overwritten on the outer periphery of the table.

  In addition, when an “area information display” item 1104 in FIG. 11 is designated, rectangular frames corresponding to the area information as a result of the block selection process in step S803 are displayed on the graphic data drawn in the display area. At this time, by clicking an arbitrary rectangular frame with the mouse, the graphic data in the frame and the graphic data for each document object can be selected. The type of graphic data to be selected follows the above-mentioned selections 1101 to 1103 and (2-5) display selection.

  FIG. 21 shows an example after the “area information display” item is instructed. Note that the display selection is “foreground” and is shown by dotted lines instead of colors.

(4) Copy operation (3) When at least one graphic data element is in a selected state by a graphic selection operation, when the “copy” 1105 in FIG. 11 is designated, the content of the graphic data in the selected state is clipboard 305. It is copied to. The graphic data copied to the clipboard 305 can be pasted and used in any application that can interpret the data format, such as the graphic editing application 304.

  The internal operation for the copy operation will be described with reference to the flowchart of FIG. In step S1401, if there is image data to be selected, it is determined whether the image data is within the selected area. If it is, the process proceeds to step S1403. If not, the process proceeds to step S1402.

  In step S1402, an image obtained by cutting only a portion that overlaps the selection region from the selection target image data is set as a new selection target, and the original image data is excluded from the selection target. In step S1403, the graphic data format in the application internal format is converted into a format suitable for saving to the clipboard. Here, the graphic editing application 304 that uses this data is converted into a format that can be interpreted.

  In step S1404, the converted graphic data is copied to the clipboard. 22 to 24 show examples of data types to be copied to the clipboard when graphic data is selected and copied. What is to be processed here is the data of the page shown in FIG. 25, in which 2501 is a foreground character area, and 2502 is a background area indicated by a dotted frame. In addition, it is assumed that the character area path drawing is selected in the “foreground details” 1015 setting in FIG.

  As the graphic data range, the operation will be described for each of cases where 1012 to 1014 are selected in (2-5) display selection assuming that the solid line 2503 is designated with the mouse.

  FIG. 22 is a diagram illustrating a case where the display of “foreground” 1012 is selected. When a selection range 2202 is specified with the mouse in the display area 2201 and a copy operation is performed, path drawing data 2203 and 2204 for two characters in the selection range are copied to the clipboard 305 from the path expression graphic data in the character area 2501. The

  FIG. 23 is a diagram for explaining a case where the “background” 1013 display is selected. When a selection range 2302 is specified with the mouse in the display area 2301 and a copy operation is performed, image data 2303 obtained by cutting only the selection range from the image data in the background area 2502 is copied to the clipboard 305.

  FIG. 24 is a diagram for explaining the case where the “foreground + background” 1014 display is selected. When a selection range 2402 is designated with the mouse in the display area 2401 and a copy operation is performed, path drawing data 2404 and 2405 for two characters in the selection range and a background area 2502 are obtained from the path expression graphic data in the character area 2501. The graphic data 2403 obtained by combining the image data 2406 obtained by cutting only the selected range from the image data is copied to the clipboard 305. Since the graphic data 2403 in FIG. 24 is described so that the path data is drawn after drawing the background image data, the application receiving the graphic data 2403 can handle the character and the background separately. It is.

(5) Write operation When the “save” item 902 in FIG. 9 is instructed, the graphic data of the application 303 is saved in a file designated by the user in a dialog (not shown) displayed immediately thereafter.

(6) Close operation When the “close” item 903 in FIG. 9 is instructed, the graphic data currently being read is discarded, and (1) the process returns to a state of waiting for file reading. When the “end” item 904 is designated, the graphic data is discarded and the application 303 is ended as in the “close” item 903.

  As described above, when multiple types of data exist within the same range when specifying the range of graphic data and copying the data in it to another means for use in another application, Accordingly, by changing the color of the range specification frame etc., the user can immediately know which data type is currently selected and copied, so that erroneous operations such as copying the wrong data type are reduced, The work burden on the user can be reduced.

  In the above embodiment, the visual effect is used to clearly indicate the type of the selected graphic drawing element. However, in addition to this, an audio warning may be given. Even in this case, since the user can immediately know which data type is currently selected and copied, erroneous operations such as copying the wrong data type can be reduced, and the user's work load can be reduced. .

  An object of the present invention is to supply a storage medium storing a program code of software that realizes the functions of the above-described embodiments to a system or apparatus, and the computer (or CPU or MPU) of the system or apparatus stores the storage medium. Needless to say, this can also be achieved by reading and executing the program code stored in.

  In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the program code itself and the storage medium storing the program code constitute the present invention.

  As a storage medium for supplying the program code, for example, a flexible disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a magnetic tape, a nonvolatile memory card, a ROM, or the like can be used.

  Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an OS (basic system or operating system) running on the computer based on the instruction of the program code. Needless to say, a case where the functions of the above-described embodiment are realized by performing part or all of the actual processing and the processing is included.

  Further, after the program code read from the storage medium is written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, the function expansion is performed based on the instruction of the program code. It goes without saying that the CPU or the like provided in the board or function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing.

It is a figure which shows the structure of the system to which this invention is applied. It is a figure which shows the internal hardware structural example of a computer apparatus. It is a figure which shows the module structural example of the software performed with a computer apparatus. It is a flowchart for demonstrating the operation | movement at the time of processing using an application. It is a figure which shows the example of the window which makes the external appearance of a graphic generation application. It is a figure for demonstrating the transition relationship of each operation. It is a flowchart for demonstrating operation | movement inside the graphic generation application performed at the time of file reading operation. It is a flowchart for demonstrating the conversion process from raster image data to graphic data. It is a figure which shows the menu expanded when the character string "file" is clicked with a mouse | mouth in a menu bar. It is a figure which shows the menu expanded when the character string "display" is clicked on a menu bar. It is a figure which shows the menu expanded when the character string "selection" is clicked on a menu bar. It is a figure for demonstrating the example which excludes the graphic drawing element which protrudes the selection range from selection object. It is a figure for demonstrating the example which expands automatically the rectangle used as a selection range so that all the graphic drawing elements which overlap with the selection range which the user instruct | indicated initially may be included. It is a flowchart for demonstrating the internal operation | movement with respect to copy operation. It is a figure which shows the example of a block selection process. It is a figure which shows the example of a table structure analysis. It is a figure which shows the example of the data which a graphic generation application has as information of a block selection process result. It is a figure which shows the example of display selection. It is a figure which shows the example of the dialog which selects foreground details. It is a figure which shows the example of a display of a character code selection state. It is a figure which shows the example after instruct | indicating the "area | region information display" item. It is a figure which shows the example of the data kind copied on a clipboard when graphic data is selected and copied. It is a figure which shows the example of the data kind copied on a clipboard when graphic data is selected and copied. It is a figure which shows the example of the data kind copied on a clipboard when graphic data is selected and copied. It is a figure which shows the data of the page which it is going to process by copy operation. It is a figure for demonstrating the process which produces | generates the graphic data of the vector expression which consists of the path drawing data which draws and paints the outline of a character.

Explanation of symbols

101 Computer device 102 Display device 103 Printer device 104 Operation device 105 Scanner device 201 CPU
202 Memory 203 Hard Disk 204 I / O Unit 301 Basic Software 302 Scan Application 303 Graphic Data Generation Application 304 Graphic Editing Application 305 Clipboard

Claims (9)

Display means for drawing and displaying graphic data;
Range specifying means for specifying a range on the display means;
Means for identifying only data within a specified range of the graphic data;
Data type specifying means for specifying the type of the graphic data;
Output means for outputting graphic data specified by the range and type,
An image processing apparatus characterized in that, when the range is designated, a different visual effect is produced according to the data type designation.   The image processing apparatus according to claim 1, wherein the visual effect changes at least one of a color, a shape, and a line shape of a frame representing a range.   The image processing apparatus according to claim 1, wherein the graphic data types are vector data and image data.   4. The image processing according to claim 3, wherein the graphic data is divided into elements such as characters, line drawings, natural images, tables, and backgrounds constituting a document page, and graphic data representing each of them. apparatus.   5. The image processing according to claim 4, wherein the types of graphic data are data that represents a contour of a document element as a vector and data that represents a line drawing as a shape drawing element such as a line, circle, or square. apparatus.   The image processing apparatus according to claim 5, wherein the type of graphic data includes data that further represents a document element as an image.   The image processing apparatus according to claim 4, wherein the types of graphic data are foreground and background of characters, line drawings, natural images, tables, and the like of a document. Drawing graphic data and displaying it on the display means;
Designating a range on the display means;
Identifying only the data within the specified range of the graphic data;
Designating the type of the graphic data;
Outputting graphic data specified by the range and type,
An image processing method characterized in that, when the range is specified, a different visual effect is produced according to the data type specification. Processing to draw graphic data and display it on the display means;
Processing for designating a range on the display means;
A process of specifying only data within a specified range of the graphic data;
Processing for specifying the type of the graphic data;
Causing the computer to execute processing for outputting graphic data specified by the range and type,
A computer program characterized in that, when the range is specified, a different visual effect is produced according to the data type specification.

Images