JP2014038629A - Image processing device, image processing method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve a problem that only an element unit is selected but an optional area is not selected, when selecting a web page displayed by a browser in the browser.SOLUTION: An image processing device inserts and displays an object displayed on a front face of a web page and indicating an area of the web page, in a structured document. An intermediate data file capable of extracting the area is prepared corresponding to the web page, on the basis of the structured document. An image of an area indicated by the object is extracted out of the intermediate data file, to be displayed.

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program for processing an image based on a structured document.

  In recent years, it has become possible to obtain various information by accessing Web pages on the Internet. These Web pages are structured documents described in a structured language such as HTML (Hyper Text Markup Language) or XHTML (Extensible Hyper Text Markup Language). A Web page can be displayed on a display and referred to by software called a browser corresponding to the structured document.

  Further, it is possible to extract and print a partial area of these Web pages as an image. In this case, a method is generally used in which a web page is displayed on a display by a browser, and a partial area in the web page being displayed is selected using a pointing device such as a mouse. (For example, see [Patent Document 1])

Japanese Patent Registration No. 035883337

  However, according to the conventional method, when the Web page area displayed in the browser is selected by the browser, only the element unit can be selected. That is, there is a problem that an arbitrary area cannot be selected for the user, for example, a part of an image element or a part of a text element.

  In order to solve the above-described problems, the present invention provides a specifying unit that specifies an extraction target region in a page in which a plurality of elements included in a structured document are arranged, and an extraction target specified by the specifying unit in the page. Extracting means for extracting image data corresponding to a region, and when a boundary of the extraction target region specified by the specifying unit corresponds to a part of one of the plurality of elements, The extraction unit selects the one element, creates image data based on the selected one element, and cuts out the image data corresponding to the part of the created image data, thereby extracting the part. The image data corresponding to is extracted.

It is a figure which shows the example of the printing system which prints based on a structured document. 2 is a block diagram showing a configuration of a PC 101. FIG. 3 is a block diagram illustrating a software configuration of the PC 101 in Embodiment 1. FIG. It is a figure which shows an example of a GUI screen. It is a figure which shows the GUI screen after the range selection button 406 is pressed. It is a figure which shows an example of a structured document. It is a figure which shows the process sequence of the structured document printing module. FIG. 10 is a block diagram illustrating a software configuration of a PC 101 according to a second embodiment. It is a figure which shows the area | region selection window 901 of Example 2. FIG. FIG. 10 is a block diagram illustrating a software configuration of a PC 101 according to a third embodiment. FIG. 10 is a diagram illustrating a GUI screen of an area selection window according to the third embodiment.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. The following embodiments do not limit the present invention according to the claims, and all combinations of features described in the present embodiments are essential to the solution means of the present invention. Not exclusively.

  FIG. 1 is a diagram illustrating an example of a printing system that performs printing based on a structured document. A PC (Personal Computer) 101 shown in FIG. 1 is connected to a printer 104 and causes the printer 104 to print an image as a print control apparatus. The PC 101 is connected to the WWW server 103 via the Internet 102, downloads a structured document written in a structured language, and displays a Web page based on the structured document. This structured document is described in a language such as HTML or XHTML. Further, based on the downloaded structured document, the printer 104 prints an image.

  FIG. 2 is a block diagram showing the configuration of the PC 101. The CPU 201 performs data operations and instructions according to a program, and controls each unit of the PC 101 and connected devices.

  A RAM 202 is a memory used as a temporary storage area, and a ROM 203 is a memory that stores various programs executed by the CPU 201. The hard disk 204 records an operating system (OS), a browser, other application software programs, data, and the like. That is, the CPU 201 reads out a program stored in the ROM 203 or the hard disk 204 to the RAM 202 and executes the program using the RAM 202 as a work memory to perform various controls.

  The USB interface 205 is an interface to which a USB cable is connected, and is an interface for performing data communication with the printer 104. Note that communication with the printer 104 may be performed by other than the USB cable, for example, by SCSI or wireless.

  The display device 206 includes a CRT, a liquid crystal display, and a graphic controller, and the PC 101 performs various displays as a display control device. For example, a Web page downloaded from the WWW server 103, a print preview image, a graphic user interface (GUI), or the like is displayed. The input device 207 is a device for the user to give various instructions to the PC 101, and includes, for example, a mouse and a keyboard. A system bus 209 exchanges data with the CPU 201, RAM 202, ROM 203, hard disk 204, and the like.

  A LAN interface 208 is an interface to which a LAN cable is connected, and performs data communication with an external WWW server 103 via a router (not shown) or the Internet 102. Note that this data communication may be performed wirelessly using, for example, a wireless interface.

  FIG. 2 shows a so-called notebook PC in which the PC 101 is integrated with a control unit such as the display device 206, the input device 207, the CPU 201, and the RAM 202. However, the present invention is not limited to this, and the PC may be a so-called desktop type in which the display device and the input device are separated from the control unit.

  FIG. 3 is a block diagram illustrating a software configuration of the PC 101 according to the first embodiment. Each block shown in FIG. 3 is obtained by classifying programs stored in the ROM 203 or the hard disk 204 by function, and these programs are executed by the CPU 201. A program that runs on the PC 101 and its data processing configuration will be described below with reference to FIG.

  In FIG. 3, a browser 301 is an application for displaying a Web page, and displays a Web page based on a structured document received from the WWW server 103. That is, a structured document file such as an HTML file is downloaded to the hard disk 204 on the PC 101 and displayed on the display device 206 based on the downloaded file. The Web page is a structured document file such as HTML or XHTML, and elements constituting the structured document such as text and images are described using tags. Also, another file called Cascading Style Sheet (hereinafter abbreviated as CSS) that specifies the display style of these elements is specified in the structured document. The browser 301 analyzes these structured document files and displays them on the display device 206.

  In FIG. 3, the structured document printing module 302 is plug-in software called from the browser 301, and the structured document printing module 302 is executed when the user gives an area selection instruction to the browser 301. The structured document printing module 302 performs processing for extracting a part of the Web page.

  In the structured document printing module 302, the area selection object insertion unit 303 inserts one or more area selection objects imitating a rectangular area for area selection into the structured document displayed in the browser. At this time, the region selection object insertion unit 303 acquires a display order called Z-order of each element in the structured document in order to insert the region selection object in the foreground. And it sets to the area | region selection object which inserts the Z-order attribute for displaying on the front rather than the existing frontmost element.

  The area selection object changing unit 304 receives an instruction to change the selection area performed by the user using the input device 207 such as a mouse, and changes the coordinates of the area selection object inserted by the area selection object insertion unit 303. As a result, the user can specify an arbitrary rectangular area of the structured document as a selection area using the input device 207 such as a mouse.

  The selection area data extraction unit 305 is executed when the user gives an instruction to start printing or print preview to the browser after selecting an area in the structured document. The selection area data extraction unit 305 can acquire the coordinates of the area selection object set by the area selection object change unit 304, extract elements in the structured document included in the selection area, and extract a partial area A simple intermediate data file. This intermediate data file is image data for printing, and it is desirable to store character information and graphic information as vector data, not bitmap data. At this time, in order to be able to extract an arbitrary area of the Web page, in other words, to be able to extract a part of the elements of the Web page, the intermediate data is intermediate data in which a clip area can be set for part of the vector data A file is desirable. For example, PDF (Portable Document Format) or EMF (Enhanced Metafile Format) can be used as the intermediate data file.

  The print layout unit 306 lays out the intermediate data file extracted by the selection area data extraction unit 305 to a paper size using print settings. This print setting includes information such as the paper size, resolution, and printable area, and is acquired from the printer driver 310 via the OS 309.

  The print preview unit 307 displays the elements laid out by the print layout unit 306 on the display device 206 as a print preview.

  When the print processing unit 308 receives a print start instruction from the user, the print processing unit 308 executes a drawing process on the printer driver 310 via the OS 309 using the arrangement information of the elements laid out by the print layout unit 306.

  The OS 309 provides an API (Application Programming Interface) for the structured document printing module 302 and the printer driver 310 to exchange print setting data and an API for performing drawing processing. The OS 309 includes various control software such as a spooler system that manages print jobs and a port monitor that outputs printer commands to a port.

  The printer driver 310 generates print data based on the drawing process executed by the print processing unit 308, converts the print data into a printer command, transmits the printer command to the printer 104 via the OS 309, and the printer 104 records it on a sheet.

  A GUI screen displayed on the display device 206 by the PC 101 as the structured document display control device of this embodiment will be described with reference to FIGS. 4 and 5. FIG. 4 is a diagram illustrating an example of a GUI screen. The browser 301 displays a web page on the GUI. Further, the browser 301 is provided with a return button 401, a forward button 402, an address input area 403, and the like for switching the Web page to be displayed. Further, a print button 404, a print preview button 405, and a range selection button 406 for selecting a range of the Web page are arranged in the window. When the user presses the range selection button 406, the browser 301 calls the structured document printing module 302. In the structured document printing module 302, the area selection object insertion unit 303 inserts the area selection object in which the initial coordinates are set into the structured document of the browser 301.

  FIG. 5 shows a GUI screen after the range selection button 406 is pressed. Reference numeral 502 in FIG. 5 denotes a region selection object including a plurality of elements inserted by the region selection object insertion unit 303. The initial coordinates of the area selection object 502 are set so that the area selection object 502 is placed at the center of the structured document display area of the browser, and a predetermined value is used as the initial size of the area selection object.

  The area selection object 502 includes a rectangular object 503 representing a selection area, and rectangular size changing objects 504 to 511 arranged at four corners and four sides of the rectangle. The rectangular object 503 is a rectangular object having a semi-transparent drawing attribute, and is configured to show through which part of the structured document is selected.

  The user can arbitrarily change the coordinates of the region selection object 502 by dragging the rectangular portion with the input device 207 such as a mouse. Also, the rectangle size can be arbitrarily changed by dragging the rectangle size changing objects 504 to 511 with a mouse or the like.

  In response to this rectangular size change instruction, the region selection object changing unit 304 changes the size and position of the region selection object 502.

  Further, as shown in FIG. 5, a print button 512 for starting printing with the selected area as a printing target is arranged. When the print button 512 is pressed, the selection area data extraction unit 305 acquires the coordinates of the area selection object 502 and extracts the structured document included in the rectangle as an intermediate data file. 5 also displays a cancel button 513 for canceling the area selection. When the cancel button 513 is pressed, the state returns to the state shown in FIG.

  FIG. 6 is a diagram illustrating an example of a structured document. The structured document 601 in FIG. 6 is described in the XHTML format. Although not shown, in the structured document 601, element layout information is described as a separate file by CSS.

  In the structured document 601, a <span> tag 602 is a tag for displaying the area selection object 502. In the <span> tag 602 in FIG. 6, a “position: absolute” attribute for designating a display position with absolute coordinates on the Web page is designated in order to enable the region selection object 502 to be displayed at an arbitrary coordinate. That is, since it is not the coordinates on the display screen or the coordinates for the browser window, the coordinates are not changed even if the browser window is moved, enlarged or reduced. In the <span> tag 602 in FIG. 6, a rectangular object having a width of 200 px and a height of 200 px is inserted at the position where the upper left coordinates of the area selection object 502 are left: 100 px and right: 100 px. Note that px indicates the number of pixels.

  In FIG. 6, the z-index attribute 603 is an attribute for designating the z-order of the area selection object 502, and here, z-index: 1000 is designated for displaying in the foreground. Thereby, when the z-index of other objects other than the region selection object 502 is less than 1000, the region selection object 502 is displayed in the foreground. Further, since the id of the <span> tag for the area selection object 502 is used to acquire the object when the size is changed, it is necessary to specify a unique ID in the structured document.

  Although not shown in FIG. 6, the area selection object 502 needs to specify the color of the rectangular area, the background attribute, the filter attribute for the transparency process, and the like. Further, although not shown in FIG. 6, in the area selection object 502, it is necessary to insert tags for the rectangular size changing objects 504 to 511 in addition to the tags for the rectangular area 503, but they are omitted here.

  FIG. 7 is a diagram showing a processing procedure of the structured document printing module 302. A program for executing the processing shown in this figure is stored in the ROM 203 and the hard disk 204. The CPU 201 reads out this program to the RAM 202 and executes it, so that the processing described here can be performed.

  First, in step 701, when the range selection button 406 of the browser 301 is pressed, the structured document printing module 302 is activated to start processing. In Step 702, a mouse event capture process is started in the displayed area of the browser 301 based on the structured document. The capture of the mouse event is processed by the area selection object changing unit 304 based on information input from the mouse connected to the PC 101. Next, in Step 703, the region selection object insertion unit 303 acquires the z-index attribute of the element in the structured document, and specifies the z-index attribute of the object displayed in the foreground.

  In Step 704, the region selection object insertion unit 303 sets the initial coordinates, the initial size, and the z-index attribute to be displayed in the foreground of the region selection object 502, and inserts them into the <body> element of the structured document. In order to insert the area selection object 502 in the foreground, the z-index attribute of this area selection object is set to a value larger than the z-index attribute of other elements acquired in Step 703.

  In Step 705, input of an instruction based on information from the mouse is awaited. When an instruction to select the cancel button 513 is input by a mouse operation, the capture is canceled at Step 710 and the region selection process is ended at Step 711. If an instruction to change the position and size of the area selection object 502 is input in Step 705, the process proceeds to Step 709. In Step 709, the position and size of the area selection object 502 in the structured document are changed according to the instruction input based on the mouse operation, and the process returns to Step 705.

  When an instruction to select the print button 512 is input in Step 705, the printing process in Step 706, Step 707, and Step 708 is executed, and the process returns to the input wait process in Step 705. First, in Step 706, the selection area data extraction unit 305 acquires the coordinates of the area selection object 502 inserted in the structured document. Next, in Step 707, the selection area data extraction unit 305 extracts the object and clip information arranged in the area overlapping the coordinates acquired in Step 706 from the Web page based on the structured document, and draws the intermediate data file. Note that this clip information indicates a position where an object to be cut out when a Web page is extracted in the above-described region. In Step 708, the print layout unit 306 arranges a print target in the print area based on the intermediate data file, and the print processing unit 308 executes the print process.

  With the above processing procedure, the structured document printing module 302 can select a partial area of the Web page displayed on the browser in accordance with the user's mouse operation, extract the selected area, and print it. Therefore, the user can designate and print a desired area of the Web page. In addition, since the selection area can be changed by a plurality of mouse operations, an arbitrary area is selected even when the size of the structured document is large and the screen needs to be scrolled to select the area. It becomes possible.

  In this embodiment, the selected area is displayed by inserting an object for area selection into the structured document. This method allows implementation in a platform-independent script language such as Java (registered trademark) Script, so that implementation with software using a script language in a Web application or the like is facilitated. Furthermore, in the present embodiment, the processing in the case where the structured document area is selected by the plug-in software of the browser has been described.

  A second embodiment which is another embodiment of the present invention will be described with reference to the drawings. The system configuration and PC configuration in the second embodiment are the same as those in FIGS. 1 and 2 of the first embodiment.

  FIG. 8 is a block diagram illustrating a software configuration of the PC 101 according to the second embodiment. A program operating on the PC 101 and its data processing configuration will be described below with reference to FIG.

  In FIG. 8, the browser 301 is the same as the browser 301 of the first embodiment, and thus details thereof are omitted. In FIG. 8, the structured document printing module 802 is plug-in software called from the browser 301, and the structured document printing module 802 is executed when the user gives an area selection instruction to the browser 301.

In the structured document printing module 802, the area selection screen control unit 803 displays an area selection window for selecting an arbitrary area of the structured document in front of the browser.

  FIG. 9 is a diagram illustrating an area selection window 901 according to the second embodiment. The area selection window 901 is a window as an area selection image composed of a transparent area and a semi-transparent selection rectangular area 902. An area selection window 901 composed of transparent and translucent areas is displayed in front of the structured document display area of the browser 301, so that the selected area can be displayed over the structured document.

  The area selection screen control unit 803 inputs an instruction to change the selection area performed by the user using the input device 207 such as a mouse, and changes the coordinates of the selection rectangular area 902 according to the instruction. As a result, the user can specify an arbitrary rectangular area of the structured document as a selection area using an input device such as a mouse.

  As described above, when the user selects an area in the structured document and instructs the browser to start printing or print preview of the area, the selected area data extraction unit 804 is executed. The selected area data extraction unit 804 acquires the coordinates of the area selection rectangle set by the area selection screen control unit 803, and the elements in the structured document included in the rectangular area are the same as those described in the first embodiment. Extract as an intermediate data file.

  The print layout unit 306, print preview unit 307, print processing unit 308, OS 309, and printer driver 310 are the same as those in FIG.

  As described above, in this embodiment, it is possible to select an arbitrary area of the structured document by displaying an area selection window composed of transparent and translucent areas on the front surface of the browser.

  In the second embodiment, the selection area is displayed in a window composed of transparent and semi-transparent areas different from the browser. However, it is necessary to change the position and size of the area selection window in accordance with the movement and size change of the browser. You need to monitor browser information. In the third embodiment, a method for solving the above problem will be described. The system configuration and the PC configuration in the third embodiment are the same as those in FIGS. 1 and 2 of the first embodiment.

FIG. 10 is a block diagram illustrating a software configuration of the PC 101 according to the third embodiment.
Hereinafter, a program operating on the PC 101 and its data processing configuration will be described with reference to FIG.

  In FIG. 10, since the browser 301 is the same as the browser 301 of the first embodiment, the details are omitted. In FIG. 10, the structured document printing module 1002 is plug-in software called from the browser 301, and when the user gives an area selection instruction to the browser 301, the structured document printing module 1002 is executed.

  In the structured document printing module 1002, the area selector window control unit 1003 generates an area selection window for selecting an arbitrary area of the Web page as a child window of the browser. The child window is a window that is arranged on the browser window and the position relative to the browser is fixed.

  FIG. 11 is a diagram illustrating a GUI screen of the area selection window according to the third embodiment. The area selector window control unit 1003 configures the area selection window 1101 as a child window of the browser, and places a child window of the same position and the same size as the area where the web page of the browser is displayed on the browser.

  As shown in FIG. 11, a web page is drawn in the area selection window 1101 based on the structured document acquired from the browser. The area selection window 1101 displays a scroll bar 1103 as necessary, and all areas of the Web page can be displayed by scrolling the scroll bar 1103.

  The area selection window 1101 further displays the area selection rectangle 1102 as a semi-transparent rectangle. The area selector window control unit 1003 inputs an instruction to change the selection area performed by the user using the input device 207 such as a mouse, changes the coordinates of the selection rectangular area 1102 in accordance with this input, and selects the selection rectangular area 1102. Redraw. As a result, the user can specify an arbitrary rectangular area of the structured document as a selection area using the input device 207 such as a mouse.

  After the user selects an area in the structured document in this way, the selected area data extraction unit 1004 is executed when the browser is instructed to start printing or print preview for the area. The selected area data extraction unit 1004 acquires the coordinates of the selected rectangle set by the area selector window control unit 1003, and, as in the first embodiment, elements in the structured document included in the rectangular area are intermediate data. Extract as a file.

  The print layout unit 306, print preview unit 307, print processing unit 308, OS 309, and printer driver 310 are the same as those in FIG.

  As described above, in this embodiment, an arbitrary area of the Web page can be selected by using the area selection window 1101 as a child window of the browser. In particular, in this embodiment, it is possible to cope with browser movement and size change without monitoring browser information.

  The present invention may supply a recording medium recording a program code of software that realizes the functions of the above-described embodiments to a system or apparatus. In that case, the object of the present invention is achieved by the computer (or CPU or MPU) of the system or apparatus reading and executing the program code stored in the recording medium. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiment, and the program code itself and the storage medium storing the program code constitute the present invention.

  The present invention is not limited to the implementation of the functions of the above-described embodiments by executing the program code read by the computer. For example, when the operating system (OS) running on the computer performs part or all of the actual processing based on the instruction of the program code, and the functions of the above-described embodiments are realized by the processing. There may be.

201 CPU
202 RAM
203 ROM
204 Hard disk 205 USB interface 206 Display device 207 Input device 208 LAN interface 209 System bus

Claims (16)

A specifying means for specifying a region to be extracted in a page on which a plurality of elements included in a structured document are arranged;
Extracting means for extracting image data corresponding to the extraction target area specified by the specifying means in the page;
When the boundary of the region to be extracted specified by the specifying unit corresponds to a part of one of the plurality of elements, the extracting unit selects the one element, and the selected 1 An image processing apparatus that creates image data based on one element and extracts the image data corresponding to the part by cutting out the image data corresponding to the part of the created image data .   The image processing apparatus according to claim 1, further comprising an output unit that outputs image data extracted by the extraction unit and arranged in a predetermined output area.   The extraction unit creates image data based on the one element so that the selected one element is arranged in the output area with a size corresponding to the predetermined output area. The image processing apparatus according to claim 1 or 2.   4. The image processing apparatus according to claim 1, wherein the output unit outputs the image data to a printing apparatus, and causes the printing apparatus to print an image based on the image data. 5. .   5. The image processing apparatus according to claim 1, wherein the output unit outputs the image data to a display device and causes the display device to display an image based on the image data. 6. .   The specifying means specifies the coordinates of the extraction target area in the page, and the extracting means extracts image data corresponding to the coordinates based on the structured document and the coordinates specified by the specifying means. The image processing apparatus according to claim 1, wherein:   The extraction unit acquires clip information indicating a position to be cut out in the one element based on the coordinates of the extraction target region and the coordinates of the one element, and the partial information is acquired based on the acquired clip information. The image processing apparatus according to claim 6, wherein the image data corresponding to is cut out.   The image processing apparatus according to claim 1, wherein the specifying unit specifies the extraction target region in a Web page based on the structured document.   9. The method according to claim 1, wherein the specifying unit specifies, as the extraction target area, an area specified by a user in the page displayed on a display device based on the structured document. The image processing apparatus according to item. Inserting means for inserting an element for specifying an area in the page, which is displayed in front of the page based on the structured document, in the structured document,
The specifying unit specifies an area specified by the element in response to a user instruction in a page displayed based on the structured document in which the element is inserted by the inserting unit. Item 10. The image processing apparatus according to Item 9.   The specifying unit changes the coordinates or size of an area specified by the element included in the structured document according to an instruction from a user, and extracts the area specified by the changed coordinate or size. The image processing apparatus according to claim 10, wherein the image processing apparatus is specified as a target area. Display control means for causing the display device to display an area selection image for specifying the area of the displayed page so as to overlap the page;
The said specifying means specifies the area | region specified by the said area | region selection image displayed by the said display control means as said extraction object area | region according to the instruction | indication by a user. The image processing apparatus according to claim 1.   The specifying unit changes a coordinate or size of a region specified by the region selection image included in the structured document according to an instruction from a user, and determines a region specified by the changed coordinate or size. The image processing device according to claim 12, wherein the image processing device is specified as the extraction target region.   The extraction unit creates image data corresponding to the part of the element of the vector data when corresponding to a part of the element of the vector data included in the structured document. The image processing apparatus according to claim 1, wherein the image data corresponding to a part of the element of the vector data is extracted by cutting out the image data corresponding to the part. Identify the extraction target area on the page where multiple elements included in the structured document are placed,
When a boundary of the specified region to be extracted corresponds to a part of one of the plurality of elements, the one element is selected, and image data based on the selected one element is selected. An image processing method characterized in that the image data corresponding to the part is extracted by cutting out the image data corresponding to the part of the generated image data. A program for causing a computer to function as each unit of the image processing apparatus according to any one of claims 1 to 14, or causing the computer to execute the image processing method according to claim 15.

Images