JP2017138829A - Image reconfiguration device, image reconfiguration method, and image reconfiguration program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To cut out an image of an arbitrary area displayed on a screen by a different application, and combine and display it.SOLUTION: An image reconfiguration device includes: an image acquisition part 123 for cutting out an image of an area specified on a window for each window of a plurality of applications from a window manager 21 for managing respective window information of the application; an image conversion part 124 for converting the image into an image at a specified scaling factor for each image of the acquired area; a display image generation part 125 for generating a display image adhered and combined at a position that designates the image for each converted image; and an image display part 126 for displaying the generated display image.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an image reconstruction device, an image reconstruction method, and an image reconstruction program.

  Conventionally, there is a need to cut out images displayed on the screen by different applications and display them in combination. For example, if the monthly sales and the monthly operating hours are displayed on the screen by different applications, if they can be combined and displayed on the screen, the efficiency of comparison between them will be improved. be able to.

  Here, in order to make it easier to compare images displayed on the screen by different applications, it is preferable to enlarge (or reduce) the display image. In order to enlarge (or reduce) the display image in this way, it is conceivable to use a magnifying glass application that accompanies Windows (registered trademark) (see Patent Document 1).

JP 2005-293508 A JP-A-2015-141434

  However, the above magnifying glass application cannot enlarge (or reduce) an image in a plurality of areas. Although it is conceivable to use a web mashup technology (see Patent Document 2), for example, an image of an arbitrary area of an application displayed on the desktop cannot be cut out and combined. Therefore, an object of the present invention is to solve the above-described problems and to cut out an image of an arbitrary area displayed on the screen by a different application and display it in combination.

  In order to solve the above-described problem, the present invention provides, from a window manager that manages window information of each application, an image acquisition unit that extracts an image of a specified area on the window for each window of a plurality of applications, and a clipping An image conversion unit that converts the image into an image with a specified magnification for each image in the region, and an image that generates a display image in which the image is pasted at the specified position for each converted image A generation unit and an image display unit that displays the generated display image are provided.

  According to the present invention, an image of an arbitrary area displayed on a screen by different applications can be cut out and displayed in combination.

FIG. 1 is a diagram illustrating an example of a display screen of the image reconstruction device. FIG. 2 is a diagram illustrating a configuration example of the system. FIG. 3 is a diagram illustrating an example of the cut / paste information. FIG. 4 is a flowchart showing a display image display procedure by the image reconstruction device. FIG. 5 is a flowchart showing a procedure for editing cut / paste information by the image reconstruction device. FIG. 6 is a diagram illustrating an example of a display screen of the image reconstruction device when editing cutout / paste information. FIG. 7 is a diagram illustrating a display example of the region to be edited in the image reconstruction device. FIG. 8 is a diagram illustrating a computer that executes an image reconstruction program.

  Hereinafter, embodiments (embodiments) for carrying out the present invention will be described with reference to the drawings. The present invention is not limited to the embodiments described below. First, the outline of the image reconstruction apparatus of the present embodiment will be described with reference to FIG.

  The image reconstruction device cuts out and enlarges (or reduces) an image of an arbitrary region displayed in a plurality of different application windows, and then displays a display image obtained by combining these images. For example, as shown in FIG. 1, the image reconstruction apparatus has an image of an arbitrary area displayed in the windows 101 and 102 of two different applications (for example, “No: 1, name: ********). ** ”on the Saturday and Sunday area images). The image reconstruction apparatus enlarges the image of the area cut out from the window 101 and then displays a display image combined with the image of the area cut out from the window 102 on the window 103.

  In this way, the user of the image reconstruction device can easily refer to an image in a region desired by the user among images displayed by different applications. As a result, it becomes easier for the user to compare images displayed by different applications.

(Constitution)
Next, a configuration example of a system including the image reconstruction device 10 according to the present embodiment will be described with reference to FIG. The system includes a plurality of applications, an OS (Operating System) 20, and an image reconstruction device 10.

  The OS 20 is basic software for operating an application on the system. The OS 20 includes a window manager 21. The window manager 21 manages the layout and appearance of each application window in a GUI (Graphical User Interface). The window manager 21 includes a window information management unit 211. The window information management unit 211 holds information (window information) indicating the layout and appearance of windows in the GUI.

  The image reconstruction device 10 includes a definition unit 11, an execution unit 12, and an editing unit 13. The definition unit 11 includes a cut / paste information management unit 111, and the cut / paste information management unit 111 holds the cut / paste information.

  Cutout / pasting information is the display image on the display window (for example, the window 103 in FIG. 1), which image of which window of the application is cut out, what percentage is enlarged (or reduced), which This is information indicating whether the area is pasted. The details of the cutting / pasting information will be described later with reference to the drawings.

  The execution unit 12 cuts out and enlarges (or reduces) an image of an arbitrary area displayed in the application window, and then displays a display image obtained by combining these images. Specifically, the execution unit 12 acquires the window information of each application window displayed on the desktop from the window information management unit 211. In addition, the execution unit 12 acquires cut / paste information from the definition unit 11. Then, the execution unit 12 cuts out an image of a predetermined area displayed in the window of the application displayed on the desktop based on the acquired window information and cut / paste information. Then, the execution unit 12 enlarges (or reduces) the clipped image, and then displays a display image obtained by combining these images.

  The execution unit 12 includes an execution management unit 121, a window acquisition unit 122, an image acquisition unit 123, an image conversion unit 124, a display image generation unit 125, and an image display unit 126.

  When the execution management unit 121 acquires the cut / paste information from the cut / paste information management unit 111, the execution management unit 121 displays a window acquisition unit 122, an image acquisition unit 123, an image conversion unit 124, a display image generation unit 125, and an image display unit 126. The display image is displayed by controlling.

  The window acquisition unit 122 acquires window information from the window information management unit 211. The image acquisition unit 123 is displayed on the window of the application displayed on the desktop based on the window information acquired by the window acquisition unit 122 and the cut / paste information acquired by the execution management unit 121. An image of a predetermined area is cut out. The image conversion unit 124 enlarges (or reduces) the image cut out by the image acquisition unit 123 based on the cut / paste information acquired by the execution management unit 121. That is, the image conversion unit 124 converts the clipped image into an image with a magnification specified in the clip / paste information.

  For each image whose magnification has been converted, the display image generation unit 125 pastes the image at a designated position and combines them to generate a display image. The image display unit 126 displays the display image generated by the display image generation unit 125 in a display window.

  The editing unit 13 accepts an editing operation for cut / paste information from the user. The editing unit 13 includes a definition information management unit 131, a cutout region management unit 132, and a pasting region management unit 133.

  The definition information management unit 131 manages cut / paste information (definition information) being edited. For example, the definition information management unit 131 displays a screen (for example, a menu screen) for accepting editing of cut / paste information on a display (not shown) connected to the image reconstruction device 10, and switches from this screen. Accepts editing of outgoing / pasted information. The menu screen includes, for example, an area for receiving an operation mode selection input, a display area for a list of cut / paste information, and a display area for detailed information of cut / paste information (see screen 201 in FIG. 6).

  For example, the definition information management unit 131 determines whether the display mode (display image display mode) or the editing mode (editing mode of cut / paste information) is received from the user depending on the area for receiving the selection of the operation mode on the menu screen. The selection input is accepted. Further, when the edit mode is selected by the user, the definition information management unit 131 displays a list of cut / paste information to be edited, and displays detailed information of the cut / paste information selected from this list. indicate. Then, the definition information management unit 131 accepts editing of the cut / paste information from the user on the detailed information display screen of the cut / paste information.

  The cut-out area management unit 132 displays a frame of the cut-out area on the window of the application based on the edit contents of the cut-out area among the edit contents of the cut-and-paste information received by the definition information management unit 131. . In addition, the pasting area management unit 133 displays a frame of the pasting area on the display window based on the editing contents of the pasting area among the editing contents of the cut / paste information received by the definition information management unit 131. By doing in this way, it becomes easy to confirm on the screen the cutout area and the pasting area that the user is editing.

  Next, the cutting / sticking information will be described in detail. The cut / paste information includes cut-out region information, image conversion method information when image conversion such as image enlargement (or reduction), and paste-up region information is performed. The information of the cut-out area is, for example, windowTitle (window name of the application window), ix, iy (cut-off start point), w, h (cut-out size) and the like shown in FIG. The information on the image conversion method is, for example, {r (x, y)} (magnification (reduction) rate of each image in the vertical and horizontal directions) shown in FIG. The information on the pasting area is, for example, dx, dy (pasting start point) shown in FIG. For example, when there are a plurality of cutout areas in the same window, the cutout / paste information includes {ix, iy, w, h, {r (x, y)}, dx as shown in FIG. , dy} are repeatedly described.

(Processing procedure)
Next, the processing procedure of the image reconstruction device 10 will be described. First, the display procedure of the display image by the image reconstruction device 10 will be described with reference to FIG.

  First, the execution unit 12 of the image reconstruction device 10 receives a display image display request from a user on a display window (S1). For example, the execution management unit 121 accepts a “display” selection input in the operation mode of the menu screen (see the screen 201 in FIG. 6). After S1, the execution management unit 121 acquires the cut / paste information group stored in the cut / paste information management unit 111 (S2), and should process it from the acquired cut / paste information group. One piece of window information (cutout / sticking information) is taken out (S3). For example, the execution management unit 121 extracts, from the cut / paste information, the cut / paste information of the window of the application selected by the user via the menu screen or the like. Further, the window acquisition unit 122 acquires the window information of the window currently displayed on the desktop from the window information management unit 211 (S4).

  After S4, the image acquisition unit 123 cuts out information (ix, iy (cutting start point), w, h (cutting size)) for the window of the processing target application indicated in the cut / paste information extracted in S3. Is obtained (S5), and an area (image of the area) having a width w and a height h is cut out from the cut start point (ix, iy) of the processing target window (S6).

  After S6, the image conversion unit 124 acquires the conversion information (r (x, y) (vertical (horizontal enlargement (reduction) rate)) of the processing target window indicated in the cutout / paste information extracted in S3. Then, an image obtained by multiplying the image of the region cut out in S6 by r is used as a pasted image (S7).

  After S7, the display image generation unit 125 acquires the start point (dx, dy (paste start point)) of the paste image for the processing target window indicated in the cut / paste information extracted in S3, and creates the paste created in S7. The image is pasted on the display image at the start point (dx, dy) (S8).

  After S8, when the execution management unit 121 determines that all the cutout information of the window extracted in S3 from the cutout / paste information has been processed (Yes in S9) and that all the windows to be processed have been processed. (Yes in S10), the image display unit 126 displays the display image created in the process up to S8 on the display window (S11).

  On the other hand, if the execution management unit 121 determines in S9 that there is unprocessed information extracted from the cutout / paste information in S3 (No in S9), the process returns to S5. In S10, if the execution management unit 121 determines that there is an unprocessed window to be processed (No in S10), the process returns to S3.

  By performing the above processing, the image reconstruction device 10 cuts out and enlarges (or reduces) an image of an arbitrary area displayed in the window of each application based on the cut / paste information, and then these images. The display image which combined can be displayed. For example, as shown in FIG. 1, the image reconstruction device 10 has an image of an arbitrary area displayed in the windows 101 and 102 of two different applications (for example, “No: 1, Name: *****”). The image of the area of “Saturday and Sunday of ***” is cut out, the image of the area cut out from the window 101 is enlarged, and the display image combined with the image of the area cut out from the window 102 is displayed on the window 103 To do.

  When the image reconstruction device 10 updates the display image in response to the update of the display contents in the application window, the processing from S2 in FIG. 4 is performed again after a predetermined time has elapsed from the processing in S11 in FIG. Just do it. The predetermined time can be appropriately set by the user, for example, on the menu screen shown in FIG.

  Next, with reference to FIG. 5, a procedure for editing cutout / paste information by the image reconstruction device 10 will be described. Here, a case will be described as an example where a set of cutout information (cutout region) and paste information (paste region) is added to the cutout / paste information held by the cutout / paste information management unit 111.

  First, when the definition information management unit 131 of the image reconstruction device 10 receives an edit request for cut / paste information from a user (S21), the cut / paste information accumulated in the cut / paste information management unit 111 is received. A group is acquired and displayed as a list (S22: list display). For example, when the definition information management unit 131 accepts a selection input of “edit” in the operation mode of the menu screen (see screen 201 in FIG. 6), the acquired cut / paste is displayed in the “list” display field of the menu screen. Display a list of information groups.

  After S22, when the definition information management unit 131 receives an input for adding a set of clipped area and paste area information (S23), the set of cutouts input in S23 is displayed in the list displayed in S22. An area and a pasting area are added (S24). For example, when the definition information management unit 131 receives selection input of the “insert” button in the “list” display field of the menu screen (see FIG. 6), the definition information management unit 131 inputs to add information on one set of cutout region and pasting region. Accept. Then, the definition information management unit 131 adds the set of cutout area and pasting area input in S23 to the “list” on the menu screen.

  After S24, the definition information management unit 131 displays a GUI that can edit and display the cut and paste information (windowTitle, ix, iy, w, h, {r (x, y)}, dx, dy) in the detailed information. Then, the editing is accepted (S25). That is, the definition information management unit 131 displays on the screen a GUI that accepts editing of detailed information of the pair of cutout areas and pasting areas added in S24. For example, the definition information management unit 131 displays in the “detailed information” column the detailed information on the pair of cutout areas and pasted areas added to the “list” display column on the menu screen (see FIG. 6). Editing input such as window name (windowTitle), cut-out area (ix, iy, w, h), magnification (rx, ry), sticking start point (dx, dy) is accepted.

  Note that editing input of cut / paste information may be accepted by numerical input on the menu screen, etc. as described above, or by operation on the application window or display window entered on the window title on the screen. May be.

  By performing the above processing, the image reconstruction device 10 can edit the cut / paste information. Here, the case of adding the cut / paste information has been described as an example, but the image reconstruction device 10 may of course edit the existing cut / paste information. In that case, the image reconstruction device 10 may receive a selection input of the cut / paste information to be edited from the user, and then accept an edit input of the selected cut / paste information.

  Note that after S25, the pasting area management unit 133 displays the pasting area (for example, the area 202 in FIG. 6) input in S25 on the display window (S26), and the cutout area management unit 132 performs S25. Then, a cut-out area (for example, the area 203 on the window indicated by reference numeral 203 in FIG. 6) may be displayed on the window of the application input in windowTitle (S27).

  In this way, the user of the image reconstruction device 10 performs editing while confirming on the screen which area on which application window is to be cut out and pasted into which area of the display window. It can be carried out.

  According to the image reconstruction device 10 described above, images of arbitrary regions displayed on the screen by different applications can be cut out and displayed in combination. As a result, the user can easily compare images displayed on the screen using different applications, and reduce mistakes during the comparison.

(Other embodiments)
In addition to the above, the image reconstruction device 10 may accept movement of the cutout area and the pasting area and a change in size from the screen as described below.

  For example, when the display mode is selected, the image reconstruction device 10 selects a clipping area on the application window or a pasting area on the display window by double clicking or the like, and the user edits the area. Display so that it can be recognized as a target.

  For example, when the pasting area indicated by reference numeral 301 on the display window shown in FIG. 7 is selected in the state where the display mode is selected, the image reconstruction device 10 refers to the area and the area. A cut-out area (area indicated by reference numeral 302) on the original application window is displayed by being surrounded by a dashed-dotted line. Thereafter, when the image reconstruction device 10 receives an instruction to move the cutout area (area indicated by reference numeral 302) on the application window on the screen, for example, the movement is reflected in the cutout / paste information. Then, the image reconstruction device 10 again creates a display image in which the information of the area indicated by the reference numeral 302 after the movement is reflected in the area indicated by the reference numeral 301 based on the cut / paste information after the movement is reflected, Display in the display window.

  In addition, when the image reconstruction apparatus 10 receives a cutout area, a movement of a pasting area, and a change in size from the screen, a menu screen (see the screen 201) may be displayed together. By doing in this way, even when the image reconstruction device 10 cannot display the cutout area and the pasted area on the screen, the user can check the cutout area and the pasted area.

  Although the image reconstruction device 10 has been described as using the cut / paste information edited by the editing unit 13, the present invention is not limited to this. For example, the image reconstruction device 10 may create a display image using the cut / paste information edited by the external device and display it on the display window.

  In addition, the image reconstruction device 10 performs the enlargement (or reduction) of the image of the region cut out from the application window according to the enlargement (reduction) ratios in the vertical and horizontal directions described in the cut and paste information. It is not limited to this. For example, when each of the areas A and B cut out from the windows of two applications includes a character image, the image reconstruction device 10 causes the character size of the area B to be the same as the character size of the area A. The size of the character image in the area A or the area B may be automatically enlarged (or reduced).

(program)
Further, it can be implemented by installing an image reconstruction program for realizing the functions of the image reconstruction device 10 described in each embodiment in a desired information processing apparatus (computer). For example, the information processing apparatus can function as the image reconstruction apparatus 10 by causing the information processing apparatus to execute the image reconstruction program provided as package software or online software. The information processing apparatus referred to here includes a desktop or notebook personal computer. In addition, the information processing apparatus includes mobile communication terminals such as smartphones, mobile phones and PHS (Personal Handyphone System), and straight terminals such as PDA (Personal Digital Assistants). Further, the image reconstruction device 10 may be implemented as a Web server or a cloud.

  An example of a computer that executes an image reconstruction program will be described below. FIG. 8 is a diagram illustrating a computer that executes an image reconstruction program. As shown in FIG. 8, the computer 1000 includes, for example, a memory 1010, a CPU (Central Processing Unit) 1020, a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network. Interface 1070. These units are connected by a bus 1080.

  The memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM (Random Access Memory) 1012. The ROM 1011 stores a boot program such as BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to the hard disk drive 1090. The disk drive interface 1040 is connected to the disk drive 1100. A removable storage medium such as a magnetic disk or an optical disk is inserted into the disk drive 1100, for example. For example, a mouse 1110 and a keyboard 1120 are connected to the serial port interface 1050. For example, a display 1130 is connected to the video adapter 1060.

  Here, as shown in FIG. 8, the hard disk drive 1090 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. Various data such as the cut / paste information described in the above-described embodiment is stored in the hard disk drive 1090 or the memory 1010, for example.

  Then, the CPU 1020 reads out the program module 1093 and the program data 1094 stored in the hard disk drive 1090 to the RAM 1012 as necessary, and executes the above-described procedures.

  Note that the program module 1093 and the program data 1094 related to the image reconstruction program are not limited to being stored in the hard disk drive 1090. For example, the program module 1093 and the program data 1094 are stored in a removable storage medium and the CPU 1020 via the disk drive 1100 or the like. It may be read out. Alternatively, the program module 1093 and the program data 1094 related to the image reconstruction program are stored in another computer connected via a network such as a LAN (Local Area Network) or a WAN (Wide Area Network), and the network interface 1070 is stored. Via the CPU 1020.

DESCRIPTION OF SYMBOLS 10 Image reconstruction apparatus 11 Definition part 12 Execution part 13 Editing part 21 Window manager 111 Cutout / paste information management part 121 Execution management part 122 Window acquisition part 123 Image acquisition part 124 Image conversion part 125 Display image generation part 126 Image display part 131 Definition Information Management Unit 132 Cutout Area Management Unit 133 Pasting Area Management Unit 211 Window Information Management Unit

Claims (4)

From a window manager that manages window information of each application, an image acquisition unit that cuts out an image of a specified area on each window of each of a plurality of applications,
An image conversion unit that converts the image into an image with a specified magnification for each image in the clipped area;
For each converted image, an image generation unit that generates a display image in which the image is pasted and combined at a specified position;
An image display unit for displaying the generated display image;
An image reconstruction apparatus comprising:   The image reconstruction device according to claim 1, further comprising an editing unit that receives designation of at least one of the image cutout region, the image conversion magnification, and the position where the image is pasted. For each window of a plurality of applications, a step of cutting out an image of a specified area on the window from a window manager that manages window information of each application;
For each image of the acquired area, converting the image into an image with a specified magnification;
Generating a combined display image by pasting the image at a designated position for each converted image;
Displaying the generated display image;
An image reconstruction method comprising: For each window of a plurality of applications, a step of cutting out an image of a specified area on the window from a window manager that manages window information of each application;
For each image of the clipped area, converting the image to an image with a specified magnification;
Generating a combined display image by pasting the image at a designated position for each converted image;
Displaying the generated display image;
Image reconstruction program for causing a computer to execute.

Images