JP4047368B2 - Image editing apparatus and control method thereof - Google Patents

Description

  The present invention relates to an image editing apparatus and a control method thereof.

  In a conventional document editing system, not only text data but also text data mixed with image data can be edited. Currently, such a document editing system is realized by running application software for document editing capable of mixing image data on a personal computer.

  FIG. 27 is a block diagram showing a hardware configuration of a conventional document editing system. In the figure, reference numeral 304 denotes a display device composed of a CRT, a video board or the like. Reference numeral 306 denotes an input device including a mouse, a keyboard, and the like, and 307 denotes a main storage device including a DRAM and the like. Reference numeral 308 denotes a non-volatile secondary storage device including a hard disk drive and a hard disk.

  An operating system 302 performs comprehensive management for effectively using the hardware 301 and the application software group 303 in the document editing system. In this operating system 302, it is possible to simultaneously execute a plurality of application software, as represented by Windows 95 which is an operating system of Microsoft Corporation, and the display device 304 has a plurality of windows as user interfaces of each application. It can be displayed at the same time.

  Of the operating system 302, reference numeral 314 is a drawing management system that displays characters, graphics, images, and the like requested on the display device 304 in response to a request from application software included in the application software group 303. Reference numeral 315 denotes an input device management system for delivering a user event input via the input device 306 to an appropriate software module of the document editing system.

  Reference numeral 316 denotes a memory management system that manages reading and writing of data to the main storage device 307. A file system 317 manages files in the secondary storage device 308. Reference numeral 318 denotes an inter-application communication unit that enables data transmission / reception between applications in the application software group 303 and activation of other applications. A typical example of such inter-application communication technology is Microsoft's OLE2 technology (see Non-Patent Document 1 for more detailed contents of OLE2 technology).

  Of the application software group 303, 319 is a document editing application. A document editing application 319 receives data storage / reading means 322 for storing and reading document data edited by a user, and image display data edited on the image editing application 321 from the image editing application 321 by inter-application communication. The image editing result display means 323 that requests the display management system 314 to display the received display data and displays it on the display device 304, and the image editing application that activates the image editing application 321 by inter-application communication in accordance with a command input from the user. The starting means 324 and other means not shown in FIG. 27 such as editing of text data are included.

  An image filing application 320 is an application that facilitates the search of a large number of files stored as files in the secondary storage device 308. The image filing application 320 may transmit the image data of the image data file designated by the user from the image data file group 330 to the document editing application 319 or the image editing application 321 by inter-application communication in accordance with the user designation. it can.

  The image editing application 321 is an application for performing image processing such as performing image processing such as blurring and color adjustment on the original image or drawing a figure on the original image. The image editing application 321 includes an image data pasting unit 325 that provides a function of switching the image received from the image filing application 320 and the current image, and a data storage / reading unit that reads / writes image data from / to the secondary storage device 308. 326, user interface management means 327 for managing user events from the display device 304 such as images, menus, and application frameworks and user devices from the input device 306, various image processing filters such as blurring, brightness adjustment, color adjustment, etc. Image editing means group 328 including a plurality of image editing means including drawing tools such as giving and character assignment, and image editing execution means for executing image editing means from image editing means group 328 in response to a user input command 331 Composed of other means not shown in the beauty Figure 27.

  Here, in the OLE2 technology, the image editing application 321 that supplies data through inter-application communication is referred to as an OLE server, and the document editing application 319 that receives data through inter-application communication is referred to as an OLE container.

  Reference numeral 329 denotes a document data file edited by the document editing application 319. FIG. 28 is an explanatory diagram showing the data structure of document data. In the figure, a header data storage unit 401 stores information such as text font, color, paper size for printing, vertical writing or horizontal writing. A text data storage unit 402 stores text data input by a user.

  Reference numeral 403 denotes an image editing application data storage unit (stores “object” in OLE2 technology), and an image editing application identification information storage unit for uniquely recognizing an image editing application 321 that supplies image data through inter-application communication. 404, an image position data storage unit 405 for storing information on the position and size of image data in document data, and a display data storage unit 406 for storing image data to be displayed on the display device 304 (in the case of OLE2 technology). Image data in the metafile format) and image data for use by the image editing application 321 specified in the image editing application identification information storage unit 404 (image data not supported by the document editing application 319). A mat also called native data in .OLE2 technology may be, an image data storage unit 407 that stores not limited to image data).

  Reference numeral 408 denotes another data storage unit including other data that is not important for explaining the present invention. When a plurality of images are embedded in a document, data storage for image editing applications corresponding to the number of images is stored. Part.

  It should be noted that Microsoft WORD ver. In recent document editing applications represented by 6.0, a sample of a document data file is often attached to application software. The user can easily create a high-quality document without creating the entire design from scratch by correcting the text in such a sample document or replacing the image.

  FIG. 29 is an explanatory diagram showing a user interface screen of a conventional document editing application 319 displayed on the display device 304. In the figure, reference numeral 501 denotes a window serving as a frame of the document editing application 319. Reference numeral 502 denotes a document 1 document, which is displayed in an image display area 503, which is a text input by a user and an image received by inter-application communication from an image editing application 321 activated as an OLE server.

  Reference numeral 504 denotes a Save button for inputting a command for saving the edited document data as a file. Reference numeral 505 denotes an Open button for opening a document file on the window 501 and making it editable. Reference numeral 506 denotes an InsertObject button for inputting a command for displaying data provided by another OLE server application in the document.

  FIG. 30 is an explanatory diagram showing a user interface screen of the image filing application 320 displayed on the display device 304. A list of image files searched by keyword search or the like from the image data file group 330 stored in the secondary storage device 308 is displayed on the display device 304. Reference numeral 601 denotes a window serving as a frame of the image filing application. Reference numeral 602 denotes a reduced image display area of the searched image file.

  FIG. 31 is an explanatory diagram showing a user interface screen of an image editing application 321 that supplies image data as an OLE server to other applications in the document editing system. In the figure, reference numeral 701 denotes a frame window of the image editing application.

  A window 702 displays the image data currently being edited. Reference numerals 703 to 713 denote buttons for inputting execution instructions of the respective image editing means constituting the image editing means group 328. Reference numeral 703 denotes a blur processing execution request button. When the user clicks this button with the mouse, the image editing application 321 performs blur processing (reducing the spatial resolution of the image) on the image data displayed in the window 702. Process).

  Reference numeral 704 denotes a sharpening processing execution request button. When the user clicks this button with the mouse, the image editing application 321 performs sharpening processing (processing for clarifying the edge of the image) on the image data displayed in the window 702. ). Reference numeral 705 denotes a brightness adjustment processing execution request button. When the user clicks this button with the mouse, the image editing application 321 performs brightness adjustment processing (appropriately adjusting the brightness of the image) on the image data displayed in the window 702. Correction process).

  Reference numeral 706 denotes a color adjustment processing execution request button. When the user clicks this button with the mouse, the image editing application 321 performs color adjustment processing (appropriately adjusting the color of the image) on the image data displayed in the window 702. Correction process).

  Reference numeral 707 denotes a background erasure processing execution request button. When the user clicks this button with the mouse, the image editing application 321 performs background erasure processing (removing a background such as a person image) on the image data displayed in the window 702. To be removed). Reference numeral 709 denotes a logo processing execution request button, and when the user clicks this button with the mouse, the image editing application 321 performs logo processing (characters from an image including characters in the subject) on the image data displayed in the window 702. A process of creating a logo by cutting out the portion and adding decoration to the characters is performed.

  Reference numeral 710 denotes a noise addition processing execution request button. When the user clicks this button with the mouse, the image editing application 321 applies noise addition processing (adds noise to the image) to the image data displayed in the window 702. To make it like a painting).

  Reference numeral 711 denotes an execution request button for deformation processing. When the user clicks this button with the mouse, the image editing application 321 performs deformation processing (processing to deform an image) on the image data displayed in the window 702. . Reference numeral 712 denotes an inversion processing execution request button. When the user clicks this button with the mouse, the image editing application 321 inverts the image data displayed in the window 702 (processing to invert the brightness of the image). Apply.

  In the conventional document editing system having the above-described configuration, when a user double-clicks an image 503 in a document with a mouse, the image editing application activation unit 324 of the document editing application 319 stores the inside of the image editing application identification information storage unit 404. The image editing application 321 that supplies image data is uniquely recognized as an OLE server application, and the image editing application 321 is activated, and the image display area 503 is set as a user interface area of the image editing application 321.

  Further, when the image editing application 321 is started as an OLE server by the OLE container application, the user interface management unit 327 sets the image display area 503 as its own user interface area to receive a mouse input from the user and is necessary for image editing. A predetermined menu is added to the menu display area 507.

  In the OLE2 technology, the state where the image editing application 321 is activated in the OLE container application is called an in-place active state. FIG. 32 is an explanatory diagram showing a state in which the image editing application 321 is activated in the in-place active state in the document editing application 319.

  In the in-place active state, diagonal lines are displayed around the image display area 503, and a menu for command input to the predetermined image editing application 321 is displayed in the menu display area 507, and a command input waiting state is entered.

  On the other hand, when the image editing application 321 is in the in-place active state, an arbitrary image is selected by pressing the mouse button in the reduced image display area of the arbitrary image of the image filing application 320, and the document is edited by dragging the mouse as it is. When dropped in the image display area 503 of the application 319, the image filing application 320 transmits the image data of the selected file to the active image editing application 321 in the in-place active state, and the image editing application 321 that has received this image data. The image data pasting means 325 pastes the received image on the image displayed in the image display area 503 until now.

  FIG. 33 shows File2. It is explanatory drawing which shows the state by which the image of bmp was affixed. In this state, when the user clicks an arbitrary menu from the image editing menus 703 to 712 displayed in the menu display area 507, the image editing execution means 331 of the image editing application 321 changes to the menu clicked by the mouse. A corresponding image editing unit is selected from the image editing unit group 328 and image editing is executed on the image data displayed in the image display area 503. Then, after execution of image editing, when the user clicks the mouse anywhere in the document 501 document window 501 other than the image display area 503, the image editing application 321 changes from the in-place active state to the inactive state. At this time, the image editing application 321 transmits the image data as the editing result to the document editing application 319.

On the other hand, the document editing application 319 displays the received editing result image data in the image display area 503. In the document editing application 319, when a user needs to save data when inputting a data saving request command, the data saving / reading means 322 of the document editing application 319 includes a header data storage unit 401, a text data storage unit. The document data corresponding to 402 is saved, and the image editing application 321 is requested to save data in the image editing application data storage unit 403. In response to this, the data storage / reading means 326 of the image editing application 321 stores the data in the image editing application data storage unit 403 in the data format shown in FIG.
Inside OLE2 from Microsoft PRESS Publishing

  However, the conventional document editing system has the following problems, and it is difficult for a user unfamiliar with image editing.

  That is, the image editing application 321 started from the document editing application 319 has to display a large number of menus corresponding to all images of all document data in order to display a predetermined menu (FIG. 32). 703-712). Selecting an image editing means suitable for the image of the document currently being edited from a large number of menus has been a heavy burden for users who are unfamiliar with image editing. In particular, when an object in an image is cut out and pasted into a document, it is difficult for a user unfamiliar with image editing to handle it.

  On the other hand, there is a problem that image data has a large amount of information, occupies a large storage area of the storage medium, and spends a lot of time for reading / writing from the storage medium. In order to avoid this problem, it is necessary to hold the image data as small as possible. Since the required image resolution is unknown when the application is not clear, the image size cannot be reduced, but the image data in the document is clearly how the image is used in the document, so the required resolution It is possible to reduce the image to the optimum size based on the above. However, in the conventional document editing system, it is difficult for a user who is unfamiliar with image editing because the user has to determine whether or not to reduce the image or the image size to be selected when the image is reduced. It was work.

  SUMMARY OF THE INVENTION An object of the present invention is to provide an image editing apparatus that can be easily handled by a user unfamiliar with image editing and a control method therefor.

In order to achieve the above object, an image editing apparatus according to the present invention is an image editing apparatus that edits an image displayed on a display unit, and designates a point on the contour of an object in the image based on a user operation. Means for extracting a contour line between the first point designated by the designation means and the second point designated following the first point based on edge information of the image; Determining means for determining whether or not a distance between the first point and a second point designated following the first point is equal to or greater than a predetermined value; and If it is determined that the distance from the second point specified after the first point is equal to or greater than a predetermined value, the contour line is extracted by the extraction means , and the first point and determine a distance between the second point specified following the first point is less than the predetermined value If it is, and control means shall be the contour of the straight line connecting between the second point specified following the first point and the first point.

An image editing apparatus control method according to the present invention is an image editing apparatus control method for editing an image displayed on a display means, and a step of designating a point on the contour of an object in the image based on a user operation. Extracting a contour line between the designated first point and the designated second point following the first point based on edge information of the image; and the first point wherein the step of distance between the second point is determined whether a predetermined value or more subsequently specified in the first point, the first point and the first point by said step of determining that followed when the distance between the second point is specified is determined to be the predetermined value or more, to extract the contour lines Ri by the step of the extraction, the said first point first If the distance between the second point specified following the first point is determined to not more than the predetermined value, the first Point and ing and a step shall be the contour of the straight line connecting between the second point specified following the first point.

The image editing apparatus according to the present invention can be easily handled by a user unfamiliar with image editing even when an object in an image is cut out. Further, according to the control method of the image editing apparatus according to the present invention, even when an object in an image is cut out, a user unfamiliar with image editing can easily handle it.

  Embodiments of an image editing apparatus and a control method thereof according to the present invention will be described.

[First Embodiment]
FIG. 1 is a block diagram showing a configuration of a document editing system according to the first embodiment. The document editing system generally includes an application software 101, an operating system 302, and hardware 301. The application software 101 includes a document editing application 319, an image filing application 320, an image editing application 102, and the like. Note that the same components as those in the document editing system described in the section of the related art are denoted by the same reference numerals, and the description thereof is omitted. FIG. 2 is an explanatory diagram showing the structure of document data.

  In FIG. 29 showing the user interface screen of the document editing application 319, when the document editing application 319 is editing the document 1, when the image display area 503 is double-clicked with the mouse pointer, the image editing application starting means 324 starts the image editing application 102. To do. FIG. 5 is a flowchart showing a processing procedure when the image editing application 102 is activated by the document editing application 319. FIG. 4 is an explanatory diagram showing a memory map in a memory area used by the image editing application 102 in the main storage device 307. In the figure, reference numeral 1101 denotes each means execution code holding area including execution format codes of each means of the image editing application 102. Reference numeral 1102 denotes a recommended image size data holding area for holding recommended image size data. Reference numeral 1103 denotes a processing script data holding area that holds processing script data. Reference numeral 1104 denotes an image data holding area for holding current image data. Reference numeral 1105 denotes an editing result image data holding area for storing and holding image data obtained as a result of image editing by the image editing means of the image editing means group 328. Reference numeral 1106 denotes another data holding area for holding other data used by the image editing application 102.

  When the image editing application 102 is activated, the processing of the document editing system will be described in detail with reference to the memory map of FIG. 4 and the flowchart of FIG. When the image editing application 102 is activated, first, image data is read from the image data storage unit 407 into the image data holding area 1104 by the data storage / reading means 105 (step S1). In the image data holding area 1104, images 1. bmp image data is stored.

  The processing script data is read from the processing script storage unit 202 into the processing script data holding area 1103 by the data storage / reading means 105 (step S2). The data storage / reading means 105 reads the recommended image size from the recommended image size storage unit 203 into the recommended image size data holding area 1102 (step S3). The user interface management unit 108 displays the image data stored in the image data holding area 1104 in the image display area 503 in the document (step S4), and ends the processing at the time of activation.

  When the image editing application 102 finishes the startup process, the document editing application 319 waits for an event (command input waiting) from the user in the in-place active state.

  FIG. 6 is an explanatory diagram showing a display screen of the display device 304 in an in-place active state and waiting for an event from a user. Unlike the conventional case shown in FIG. 32, the user interface management means 108 of the image editing application 102 does not add a menu for inputting an image editing command to the menu display area 507 of the document editing application 319.

  FIG. 7 is an explanatory diagram showing input of a command for transmitting image data from the image filing application 320 to the image editing application 102 by inter-application communication. In the document editing system, the mouse cursor is moved onto the image to be transmitted by the image filing application 320 (for example, File2.bmp in FIG. 7), the mouse button is pressed, and the mouse cursor is moved to the image display area while keeping the button pressed. Move up to 503 and release the mouse button. Such an operation is generally called drag and drop. In the document editing system, an image data transmission command is sent to the inter-application communication unit 318 by performing a drag and drop operation from the reduced image of the image filing application 320 to the image editing application 102 in the in-place active state in the document editing application 319. Can be entered.

  Specifically, File2. A case where bmp image data is dragged and dropped by the image editing application 102 will be described as an example. FIG. 8 is a flowchart showing a processing procedure of the document editing system when an image data transmission command is input from the user by a drag and drop operation.

  When the user inputs an image data transmission command by a drag and drop operation, the process is started. First, the image data replacement means 103 stores the image 1. bmp deletes the image data of bmp, and instead receives the File2. By storing the bmp image data in the image data holding area 1104, the image data in the image data holding area 1104 is replaced (step S51).

  The image size conversion means 112 is a file 2... Stored in the image data holding area 1104. The bmp image data is resized to the recommended image size stored in the recommended image size data holding area 1102 (step S52).

  The process script analysis unit 106 analyzes the process script data stored in the process script data holding area 1103 (step S53).

  FIG. 3 is an explanatory diagram showing processing script data. FIG. 2A describes two processing procedures, processing procedure 1 and processing procedure 2. Processing procedure 1 includes setting Parameter 1 to a value of 10 to perform blurring processing on the original image, and further setting Parameter1 to a value of 20 to perform illustration processing on the image resulting from the blurring processing. A processing script is described.

  In the processing procedure 2, the parameter 1 is set to the value 10 and the original image is sharpened, and the parameter 10 is set to the value 15 and the parameter 2 is set to the value 30 and the noise processing is performed on the result image of the sharpening process. A processing script whose content is to apply is described. A plurality of processing procedures are described in the processing script data in FIG. Here, such a processing script is called a selection-type processing script.

  Further, only one processing procedure is described in the processing script data in FIG. Such a processing script is referred to herein as a non-selection type processing script.

  Based on the analysis result in step S53, it is determined whether the processing script is a selection type or a non-selection type (step S54). If the processing script data stored in the processing script data holding area 1103 is a selection type as shown in FIG. 5A, the process proceeds to step S57. If the processing script data is a non-selection type, the process proceeds to step S55. Proceed.

  In step S55, the image editing automatic execution means 107 performs processing described in the processing script data for the image data stored in the image data holding area 1104 (image data of file2.bmp resized in step S52). The image editing unit corresponding to the procedure is searched from the image editing unit group 328 and executed. Parameters for executing each image editing means also follow the description of the processing script data.

  Then, the user interface management means 108 displays the image data of the edited result in accordance with the processing script stored in the image data holding area 1104 in the image display area 503 (step S56), and the process is terminated.

  On the other hand, if the processing script is the selection type in step S54, the value 1 is substituted into the counter k (step S57). The image data stored in the image data holding area 1104 (file2.bmp image data resized in step S52) is copied to the editing result image data holding area 1105 according to the processing procedure k.

  FIG. 9 is an explanatory diagram showing a memory map of the editing result image data holding area 1105 according to the processing procedure k. The image editing automatic execution means 107 executes the processing procedure k described in the processing script data for the image data stored in the editing result image data holding area 1105 (step S59). When executing the process, the image editing unit 328 is searched for an image editing unit corresponding to the processing procedure k. Here, the parameters for executing the image editing means also follow the description of the processing procedure k.

  It is determined whether or not the processing procedure k is the last processing procedure in the processing script data (step S60). For example, when the processing script data is FIG. 3A, since the processing procedure 2 is the last processing procedure of the processing script, the determination in step S60 is yes when k = 2, and the processing proceeds to step S62. . If the process procedure k is not the last process procedure of the process script, the determination is no, and the process proceeds to step S61 to increment the value of the counter k, and then the process of step S58 is performed again.

  In step S62, a candidate image display window is displayed. FIG. 10 is an explanatory diagram showing a candidate image display window. In the candidate image display window, images of the editing results of processing procedure 1 to processing procedure n stored in the editing result image data holding area 1105 are displayed. The user can select an image having a desired editing result from the candidate image display window.

  In the present embodiment, a command for selecting an image as an editing result is input by moving the mouse cursor over a desired image and clicking a mouse button.

  It is determined whether or not a selection command has been input by the user (step S63). Until the selection command is input, the determination in step S63 is no, and the determination in step S63 is repeated. When the user inputs a selection command by clicking the mouse, the determination in step S63 is yes. Then, the image data stored in the image data holding area 1104 is deleted (step S64). The image data of the editing result by the processing procedure selected by the user is copied from the editing result image data holding area 1105 to the image data holding area 1104 (step S65).

  The candidate image display window shown in FIG. 10 is closed (step S66). All the image data of the edited result by the processing procedure 1 to the processing procedure n stored in the editing result image data holding area 1105 is deleted (step S67), and the editing result image data is displayed in the image display area 503 (step S67). S56), the process is terminated.

  After the image editing process by the image editing application 102 is completed, the user clicks a part other than the image display area 503 in the window 502 of the document document 1 so that the image editing application 102 is inactive from the in-place active state. It becomes a state. The image editing application 102 stores the recommended image size data, the processing script data, and the editing result image data in the recommended image size storage unit 203, the processing script storage unit 202, and the image data storage unit 407, respectively, before entering the inactive state. At the same time, display data is created and stored in the display data storage unit 406.

  On the other hand, the image editing result display means 323 of the document editing application 319 displays the display data stored in the display data storage unit 406 in the image display area 503.

  In the document editing system according to the present embodiment, when a user inputs an image replacement command by a drag-and-drop operation, if the processing script is a non-selection type, the image editing is performed fully automatically, so that the user is not familiar with image editing. Can reduce the burden of a user having to input a number of image editing commands and parameters. Further, even if the processing script is a selection type, it is possible to easily perform image editing that is preferable for the user simply by selecting a desired image while viewing several automatically edited processing result images. Therefore, it is not necessary to perform a burdensome operation such as selecting an image editing means suitable for the image of the document currently being edited from many menus, and even a user unfamiliar with image editing can easily handle it. .

[Second Embodiment]
A document editing system according to the second embodiment will be described. Since the document editing system in the present embodiment has the same hardware configuration as the document editing system in the above-described embodiment, the description thereof is omitted. FIG. 11 is a flowchart showing a processing procedure of the document editing system according to the second embodiment. When the user inputs an image data transmission command by a drag and drop operation in the in-place active state (see FIG. 7), the process is started.

  The image data exchanging means 103 is an image 1. bmp deletes the image data of the bmp and instead receives the File2. By storing the bmp image data in the image data holding area 1104, the image data in the image data holding area 1104 is replaced (step S81). The image size conversion means 112 is a file 2... Stored in the image data holding area 1104. The bmp image data is resized to the recommended image size stored in the recommended image size data holding area 1102 (step S82). The processing script analysis means 106 analyzes the processing script data stored in the processing script data holding area 1103 (step S83).

  FIG. 12 is an explanatory diagram showing processing script data according to the second embodiment. In the processing script data in the present embodiment, the name of a template to be used in image editing is described. The template is a window in which only the user interface of the image editing means required for each purpose of image editing is compactly collected among the image editing means included in the image editing means group 328. The “portrait effect template” described in the processing script data of FIG. 12 is an image that “edits an image in which a person is photographed into a preferable image for pasting as a face photograph of a letter document sender”. A window with a minimal user interface to achieve the purpose of editing.

  The user interface selection unit 109 selects a template described in the processing script according to the analysis result in step S83, and displays each template window (step S84). For example, when the processing script data designates the portrait effect template of FIG. 12, the window shown in FIG. 13 is displayed on the display device 304 of the document editing system. FIG. 13 is an explanatory diagram showing a user interface of a portrait effect template. Details of image editing performed by the portrait effect template will be described later. In this way, the user interface selection means 109 selectively displays a minimum user interface that matches the purpose, so that the user can immediately determine which menu should be selected and is not used to image editing. User's operability can be improved.

  User interaction processing for each template is executed via the template window (step S85), and image editing is performed on the image stored in the image data holding area 1104. When the user finishes the image editing by the interactive process and clicks the OK button on the template with the mouse, the template window is closed (step S86). The user interface management unit 108 displays the editing result image data stored in the image data holding area 1104 in the image display area 503 (step S87), and ends the process.

  The document editing system according to the present embodiment does not display a predetermined menu or tool, but selectively displays a minimum user interface in order to achieve the purpose of image editing in accordance with the description of the processing script. Therefore, a user unfamiliar with image editing does not have to select a menu suitable for the purpose from among a large number of image editing menus, and the burden on the user can be reduced.

  The user interface of the portrait effect template in FIG. 13 will be described in detail. The portrait effect is an image edit used for a photograph of the author's face on a document or an attendee's face on a minutes. The most important process in the portrait effect is to remove unnecessary background from the face photo. In order to realize this function, the portrait effect has two modes.

  One of them is a cut-out mode indicated by 2000 in FIG. 4A, where cut-out is performed in a form designated by an ellipse 2009. In this case, the portion protruding from the ellipse 2009 is deleted when the OK button 2004 is pressed. The user moves the ellipse 2009 to the part (the part centered on the face) where he / she wants to leave, adjusts the size, and deletes it with the OK button 2004. This makes it impossible to completely remove the background, but it is very easy to cut the extra part that appears in the background, and the effect is the same as when a paper photo is cut out with an ellipse and pasted into an album. be able to.

  The other one is a background erasing mode indicated by 2001 in FIG. 1B, and this mode is switched by selecting a radio button 2002. The user designates the contour of the subject and presses the erase button 2006 to erase the image outside the contour. In this case, the work amount of the user is larger than in the cut-out mode, but the background can be completely removed.

  FIG. 14 is an explanatory diagram showing a processing procedure in the cut-out mode. Reference numeral 2102 denotes an image display screen 2010 when clipping is selected with the radio button 2002 and a rectangle is selected with the radio button 2003. Reference numeral 2100 denotes a rectangle displayed on the screen by default. The rectangle 2100 is located in the center of the screen and has a length and width equal to half the length of the screen. Reference numeral 2101 denotes a point for changing the length and size of the rectangle. The user performs an operation by dragging the mouse down on the point 2101 and dragging it as it is. Reference numeral 2104 denotes the center of the rectangle 2100, which is used when the rectangle is moved without changing its size. Similarly, the user can move by moving the mouse down at the center 2104 of the rectangle 2100 and dragging. The user designates a desired position to be cut out as shown in an image display screen 2103 by performing an operation of changing the size.

  When an OK button 2004 is pressed, the image outside the rectangle is deleted, and a cutout image 2105 is obtained. FIG. 15 is a flowchart showing the processing procedure of the cut-out mode. In the figure, step S100 is an event loop. When an event such as a mouse button is pressed or a menu is selected or a button is pressed, the message is passed to the application software.

  If an event occurs, it is determined whether or not the message is that the mouse button has been pressed (step S101). If the message is that the mouse button has been pressed, whether or not the point is on the rectangular corner point 2101. Is checked (step S102). If the point is on the corner point 2101 of the rectangle, the fact that the mouse is down on the point 2101 of the rectangle corner is set as a variable (step S105). If it is not on the corner point 2101 of the rectangle in step S102, it is checked whether it is on the center 2104 of the rectangle (step S103). (Step S104).

  On the other hand, if the mouse button is not pressed in step S101, it is determined whether or not the mouse button has been released (step S106). When the mouse button is released, it is checked whether the rectangle is being resized with reference to the previous variable (step S107). If resizing is in progress, the size and position of a new rectangle are calculated from the position where the mouse button was released (step S108). Holding this, drawing is performed according to a new rectangle (step S109).

  Then, it is checked whether or not the OK button 2004 has been pressed (step S110), and when the OK button 2004 is pressed, an image area corresponding to this rectangle is secured in the memory (step S111). Image data located within the rectangle is copied to this area (step S112). The original image data is deleted from the memory (step S113), and the window 2000 in FIG. 13 is deleted. As a result, the user's operation returns to the document, and the image 2105 of FIG. 14 is pasted on the document.

  FIG. 16 is an explanatory diagram showing an operation processing procedure in the background erasing mode. The background erase mode is a state when the radio button 2002 is selected as background erase as shown in a screen 2001 in FIG. On the screen 2300, first, the user designates the start point 2304 on the outline of the person by clicking the mouse. Subsequently, when the next point 2306 is clicked with the mouse, the contour line between them is automatically traced to display the contour line 2305.

  Details of the background erasing mode processing will be described later. Further, when the next point 2307 is designated on the screen 2301, the previous outline 2305 is confirmed and changed from a gray line (shown by a broken line) to a black line. Trace is performed between the point 2306 and the next point 2307, and as a result, the outline 2308 is displayed. By repeating the above operation, a closed outline 2309 of the screen 2302 is created. The lower side of the outline 2309 is horizontal when the user double-clicks, this means the final point, and when the distance between the start point and the final point is more than a predetermined value, the interval is horizontal Tie like so. Although details will be described later, in the case of a portrait, it is desirable that the lower side is cut horizontally, and the processing for that purpose is automatically performed. Then, when the Erase button 2006 in FIG. 13 is pressed, the outside of the contour line is erased, and an image of only a subject with no background on the screen 2303 can be acquired.

  FIG. 17 is an explanatory diagram showing an operation method in the background erasing mode. In the figure, an outline 2400 is a determined part at this point, and an outline 2401 is a previous designated part and is still editable. Note that these contour lines are displayed on the contour of the background image, but the image is omitted here.

  The shape of the mouse pointer at the time of operation is either a cross 2403 or a double circle 2405. The shape of the mouse pointer is automatically changed according to the distance from the final point 2409. When the distance 2402 between the final point 2409 and the mouse pointer is less than or equal to a predetermined value, the shape of the mouse pointer becomes a cross 2403. When the distance 2404 is larger than a predetermined value, the shape of the mouse pointer becomes a double circle. When the mouse is clicked at the time of the double circle 2405, a contour line is extracted between the final point 2409 and the clicked point by edge tracking processing based on the edge information of the image. Details of this processing will be described later.

  On the other hand, in the case of the cross 2403, the final point and the clicked point are simply connected with a straight line. The reason for having two modes in this way is as follows. In the edge tracking process to be described later, only the contour desired by the user is not necessarily extracted, and an erroneous result may be obtained. For example, when a person with a strong edge is reflected in the background of the person, the edge may be traced as an outline. In this case, in order to provide a method that keeps up to the point where the person's outline is correctly extracted and skips the erroneous part with a short straight line, the outline extraction process is performed at a long distance, and the short distance is obtained. Then, it cuts with a straight line and skips the wrong part, and does not require the user to perform troublesome operations such as button switching. About 16 pixels is appropriate as the threshold for long and short distances.

  If the contour extraction result 2401 is not satisfactory for the user, it can be canceled by pressing the right mouse button or the Undo button 2007 in FIG. As a result, only the confirmed portion 2406 remains, and the user performs the operation again. In addition, if the result of the contour line extraction 2401 is held halfway and after that, if it is desired to cancel, the user clicks on the portion 2407 to be held with the mouse. Then, as shown by the contour line 2408, the part 2407 to be held is determined. The above operation is repeated to complete the outline of the human part. Further, if it is desired to cancel all of the results as a result during the process, pressing the Clear button 2008 in FIG. 13 deletes all the contour lines.

  A data structure for managing the contour data obtained by the above operation will be described. FIG. 18 is an explanatory diagram showing a data structure for managing contour data. A point sequence (array for storing x, y coordinates) point [] 2500 stores a determined point sequence (contour lines determined in black as 2400, 2406, and 2408 in FIG. 17), and the point sequence subpoint [] An array 2501 stores an undetermined point sequence (a contour line portion displayed as undetermined as indicated by 2401 in FIG. 17). Initially, the point sequence of the trace result is stored in subpoint [], and when confirmed by operation, it is moved to point []. When the Clear button 2008 in FIG. 13 is pressed, all the contents are cleared for both point [] and subpoint [].

  When the right button of the mouse is pressed or when the Undo button 2007 is pressed, the contents of the subpoint [] are cleared, and the last value of the point [] is stored at the tip of the subpoint []. In the drawing, the value of point [] is displayed in black, and the value of subpoint [] is displayed in gray, so that confirmation is made and unconfirmed is clearly indicated to the user.

  FIG. 19 is a flowchart showing a procedure for managing contour data. In the figure, step S130 is an event loop. When it is determined that an event has occurred and the mouse button has been clicked (step S131), first, it is checked whether the content of both the point sequence point [] and the point sequence subpoint [] is empty (step S132). Both are empty when the user first clicks. The vicinity of the clicked point (the range of ± 2 pixels in the x and y directions from the clicked point) is examined, and the point having the strongest edge that is equal to or greater than a predetermined value is newly set as the designated point (step S133). This is a process for automatically converting to the optimum position even if the user clicks on the outline roughly to some extent. If there is no edge strength greater than a predetermined value, the clicked point is used as the designated point. The designated point is placed at the beginning (first element) of subpoint [] (step S134).

  If the contents of both the point sequence point [] and the point sequence subpoint [] are not empty in step S132, it is checked whether there is only one value in the subpoint [] (step S135). Subpoint [] has only one value when the user cancels with the right button and clicks on the next point, or when the last halfway through the contour is confirmed and then the next This is when the point is clicked. At this time, tracing is performed with the first value of the subpoint [] as the start point and the clicked point as the end point, and the points in between are stored in the subpoint [] (step S140).

  If subpoint [] contains more than one value, the previous undetermined part (gray part) is OK, and when the next trace is performed, the middle part of the previous undetermined part Either confirm it or cancel the rest. It is checked whether any of the points in subpoint [] is present in the vicinity of the clicked point (range of ± 2 pixels in the x and y directions from the clicked point) (step S136). In some cases, it is determined that the process is confirmed halfway and the rest is cancelled. In the case where there is not, the previous unconfirmed part (gray part) is OK and the next trace is performed.

  The previous unconfirmed part (gray part) is OK, and when performing the next trace, the value of subpoint [] is added to the end of point [] to make the unconfirmed state (step S138). Then, the value of the subpoint [] is cleared, and the last value of the point [] is put into the first element of the subpoint [] (step S139). Thereafter, the process of step S140 is performed.

  On the other hand, when canceling the remaining part by confirming partway, the elements up to the value in the found subpoint are added to the end of the point [] (step S137). Then, the processes of steps S139 and S140 are performed as described above.

  Contour lines are extracted by repeating the above operation, and the process when the user double-clicks last will be described. FIG. 20 is a flowchart showing a processing procedure for horizontally cutting the bottom side. Even when the user presses the Erase button 2006 halfway through the contour line, the processing after step S171 in FIG. 20 is performed prior to the processing for erasing the background.

  When an event occurs (step S169) and it is determined that the generated event is a double click of the mouse button (step S170), the first element (starting point) and the last element (final point) of point [] It is checked whether or not the distance is greater than or equal to a predetermined value (step S171).

  When the distance is greater than or equal to a predetermined value, a process of automatically cutting the lower side horizontally is performed. As a result, it is possible to save the user from having to explicitly instruct. The point that is located at the top of the start point and the end point (the y coordinate is on the upper side) is acquired (step S172). If the starting point is above, a new point is created in which the x coordinate is the same as the starting point and the y coordinate is the y coordinate of the final point, and this is added as the first element of point [] (step S173). Then, by adding this point to the final element of point [], the lower side forms a horizontal and closed contour line.

  When the distance between the first element (starting point) and the last element (final point) of point [] is less than a predetermined value in step S171, the value of the starting point is added as the final point to point [] (step S174), Form a closed contour.

  FIG. 21 is a flowchart showing the processing procedure of step S140 of FIG. First, it is checked whether or not the distance between the starting point (the value of the first element of subpoint []) and the clicked point is a predetermined value or more (step S190). When the distance is equal to or less than a predetermined value (for example, 16 pixels or less), the clicked point is added to the point string subpoint [] (step S198), and the process ends. As a result, when the distance between the two points is short, a simple straight line is realized. When the distance is larger than the predetermined distance, the following processing is performed.

  The vicinity of the clicked point (x, y direction ± 2 pixels) is searched, and the point with the strongest edge value is set as the end point (step S191). This is the same processing as step S133 in FIG. First, the starting point is set as the attention point (step S192), and if the end point is not reached (step S193), the next attention point is obtained (step S194). The next point of interest is a point that approaches the end point direction by one pixel toward the end point direction, and details of processing for obtaining this will be described later. The obtained attention point is added to the point sequence subpoint [] (step S195). The case where the obtained attention point is not closer than the previous attention point is counted (step S196). When this count value exceeds a predetermined number of times (for example, several hundred times) (step S197), there is a possibility of tracing in the wrong direction, so the processing is stopped. On the other hand, when the attention point has reached the end point (step S193), the process ends.

  FIG. 22 is an explanatory diagram showing the processing operation of step S194 of FIG. FIG. 23 is a flowchart showing a processing procedure for obtaining the next attention point. In FIG. 22, 2901 is the current attention point, and 2900 is the end point. Coefficients a0 to a7 are assigned to the eight pixels around the current attention point 2901 from the closest distance in the end point direction (steps S200 and S201). a0 is the largest value and subsequently decreases sequentially. The coefficients a5 to a7 have the value 0. The pixel edge value corresponding to this coefficient is multiplied (step S202), and the pixel having the largest value not less than a predetermined value is set as the next point of interest (steps S203, S204, S205).

  As a result, it is possible to gradually approach the end point. When all the pixels are less than the predetermined value, the pixel closest to the end point (in FIG. 22, the pixel holding the coefficient of a0) is set as the next attention point (step S206), and this number is counted. The weak part of the edge will approach the end point for the time being.

  Although all the processes for extracting the contour line have been described above, the following method is more effective as this application. In FIG. 21, FIG. 22, and FIG. 23, the attention point is tracked from the start point to the end point. However, the contour line can be extracted from the end point to the start point by exactly the same method. This processing is the same as that described in FIGS. 21, 22, and 23, except that the direction from the start point toward the end point is the same as the direction from the end point toward the start point. In this case, when the next point of interest is determined in step S207 of FIG. 23, the number of times when the edge is weak and not more than a predetermined value is counted, so that the end point from the start point and the start point from the end point are counted. Compared with the case of aiming at, the one with the smaller number of faults is adopted as the contour line.

  FIG. 24 is an explanatory diagram showing a case where contour extraction is performed in both directions. In the figure, reference numeral 3100 denotes a case where an edge to be traced is interrupted and an image in which a stronger edge different from the middle branches.

  In the case of a natural image, there are actually more cases where there are discontinuities in this way than when edges are continuous. At this time, if tracing is performed from the start point to the end point as in 3101, the route is drawn to a stronger edge on the way, and reaches the end point when this is interrupted. However, the trace from the end point to the start point is correctly traced as indicated by 3102. The previous fault count is used as an effective measure to determine which of these is more appropriate. In 3101, there is a fault state from when it is pulled to a strong edge until it reaches the end point when it is interrupted. 3102 results will be selected.

  FIG. 25 is a flowchart showing a processing procedure for erasing the background. A rectangle circumscribing the obtained contour is obtained (step S220). The image data in the contour line is copied into this rectangle (step S221). Then, the pixel on the contour line is blurred by the coefficient shown in FIG. 26 (step S222). FIG. 26 is an explanatory diagram showing a filter for blurring processing. This produces a double effect of blurring the cut-out portion of the image so that it can be pasted on the document without a sense of incongruity, and a slight error in contour extraction. Since it is an image used on a document to the last and used as a portrait photograph of a part of the document, it is sufficient that the quality required for it is lower than that for design and high-definition image processing. The processing described above for this required quality can provide sufficient results and can be realized with a much simpler operation than the conventional clipping method.

  As described above, in the document editing system according to the present embodiment, sample images are prepared on a document sample having a certain purpose at a position and size suitable for it. When this image is newly replaced, a processing window that matches the purpose of the image on the document is activated, and the user can apply an effective effect with a simple operation. As a result, the user can create a high-quality document without having a sense such as a layout for a document including an image and knowledge of applying an effective effect to the image.

  Of the above-mentioned effective effects, the portrait effect has a mode of cutting out into a shape such as an ellipse and two modes of erasing the background along the outline, and a portrait image can be easily created. In the mode to erase the background along the contour, the function to easily cut the lower side horizontally, the function to skip the branch point with a straight line, and to specify the location where the contour is accidentally extracted A function that traces between points from both sides and automatically selects the best results can provide an easy-to-use operation method that places less burden on the user.

  The present invention may be applied to a system composed of a plurality of devices or an apparatus composed of a single device. Needless to say, the present invention can also be applied to a case where the present invention is achieved by supplying a program to a system or apparatus. In this case, the system or apparatus can enjoy the effects of the present invention by reading the storage medium storing the program represented by the software for achieving the present invention into the system or apparatus.

It is a block diagram which shows the structure of the document editing system in 1st Embodiment. It is explanatory drawing which shows the structure of document data. It is explanatory drawing which shows process script data. 3 is an explanatory diagram showing a memory map in a memory area used by the image editing application 102 in the main storage device 307. FIG. 10 is a flowchart illustrating a processing procedure when the image editing application is started by the document editing application 319. It is explanatory drawing which shows the display screen of the display apparatus 304 in the in-place active state and the state waiting for the event from a user. FIG. 6 is an explanatory diagram showing input of a command for transmitting image data from the image filing application 320 to the image editing application 102 through inter-application communication. It is a flowchart which shows the process sequence of a document edit system when the image data transmission command is input from the user by drag and drop operation. FIG. 11 is an explanatory diagram showing a memory map of an editing result image data holding area 1105 according to a processing procedure k. It is explanatory drawing which shows a candidate image display window. It is a flowchart which shows the process sequence of the document editing system in 2nd Embodiment. It is explanatory drawing which shows the process script data in 2nd Embodiment. It is explanatory drawing which shows the user interface of a portrait effect template. It is explanatory drawing which shows the process sequence of clipping mode. It is a flowchart which shows the process sequence of clipping mode. It is explanatory drawing which shows the operation | movement process procedure of background erasing mode. It is explanatory drawing which shows the operating method of background erase mode. It is explanatory drawing which shows the data structure which manages outline data. It is a flowchart which shows the procedure which manages outline data. It is a flowchart which shows the process sequence which cuts a base horizontally. It is a flowchart which shows the process sequence of step S140 of FIG. It is explanatory drawing which shows the processing operation of step S194 of FIG. It is a flowchart which shows the process sequence which calculates | requires the next attention point. It is explanatory drawing which shows the case where outline extraction is performed bidirectionally. It is a flowchart which shows the process sequence which erase | eliminates a background. It is explanatory drawing which shows the filter of a blurring process. It is a block diagram which shows the structure of the hardware of the conventional document editing system. It is explanatory drawing which shows the data structure of document data. FIG. 10 is an explanatory diagram showing a user interface screen of a conventional document editing application 319 displayed on a display device 304. 6 is an explanatory diagram showing a user interface screen of an image filing application 320 displayed on a display device 304. FIG. It is explanatory drawing which shows the user interface screen of the image editing application 321 which supplies image data to another application as an OLE server in a document editing system. 10 is an explanatory diagram showing a state in which an image editing application 321 is activated in an in-place active state within a document editing application 319. FIG. File2. It is explanatory drawing which shows the state by which the image of bmp was affixed.

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 Application software 102 Image editing application 301 Hardware 302 Operating system 319 Document editing application

Claims (4)

In an image editing apparatus for editing an image displayed on a display means,
Designation means for designating a point on the contour of the object in the image based on a user operation ;
Extraction means for extracting a contour line between the first point designated by the designation means and the second point designated following the first point based on edge information of the image;
Determining means for determining whether or not a distance between the first point and a second point designated following the first point is equal to or greater than a predetermined value;
When the distance between the first point and the second point designated following the first point is determined to be greater than or equal to a predetermined value by the determining means, the contour line is determined by the extracting means. extracting, when the distance between the second point specified following the first point and the first point is determined to not more than the predetermined value, the said first point first image editing apparatus and a control means shall be the contour of the straight line connecting between the second point specified following the first point. The image editing apparatus according to claim 1, wherein the designation unit designates a point based on a position of a mouse cursor displayed by the display unit. An image editing apparatus control method for editing an image displayed on a display means,
Designating a point on the contour of the object in the image based on a user operation ;
Extracting a contour line between the designated first point and the designated second point following the first point based on edge information of the image;
Determining whether a distance between the first point and a second point designated following the first point is a predetermined value or more;
If it is determined in the determining step that the distance between the first point and the second point specified following the first point is equal to or greater than a predetermined value, the extracting step When the contour line is extracted and it is determined that the distance between the first point and the second point designated following the first point is not greater than or equal to a predetermined value, the first point control method according to a straight line connecting between the second point specified subsequent to the point and the first point and a to that process and the contour line. 4. The control method according to claim 3, wherein in the step of specifying, a point is specified based on a position of a mouse cursor displayed by the display means.

Images