JP5472019B2 - Image composition apparatus, image composition program, and image composition method - Google Patents

Description

  The present invention relates to an image composition device and an image composition program, and more particularly to an image composition device and an image composition program that compose an image using an electronic pen.

  In recent years, electronic pens that output handwriting input with a pen as data have been used as input devices. This electronic pen is equipped with a small camera at the pen tip, and the position of the pen tip is specified by reading a position detection pattern called an Anoto (registered trademark) pattern printed on the paper with a small camera and handwritten. Handwriting such as letters, figures, and pictures is output as handwritten data.

  As a system using such an electronic pen, for example, in Patent Document 1 below, a digital pen with a built-in digital camera and a writing function, digital pen dedicated paper, and photo image data from the digital pen are received. A printing system including a communication device that transmits a received image to a server and receives edited data from the server, and at least a photographic image information reading unit, and an output device that is connected to the communication device and outputs a printed matter. It is disclosed.

JP 2006-163952 A

  By using the technique of the above-mentioned patent document 1, an image preliminarily arranged on a sheet and a handwritten image handwritten on the image can be synthesized to output a synthesized image, but the image is arranged In order to create a sheet, the user must select an image and specify where the selected image is to be placed on the sheet, which requires complicated operations.

  In particular, when creating a document etc. in which a large number of images such as images taken with a digital camera or images scanned with a scanner are arranged at predetermined positions on each page, it is troublesome to associate the images with the positions on the paper. There is a problem that the image cannot be easily combined.

  The present invention has been made in view of the above-described problems, and a main object thereof is to provide an image composition apparatus and an image composition program that can easily compose an image at a predetermined position on a sheet or a screen. It is in.

In order to achieve the above object, an image composition device of the present invention includes one or a plurality of handwritten images representing a movement locus of an electronic pen on a sheet or a screen, and at least one time information for specifying a time when the image is handwritten. a handwritten information is associated in advance time information associated with the image data taken by a digital camera, and / or an input unit for repeatedly inputting the image data read by the scanner, and corresponding to the handwritten image Based on at least one attached time information and time information associated with the image data, image data captured or stored at a time closest to the time when the handwritten image is handwritten on each handwritten image. An input image or an image based on the image data associated with the handwritten image within the closed region specified by the handwritten image and the control unit to be associated An image processing section for combining the location information of the image data, and an output unit for outputting the combined result, the one in which at least comprises.

In addition, the present invention is an image composition program that operates on an apparatus including an input unit and an output unit, and the apparatus includes one or more that represents a movement locus of an electronic pen on a sheet or a screen on the input unit. Handwritten image associated with handwritten information associated with at least one time information specifying the time when the image was handwritten , image data captured with a digital camera and associated with time information in advance , and / or scanner Each of the handwritten images based on the first process of repeatedly inputting the image data read in step 1, at least one time information associated with the handwritten image and the time information associated with the image data. second processing for associating the image data photographed or stored at the closest time to the time when the handwriting the handwriting image, closed specified by the handwritten image A third process for synthesizing the input image based on the image data associated with the handwritten image or the storage location information of the image data in the area, and a fourth process for causing the output unit to output a synthesis result are executed. It is something to be made.

  According to the image synthesizing apparatus and the image synthesizing program of the present invention, it is possible to easily synthesize an image at a predetermined position on a sheet or a screen.

  The reason is that the image synthesizing device (image synthesizing program) is one or a plurality of handwritten images handwritten on the screen on which the position detection pattern is printed or the screen on which the position detection pattern is displayed and the handwritten image from the electronic pen. Handwritten information associated with at least one piece of time information specifying the time at which the handwriting is handwritten is acquired, and image data associated with the time information is obtained in advance from a digital camera or scanner, and is associated with the handwritten image. The handwritten image and the image data are associated with each other based on the at least one piece of time information and the time information associated with the image data, and the image data associated with each image composition area based on the time information This is because control for combining the image based on the image or the storage location information of the image data is performed.

It is a figure which shows the structure and schematic operation | movement of an image composition system concerning one Example of this invention. It is a block diagram which shows the structure of the image composition apparatus which concerns on one Example of this invention. It is a block diagram which shows the structure of the electronic pen which concerns on one Example of this invention. It is a block diagram which shows the structure of the digital camera which concerns on one Example of this invention. It is a figure which shows an example of an Anoto (trademark) pattern. It is a flowchart figure which shows the whole process of the image composition system which concerns on one Example of this invention. It is a flowchart figure which shows the creation process of the handwritten input paper based on one Example of this invention. It is a figure explaining the creation process of handwritten input paper. It is a flowchart figure which shows the process of the image compositing instruction | indication concerning one Example of this invention. It is a figure explaining the process of an image compositing instruction | indication. It is a flowchart figure which shows the handwritten information acquisition process which concerns on one Example of this invention. It is a flowchart figure which shows the input image taking-in process based on one Example of this invention. It is a flowchart figure which shows the synthesized image selection process based on one Example of this invention. It is a figure explaining a composite image selection process. It is a flowchart figure which shows the image composition process which concerns on one Example of this invention. It is a figure explaining an image composition process.

  As shown in the background art, various systems using an electronic pen have been proposed. Patent Document 1 outputs a digital photo image on a paper dedicated to a digital pen and fills it with a digital pen. By doing so, a printing system for printing a photographic image with a handwritten message is disclosed. However, this system synthesizes a handwritten image handwritten with an image previously arranged on a sheet. When an image is arranged on a sheet, the user determines which image is arranged at which position on the sheet. However, there is a problem that a complicated operation is required.

  Therefore, in one embodiment of the present invention, an electronic pen is used for processing to place an image at a predetermined position on a sheet or screen. Specifically, a position detection pattern that is recognizable with an electronic pen is output on paper or a screen, and handwritten information handwritten on the paper or screen with an electronic pen, and image data or a scanner photographed with a digital camera are read. Get image data. The handwritten image and the image data are associated with each other based on the handwritten time information and the time information previously associated with the image data.

  For example, the user takes a picture using a digital camera or reads a document using a scanner, and the electronic pen is used to display a position detection pattern on a sheet of paper on which a position detection pattern is printed. One or more images are handwritten on the screen, and the image composition device (image composition program) specifies image data captured by a digital camera or read by a scanner at a time close to the time when the image was handwritten with an electronic pen. Then, a composite image is generated in which an image based on image data or storage location information of the image data is arranged in an image composition area specified by the handwritten image. Thereby, the operation of the electronic pen can be associated with the operation of the digital camera or the scanner, and a composite image in which a desired image is arranged at each position designated by the electronic pen can be easily created.

  Hereinafter, a case where an image based on image data acquired by a digital camera is arranged on a sheet using a sheet on which a position detection pattern is printed will be described in detail.

  In order to describe the above-described embodiment of the present invention in further detail, an image composition apparatus and an image composition program according to an embodiment of the present invention will be described with reference to FIGS. FIG. 1 is a diagram showing the configuration and schematic operation of the image composition system of this embodiment, and FIG. 2 is a block diagram showing the configuration of the image composition device. 3 is a block diagram illustrating the configuration of the electronic pen, FIG. 4 is a block diagram illustrating the configuration of the digital camera, and FIG. 5 is a diagram illustrating an example of an Anoto (registered trademark) pattern. 6, 7, 9, 11 to 13, and 15 are flowcharts illustrating processing of the image composition system, and FIGS. 8, 10, 14, and 16 are diagrams illustrating processing of the image composition system.

  As shown in FIG. 1, the image composition system of the present embodiment includes an image composition device 10 that performs image composition processing, an electronic pen 20 that detects a position detection pattern represented by an Anoto (registered trademark) pattern, It is composed of a digital camera 30 that provides an image to be synthesized (referred to as an input image).

  The schematic operation of the image composition system will be described. First, the image composition apparatus 10 creates a paper (referred to as handwritten input paper) on which a position detection pattern is printed. In addition to writing characters and diagrams on the created handwriting input sheet with the electronic pen 20, a predetermined operation (an operation for drawing an image composition area) for instructing which position on which page an input image is to be input is also performed. Do. Information (referred to as handwritten information) stored in the electronic pen 20 that has performed such handwriting includes time information handwritten along with a handwritten image indicating the movement locus of the electronic pen 20.

  On the other hand, the input image to be synthesized is taken by the digital camera 30 together with the handwriting work, and stored in a memory card, the image synthesis device 10, a server, or the like as image data in a predetermined format. At that time, time information such as the photographing time and the storage time of the image data is written in the header of the image data.

  Thereafter, the image composition device 10 captures handwritten information from the electronic pen 20 and also captures image data (or obtains storage location information of the image data) from the digital camera 30 or a server, and analyzes the handwritten image. When the handwritten image represents the image composition area, the image composition area and the image data are associated with each other by comparing the handwritten time information with the time information associated with the image data. The image based on the image data or the storage location information of the image data is synthesized, and the synthesized image is printed or output as a file.

  In the present embodiment, the image composition apparatus 10 is an apparatus having a printing function for creating handwritten input paper (for example, an image forming apparatus such as a copier or a multi function peripheral (MFP)). When the position detection pattern is displayed on the screen, a device having a display function (for example, electronic paper, electronic book, portable terminal, computer device, etc.) can be used. Hereinafter, the configuration of each apparatus will be described with reference to the drawings.

[Image composition device]
As illustrated in FIG. 2, the image composition device 10 includes a controller unit and various devices controlled by the controller unit. The controller unit includes a control unit 11, a communication unit 12, a display / operation unit 13, an image. It comprises an input unit 14, an image output unit 15, a data storage unit 16, an image processing unit 17, and the like.

  The control unit 11 includes a CPU (Central Processing Unit) and a memory such as a ROM (Read Only Memory) and a RAM (Random Access Memory). The CPU expands a control program stored in the ROM to the RAM. And execute overall control of the image composition apparatus 10. In the present embodiment, in particular, a process for reading a position detection pattern from a memory or an external computer device, controlling the image output unit 15 to cause the image output device 19 to print handwritten input paper, or handwriting information from the electronic pen 20. Image data is acquired from the digital camera 30, the own device, another device (a server connected to the image composition device 10 via a network), or the like (or the storage location information of the image data is acquired), and the handwritten image is an image. When representing a composite area, the handwritten time information and the time information associated with the image data are compared to associate the image composition area with the image data, and the image processing unit 17 is controlled to control each image composition. An image based on the image data or storage location information (link destination address, etc.) of the image data is synthesized in the area, and the image output unit 15 is controlled to Or to print the synthesized image to the output device 19, via the communication unit 12 performs processing and outputs the synthesized image data.

  The communication unit 12 controls various protocols, inputs an image data from the digital camera 30 or a server, inputs handwritten information from the electronic pen 20, and downloads a position detection pattern from a computer device. It functions as an output unit that transmits the composite image data to an external computer device.

  The display / operation unit 13 is a touch panel in which transparent electrodes are arranged in a grid on an LCD (Liquid Crystal Display) or an organic EL (electroluminescence) display, and receives information display to the user and information input from the user.

  The image input unit 14 is a part that controls image input from an image input device 18 (for example, a scanner). The image input unit 14 includes an image to be combined with a position detection pattern when a handwritten input sheet is created, Control uptake. In addition, when using only the image acquired from the digital camera 30, the own apparatus, a server, etc., the image input part 14 and the image input device 18 can also be abbreviate | omitted.

  The image output unit 15 is a part that controls image output to an image output device 19 (for example, a printer). The image output unit 15 prints a position detection pattern, and combines an image or a storage location information of the image in an image combining area. Control printing of images.

  The data storage unit 16 is an HDD (Hard Disk Drive) or the like, and stores image data, handwritten information, and other system information.

  The image processing unit 17 performs processing such as composition of a position detection pattern on a handwriting input sheet, composition of an image on an image composition area, and scaling, rotation, and trimming of an image to be synthesized on the image composition area.

  In FIG. 2, the image synthesizing process is performed by the image processing unit 17, but handwritten information and image data are input, and the handwritten image and the time information of the image data are input based on the time information of the handwritten information and the time information of the image data. A series of processes for associating with image data and synthesizing and outputting an image based on the image data in an image synthesis area specified by a handwritten image may be executed by an image synthesis program operating in the control unit 11.

[Electronic pen]
As shown in FIG. 3, the electronic pen 20 includes a control unit 21, a communication unit 22, a battery 23, an operation unit 24, an imaging unit 25, a pressure detection unit 26, a writing unit 27, a clock unit 28, a data storage unit 29, and the like. Composed.

  The control unit 21 includes a CPU and a memory such as a ROM or a RAM. The CPU expands and executes a control program stored in the ROM on the RAM and performs overall control of the electronic pen 20.

  The communication unit 22 controls transmission of handwritten information to the image composition device 10 by short-range wireless communication represented by Bluetooth (registered trademark), infrared communication, or wired communication.

  The battery 23 is a primary battery or a secondary battery, and supplies power to each part of the electronic pen 20.

  The operation unit 24 is a button or the like provided on the main body of the electronic pen 20, and performs an instruction to start / end handwriting, an instruction to transmit handwritten information to the image composition device 10, and the like.

  The imaging unit 25 is a small CCD or CMOS (Complementary Metal Oxide Semiconductor) video camera for reading a position detection pattern.

  The pressure detection unit 26 is a piezoelectric element or the like for detecting the writing pressure of the electronic pen 20.

  The writing unit 27 specifies the position of the electronic pen 20 on the handwriting input sheet based on the position detection pattern read by the imaging unit 25 when the pressure detection unit 26 detects the writing pressure, and the movement locus of the electronic pen 20. The handwritten image specified by is generated. Note that the handwritten image in this specification means an image that can be recognized as one object, not an image that is specified by the entire locus of movement of the electronic pen 20 on the handwriting input sheet. For example, a character or figure drawn with a single stroke without the electronic pen 20 leaving the handwriting input sheet can be recognized as one object, and constitutes a unit of a handwritten image. Further, one unit of the handwritten image need not be limited to an image obtained by one-stroke writing. For example, the electronic pen 20 moves away from the handwriting input sheet, but a character or a figure drawn within a predetermined time may be treated as one handwritten image. good. Furthermore, although the electronic pen 20 is separated from the handwriting input sheet, characters and figures whose individual trajectories are gathered within a predetermined distance range may be handled as one handwritten image. Accordingly, a figure such as a rectangle may be drawn with a single stroke, drawn with a stroke order such as a “mouth” of a Chinese character, or handled as one handwritten image.

  The clock unit 28 specifies the time when the writing unit 27 generates the handwritten image.

  The data storage unit 29 is a flash memory or the like, and stores the handwritten image generated by the writing unit 27 and the time information specified by the clock unit 28 in association with each other. At this time, it is sufficient that at least one time information can be associated with each handwritten image. For example, when the imaging unit 25 acquires time information every time the position detection pattern is read, the time information may be associated with each point of the handwritten image, or the start point or the end point of the movement locus of the electronic pen 20 may be used. When acquiring time information at a specific point, time information corresponding to the number of points may be associated with each handwritten image.

  The electronic pen 20 recognizes a position detection pattern printed on paper or displayed on a screen, specifies a position on the paper or the screen, and outputs handwritten information in association with handwritten images and time information. The configuration, shape, communication method, and the like are not particularly limited.

[Digital camera]
As shown in FIG. 4, the digital camera 30 includes a control unit 31, a communication unit 32, a battery 33, a display / operation unit 34, an imaging unit 35, a clock unit 36, a data storage unit 37, and the like.

  The control unit 31 includes a CPU and a memory such as a ROM or a RAM. The CPU expands and executes a control program stored in the ROM on the RAM and performs overall control of the digital camera 30.

  The communication unit 32 controls transmission of image data to the image composition device 10 by short-range wireless communication represented by Bluetooth (registered trademark), infrared communication, or wired communication.

  The battery 33 is a primary battery or a secondary battery, and supplies power to each part of the digital camera 30.

  The display / operation unit 34 is a touch panel in which transparent electrodes are arranged in a grid pattern on an LCD or an organic EL display, a button provided on the digital camera 30 body, and the like. A storage instruction, an instruction to transmit image data to the image composition apparatus 10, and the like are given.

  The imaging unit 35 is a CCD or CMOS video camera for acquiring image data to be combined.

  The clock unit 36 specifies the time when the image was captured and the time when the image data was saved.

  The data storage unit 37 is a flash memory or the like, and adds the time information specified by the clock unit 36 to the image data acquired by the imaging unit 35 (for example, describes the time information in the header of the image data) and stores it.

  The digital camera 30 only needs to have a function of adding time information to image data and outputting the image data, and the configuration, shape, recording format of the image data, and the like are not particularly limited. In this embodiment, the image data is stored in the digital camera 30, but the image data acquired by the digital camera 30 may be stored in the image composition device 10 or a server. In this embodiment, the image data is acquired from the digital camera 30. However, the image data may be acquired from a mobile phone equipped with a digital camera function, or trimmed with respect to pre-stored image data. Alternatively, image data to which image processing time information is added may be acquired from a computer device that performs image processing such as size conversion and color conversion. Further, when only the image data acquired by the image input device 18 (scanner) of the image composition device 10 is used, the digital camera 30 is unnecessary.

  FIG. 5 is an example of an Anoto (registered trademark) pattern printed on handwritten input paper or displayed on the screen. This Anoto (registered trademark) pattern is a dot pattern arranged in a grid pattern, and each dot is slightly shifted from the grid perpendicular to each other at intervals of 0.3 mm in either the top, bottom, left, or right direction. The data is read in a matrix unit of 6 dots vertically and 6 dots horizontally, for a total of 36 dots. This Anoto (registered trademark) pattern is a unique value at any point, and the trajectory written with the electronic pen 20 can be recognized as different position information.

  Hereinafter, the operation of the image composition system of the present embodiment will be described with reference to the flowcharts of FIGS. 6, 7, 9, 11 to 13, and 15 and the explanatory diagrams of FIGS.

  The schematic operation of the image composition system of this embodiment is as shown in FIG. First, the image synthesizing apparatus 10 creates a handwriting input sheet on which a position detection pattern is printed (S100), and the user performs handwriting on the handwriting input sheet using the electronic pen 20 and inputs an image composition instruction (S200). . Next, the image composition device 10 captures handwritten information from the electronic pen 20 (S300), and captures image data (or specifies storage location information of the image data) from the digital camera 30, its own device, a server, or the like. (S400). Then, the image composition device 10 analyzes the handwritten information to recognize the image composition instruction, selects image data to be composed in the image composition area based on the time information (S500), and selects the image data selected in the image composition area. Based on the input image or the storage location information of the image data (S600), a synthesized image is output (S700). Hereinafter, each step will be described individually.

[Create handwritten input paper in S100]
As shown in FIG. 7, the user operates the display / operation unit 13 of the image composition device 10 to instruct the creation of handwritten input paper (S101), and selects the original image to be synthesized on the handwritten input paper (S102). . Here, it is possible to select whether or not to synthesize a document image, and whether or not to synthesize a document image, whether a file name is designated and input from a computer device, or an image is captured by an image input device 18 (scanner). Shall.

  Next, the controller unit (control unit 11) of the image synthesizing apparatus 10 reads the document image based on the selection result of S102, and then selects the page from the position detection patterns stored in advance in the data storage unit 16. The position detection pattern to be printed is acquired (S103), and the image processing unit 17 synthesizes the position detection pattern with the document image (S104). Then, the image output unit 15 sends the composite image to the image output device 19 (printer), and the image output device 19 prints the composite image on handwritten input paper (S105). As will be described later, the handwritten input sheet is provided with a command button for instructing synthesis of an input image. Thereafter, the control unit 11 determines whether there is a next page (S106). If there is a next page, the control unit 11 returns to S103 and repeats the same processing.

  FIG. 8 is a diagram schematically showing the handwriting input paper creation process. The data storage unit 16 stores position detection patterns for a plurality of pages, and different position detection patterns are printed on each page. On the handwritten input paper, if no original image is selected in S102, only the position detection pattern is printed. If a file is specified in S102, a composite image obtained by combining the print image with the position detection pattern is printed. If printing is selected and image capture is selected in step S102, a combined image obtained by combining the position detection pattern with the scanned image is printed.

  In this embodiment, the handwriting input sheet is printed. However, when handwriting is performed on the screen, an image of only the position detection pattern, an image obtained by combining the position detection pattern with the print image, and a scan image are used. An image obtained by combining the position detection patterns may be displayed on the display unit.

[Image composition instruction in S200]
First, the user captures a photograph, a moving image, or the like to be combined with the digital camera 30, and stores image data associated with shooting or storage time information in the data storage unit 37, the image combining apparatus 10, a server, or the like. In parallel with shooting, an image composition instruction is issued using handwritten input paper. Specifically, as shown in FIG. 9, the command button of the page in which the image is to be captured is touched (S201), and the electronic pen 20 is drawn so that a closed region is created, thereby instructing image synthesis (S202). ). This process is repeated for the number of images to be combined. At this time, in the data storage unit 29 of the handwriting pen 20, a series of position information (handwritten image) specified by the position detection pattern recognized by the imaging unit 25 and time information at which handwriting is performed are stored in association with each other. The

  The image composition instruction is performed by inputting the location and size of the image to be captured with the electronic pen 20. For example, three instruction methods shown in FIG. 10 are conceivable.

  The first instruction method is the method shown in the flowchart of FIG. 9. After touching the command button portion printed in advance on the handwritten input paper with the electronic pen 20, the outline of the closed region (here, a circle) is displayed. It is a method of drawing. In this case, when a command button input and subsequent contour drawing of a closed region are detected, it is recognized as an image synthesis instruction. It is assumed that image composition is instructed by this method in capturing handwritten information shown below.

  The second instruction method is a method of drawing an outline (here, a rectangle) of a closed region and drawing a specific mark (here, a check mark) therein. In this case, it is recognized that the outline of the closed region has been drawn, and when a specific mark is subsequently input therein, it is recognized as an image composition instruction.

  The third instruction method is a method of drawing a specific character (here, the character “photo”), and then drawing a closed region outline (here, a rectangle) so as to surround the character. In this case, when a specific character is recognized and a contour of a closed region is subsequently input, it is recognized as an image synthesis instruction.

  The method for instructing image composition is not limited to the above. For example, in the first instruction method, the command button may be touched after drawing the outline of the closed area. In the second instruction method, the closed area is surrounded so as to surround the mark after drawing a specific mark. The contour may be drawn, or in the third instruction method, after the contour of the closed region is drawn, a specific character may be drawn therein. Further, although a circle is drawn in the first instruction method and a rectangle is drawn in the second and third instruction methods, any figure may be used as long as it represents a contour of the closed region.

[Capturing handwritten information in S300]
As shown in FIG. 11, first, the controller (control unit 11) of the image composition device 10 communicates with the electronic pen 20 through wireless communication, infrared communication, wired communication, or the like (S301), and is stored in the electronic pen 20. Handwritten information (handwritten image and time information) is acquired (S302), and the page is recognized from the position information of the acquired handwritten image (S303).

  Next, the control unit 11 detects an image composition instruction command by checking whether the position information of the handwritten image is a predefined command button region (S304). When the image composition instruction command is not detected, the control unit 11 stores the handwritten image in the data storage unit 16 as normal handwritten information (S309).

  On the other hand, when the image composition instruction command is detected, the control unit 11 reads time information associated with the handwritten image (S305), and detects a closed region indicated by the handwritten image (S306). Then, the control unit 11 calculates the position of the circumscribed rectangle of the closed region (S307), and lists image composition instruction information describing page information, the size and position (coordinates) of the rectangle, and time information. The data is stored in the data storage unit 16 (S308).

  Next, the control unit 11 determines whether there is next handwritten information (S310), and if there is next handwritten information, returns to S302 and repeats the same processing.

[Import image data in S400]
As shown in FIG. 12, the controller (control unit 11) of the image composition device 10 communicates with a digital camera 30 or a server in which image data is stored by wireless communication, infrared communication, wired communication, or the like (S401). The display / operation unit 13 displays the data storage unit 37 of the digital camera 30, its own device, a folder such as a server, and the like. Next, when the user operates the display / operation unit 13 to select a folder in which image data is stored (S402), the control unit 11 acquires image data from the selected folder (S403), and the data The data is stored in the storage unit 16 (S404). Information such as shooting time, storage time, file name (file number), and image size is added to the stored image data.

  Here, the folder of the digital camera 30 is displayed on the display / operation unit 13 and the user is allowed to select a desired folder, but the display and selection of the folder is omitted, and the digital camera 30 is omitted. All image data stored in the file may be captured. In this example, the image data is captured and stored in the data storage unit 16. However, when the storage location information of the image data is combined in an image combining step described later, it is not always necessary to acquire the image data. What is necessary is just to acquire the directory information of the folder where the data is stored and store it in the data storage unit 16.

[Select input image in S500]
As shown in FIG. 13, the controller (control unit 11) of the image composition device 10 reads a list of image composition instruction information from the data storage unit 16 (S501), selects one image composition instruction information from the list. Then, the time information described in the instruction information is acquired (S502). Next, the control unit 11 acquires time information associated with the image data stored in the data storage unit 16 (or stored in the data storage unit 37 or the server of the digital camera 30) (S503). Are compared with each other, and the image data at the time closest to or earlier than the time described in the image composition instruction information is selected (S504).

  Then, the control unit 11 displays the file name and thumbnail image of the selected image data on the display / operation unit 13 and confirms with the user, and then adds the file name of the selected image data to the list of image composition instruction information. And save (S506). Thereafter, the control unit 11 determines whether there is next image composition instruction information (S507). If there is next image composition instruction information, the control unit 11 returns to S502 and repeats the same processing.

  Here, the file name and thumbnail image of the selected image data are displayed on the display / operation unit 13 and confirmed by the user, but this step may be omitted and immediately added to the list. Also, here, the image data captured / saved at the time closest to and before the time described in the image compositing instruction information is selected, but details of the selection criteria are separately provided by the user. Can also be specified. For example, when selecting image data corresponding to the Nth image composition instruction in the list, a plurality of images captured / saved between the time of the (N-1) th image composition instruction and the time of the Nth image composition instruction It is also possible to use a method of displaying a list of image data and selecting by the user, a method of selecting image data captured / saved at the time closest to or later than the time of the image compositing instruction, etc. is there.

  FIG. 14 is a diagram schematically showing the input image selection process. Image data captured from the digital camera 30 or the like (or stored in the digital camera 30, the image composition device 10, the server, or the like) is associated with time information such as a shooting time or a storage time. On the other hand, handwritten information taken from the electronic pen 20 is also associated with handwritten time information, and page information, a rectangular size and position, and time information are described in the image composition instruction information. Based on these time information, the handwritten rectangle is associated with the input image to be combined with the rectangle.

  For example, the time when the rectangle No.1 was handwritten is 12:12, and the image data with the file name # 1 is taken / saved at 12:12, which is the same time as the handwritten time The image data # 1 is selected as the image data to be combined with the No. 1 rectangle, and the file name # 1 is added to the image combination instruction information list. The time when the rectangle No.2 was handwritten was 13:01, the image data of the file name # 2 was shot / saved at 13:00, the closest time before the handwritten time Therefore, the image data # 2 is selected as the image data to be combined with the No. 2 rectangle, and the file name # 2 is added to the image combination instruction information list. Also, for the No. 3 and No. 4 rectangles, # 3 and # 5 image data are selected according to the same selection criteria, and the file names # 3 and # 5 are added to the list of image composition instruction information. .

[Image composition in S600]
As shown in FIG. 15, the controller (image processing unit 17) of the image synthesizing apparatus 10 stores, from the data storage unit 16, a list of handwritten images and image synthesis instruction information and image data specified by the list (or storage of image data). (Location information) is read (S601 to S603). Next, the image processing unit 17 compares the size of the rectangle defined by the image composition instruction information with the size of the image based on the image data to be synthesized with the rectangle, and performs centering so that the image fits in the rectangle appropriately. Then, a value such as zooming is determined (S604). Then, the image processing unit 17 performs a scaling process and a centering process on each image based on the determined value (S605, S606), and then performs an image composition process (S607), and composite image data Is stored in the data storage unit 16 (S608).

  FIG. 16 is a diagram schematically illustrating the image composition process. For example, if the size of the rectangle differs from the input image to be combined, the image is enlarged / reduced so that the image fits in the rectangle. If the aspect ratio of the input image to be combined and the rectangle is different, the image is approximately at the center of the rectangle. Adjust the centering so that it is positioned. Then, a page image is generated by combining the input image (or input image storage location information) and the handwritten image with the document image. Although the input image and the handwritten image are combined with the original image here, only the input image may be combined with the original image, or only the input image and the handwritten image may be combined.

[Composite image output in S700]
The controller (control unit 11) of the image composition device 10 sends the composite image data generated by the image processing unit 17 to the image output unit 15, and the image output unit 15 controls the image output device 19 to input the input image into the document image. (Or the input image storage location information) and a combined image obtained by combining the handwritten image is printed. The composite image data may be output to the user's computer device and the composite image may be displayed on the display unit of the computer device. When the storage location information of the input image is combined, click the storage location information. Thus, the input image may be displayed by taking in the image data from the storage location. Alternatively, the composite image data may be output by saving the composite image data to a server, a USB (Universal Serial Bus) memory, or the like, or the composite image data may be transmitted as an email attachment.

  As described above, in the image composition device 10 according to the present embodiment, when image data is acquired from the digital camera 30 or the like and handwriting information is acquired from the electronic pen 20, it is determined whether the image composition region is handwritten, When the image composition area is handwritten, the handwritten time is compared with the image data capture / save time, and the image data to be combined with the image composition area is selected according to a predetermined rule, and the image composition area is selected. Since control is performed to output an image based on the image data or composite image data obtained by combining the storage location information of the image data, the image can be easily combined at a predetermined position on the paper or the screen.

  The present invention is not limited to the above-described embodiments, and the configuration and control of each device can be changed as appropriate without departing from the spirit of the present invention.

  The present invention can be used for an image composition device that composes an input image at a predetermined position on a sheet or a screen and a system including the image composition device.

DESCRIPTION OF SYMBOLS 10 Image composition apparatus 11 Control part 12 Communication part 13 Display / operation part 14 Image input part 15 Image output part 16 Data storage part 17 Image processing part 18 Image input device 19 Image output device 20 Electronic pen 21 Control part 22 Communication part 23 Battery 24 Operation Unit 25 Imaging Unit 26 Pressure Detection Unit 27 Writing Unit 28 Clock Unit 29 Data Storage Unit 30 Digital Camera 31 Control Unit 32 Communication Unit 33 Battery 34 Display / Operation Unit 35 Imaging Unit 36 Clock Unit 37 Data Storage Unit

Claims (11)

Time information is associated in advance with handwritten information in which one or a plurality of handwritten images representing the movement trajectory of the electronic pen on the paper or the screen and at least one time information specifying the time when the image is handwritten are associated with each other. was, the image data taken by a digital camera, and / or an input unit for repeatedly inputting the image data read by the scanner, and
Based on at least one time information associated with the handwritten image and time information associated with the image data, each handwritten image is captured or stored at a time closest to the time when the handwritten image is handwritten. A control unit for associating the processed image data ,
An image processing unit that synthesizes an input image based on image data associated with the handwritten image or storage location information of the image data in a closed region specified by the handwritten image;
An output unit that outputs a synthesis result,
An image composition device characterized by the above. The control unit determines whether the handwritten image is a predetermined specific image, and associates the handwritten image with the image data when the handwritten image is the specific image.
The image synthesizing device according to claim 1. The specific image is an image in which a contour of a closed region is drawn after pointing to a predetermined command region on paper or a screen, and an image in which a predetermined character or symbol is drawn in the closed region after the contour of the closed region is drawn , After drawing a predetermined character or symbol, any one of the images depicting the outline of the closed region surrounding the predetermined character or symbol,
The image synthesizing apparatus according to claim 2. The image processing unit compares the size of the input image with the size of the closed region or a rectangle circumscribing the closed region, and adjusts the arrangement and / or magnification of the input image based on the comparison result .
The image synthesizing apparatus according to claim 1, wherein the image synthesizing apparatus is an image synthesizing apparatus. The output unit prints an original image and a pattern that allows the electronic pen to recognize a position on the paper on a handwritten input paper, and outputs the input image or the input image within the closed region of the original image on the output paper. Printing an image obtained by combining the storage location information or an image obtained by combining a handwritten image other than the specific image ;
The image composition device according to claim 1, wherein An image synthesis program that operates on an apparatus including an input unit and an output unit,
In the device,
Handwritten information in which one or a plurality of handwritten images representing the movement locus of the electronic pen on the paper or the screen and at least one time information specifying the time when the image is handwritten are associated with the input unit; First processing for repeatedly inputting image data captured by a digital camera and / or image data read by a scanner, associated with time information;
Based on at least one time information associated with the handwritten image and time information associated with the image data, each handwritten image is captured or stored at a time closest to the time when the handwritten image is handwritten. A second process for associating the processed image data,
A third process for synthesizing the input image based on the image data associated with the handwritten image or the storage location information of the image data in the closed region specified by the handwritten image;
Causing the output unit to execute a fourth process of outputting a synthesis result ;
An image composition program characterized by that . In the second process, it is determined whether the handwritten image is a predetermined specific image, and when the handwritten image is the specific image, the handwritten image is associated with the image data .
An image composition program according to claim 6 . The specific image is an image in which a contour of a closed region is drawn after pointing to a predetermined command region on paper or a screen, and an image in which a predetermined character or symbol is drawn in the closed region after the contour of the closed region is drawn , After drawing a predetermined character or symbol, any one of the images depicting the outline of the closed region surrounding the predetermined character or symbol ,
The image composition program according to claim 7. In the third process, the size of the input image is compared with the size of the closed region or a rectangle circumscribing the closed region, and the arrangement and / or magnification of the input image is adjusted based on the comparison result .
The image composition program according to any one of claims 6 to 8 . An image composition method in an apparatus including an input unit and an output unit,
  Handwritten information in which one or a plurality of handwritten images representing the movement locus of the electronic pen on the paper or the screen and at least one time information specifying the time when the image is handwritten are associated with the input unit; A first process of repeatedly inputting image data captured by a digital camera and / or image data read by a scanner, associated with time information;
  Based on at least one time information associated with the handwritten image and time information associated with the image data, each handwritten image is captured or stored at a time closest to the time when the handwritten image is handwritten. A second process for associating the processed image data;
  A third process for synthesizing an input image based on image data associated with the handwritten image or storage location information of the image data in a closed region specified by the handwritten image;
  And a fourth process for causing the output unit to output a synthesis result.
  An image composition method characterized by the above. In the second process, it is determined whether the handwritten image is a predetermined specific image, and when the handwritten image is the specific image, the handwritten image is associated with the image data.
  The image synthesizing method according to claim 10.

Images