JP5381589B2 - Image processing apparatus and method, and program - Google Patents

Description

  The present invention relates to an image display apparatus and method, and a program, and more particularly, to an image display apparatus and method, and a program that can easily transmit a still image or a moving image captured in real time as an attachment to an e-mail.

  A technique for transmitting an e-mail with an image message composed of a still image or a moving image is generally spread.

  Conventionally, when a still image or a moving image is attached and sent as an e-mail image message, the user first operates the keyboard or operation buttons of an electronic device such as a personal computer to send the destination, subject, and text. To create an email. Next, the user operates the keyboard and operation buttons to select a desired image for attachment transmission. Then, the user operates the keyboard and operation buttons to execute an operation of attaching the selected image message to the generated electronic mail, and then executes an operation of transmitting the electronic mail.

  At this time, in selecting an image, a method is proposed in which a plurality of still images captured in advance are displayed as a list of thumbnail images, and the selected image is displayed by selecting one of the desired thumbnail images. (See Patent Document 1).

  After selecting an image message to be attached by such a method, the user performs an operation of attaching the selected image message to the e-mail, and then transmits the e-mail.

  In addition, by touching the surface of the display unit in association with display information that is a processing result of the displayed electronic device, a so-called touch panel that integrates the display unit and the operation input unit of the electronic device. There is something that can input various commands. The above-described list of thumbnail images can be displayed on the touch panel, and an image can be selected by touching a portion where a desired thumbnail image is displayed. As such software using a touch panel, for example, what is called Touch Pack (trademark) has been proposed (Non-Patent Document 1).

JP 2008-146453 A

http://japanese.engadget.com/2009/05/28/windows-7-and-microsoft-touch-pack/

  By the way, many recent electronic devices have a camera function for imaging. For this reason, there are increasing opportunities for a user to directly capture an image with an electronic device or the like and send the image as an email message.

  However, when an electronic device is used in the above-described manner, an image that has been captured in advance can be selected, but an image message is generated after being captured in real time, and attached to an e-mail. To do so, the operation was complicated and troublesome.

  In particular, when an electronic device employing a touch panel having both a display function and an input operation function is used, the operation becomes even more troublesome. That is, to capture an image in real time with an electronic device, edit the email destination address, subject, and text, attach the captured image as an image message, and send an email with a software keyboard, etc. The operation was cumbersome and troublesome because it had to be used.

  In addition, in order to perform processing such as generating an image message using an image captured in real time, a more complicated operation may be required, and the image is edited while viewing the image captured in real time. , I could not enjoy viewing the edited results.

  The present invention has been made in view of such a situation, and in particular, when an electronic device having an imaging function and a touch panel is used, a still image or a moving image is captured in real time, further edited, and emailed. This makes it easy to create an image message to be attached. Further, the present invention is to edit an image captured in real time and display the generated image message immediately so that it can be enjoyed. Furthermore, when using an electronic device having the same function, the present invention enables editing and transmission of an e-mail destination address, subject, and body text to which a created image message is attached with an easy operation. It is.

An image processing apparatus according to an aspect of the present invention is an image processing apparatus having a display unit and an imaging unit, an operation unit that generates an operation signal by a user's contact with the display unit, and an image on the display unit. By operating a bulletin board display control means for displaying a bulletin board for posting a message and displaying a pre-generated image message on the bulletin board, and an imaging mode transition operator displayed in the bulletin board, the imaging unit The captured image display control means for displaying the image picked up on the bulletin board and the operation signal for instructing the start and the end of the recording of the image picked up by the image pickup unit are generated by the operation means. When, among the image captured by the imaging unit, from the start of the recording is instructed timing, consists moving from the timing the end of the recording is instructed An image message generating means for generating an image message, to the image message, and displayed superimposed on the play button, on the basis of the operation signal, when said that the playback button has been operated is recognized, the image Moving image reproduction means for reproducing a moving image corresponding to the message , wherein the bulletin board display control means adds the image message newly generated by the image message generation means to the previously generated image message, and When the image message is dragged at two points based on the operation signal and the distance of the dragged position is recognized to be enlarged or reduced based on the operation signal, the recognized enlargement or In response to the reduction, the image message is enlarged or reduced and displayed, and the image message is displayed based on the operation signal. Sage is dragged in one point, during a rotation operation so as to draw a circular arc around a predetermined position has been recognized, and displays the image message rotates according to the rotation operation.
The moving image reproduction means displays the reproduction button for each of the plurality of image messages, and when it is recognized that the reproduction button is pressed based on the operation signal, the plurality of image messages are displayed. The video corresponding to each is played and displayed, and the pause button is superimposed and displayed. When it is recognized that the pause button is pressed, the video being played is paused and displayed. The reproduction button can be superimposed and displayed.

  The image message generating means may perform predetermined editing on the displayed image to generate an image message to be attached to an e-mail.

  The image message generating means can generate an image message consisting of a still image when an operation signal instructing capture of an image captured by the imaging unit is generated by the operating means.

  The image message generating means includes at least an animation image, a stamp image, a watercolor pen style paint, a pen style paint, and a frame on the image captured by the imaging unit based on an operation signal generated by the operation means. The image message can be added from the image captured by the image capturing unit based on the operation signal generated by the operation means. Can be generated.

  The bulletin board display control means can display the image messages generated by the image message generation means sequentially in random positions at random sizes so as to overlap the generated time series. .

  In the bulletin board display control means, when any one of the plurality of displayed image messages is selected based on an operation signal corresponding to an operation by the user's contact, the selected image message is the highest. It can be made to display so that it may become the front.

  In the bulletin board display control means, when any drag of the displayed image message is recognized based on the operation signal corresponding to the operation by the user's contact, the corresponding image message is displayed on the display unit. In the display surface, it can be moved and displayed in accordance with the movement.

  Based on an operation signal corresponding to an operation by the user's contact, any drag of the displayed image message is recognized by the operation means and moved to a predetermined position in the display surface of the display unit. In the case of being dropped, a transmission means for transmitting an electronic mail with a corresponding image message attached thereto may be further included.

An image processing method according to an aspect of the present invention is an image processing method of an image processing apparatus having a display unit and an imaging unit, and includes an operation step of generating an operation signal by a user's contact with the display unit, and the display A bulletin board displaying an image message is displayed on the section, a bulletin board display control step for displaying a pre-generated image message on the bulletin board, and an imaging mode transition operator displayed in the bulletin board are operated. A captured image display control step for displaying an image captured by the imaging unit on the bulletin board , and an instruction to start and end the recording of the image captured by the imaging unit by the processing of the operation step When the operation signal is generated, the end of the recording is indicated from the timing when the start of the recording is instructed among the images captured by the imaging unit. An image message generation step of generating an image message comprising a moving image from the timing, with respect to the image message, and displayed superimposed on the play button, based on the operation signal, said play button is operated Is recognized, the moving image reproduction step of reproducing the moving image corresponding to the image message, the processing of the bulletin board display control step, the image message newly generated by the processing of the image message generation step, In addition to the image message generated in advance, the image message is displayed on the bulletin board, and the image message is dragged at two points based on the operation signal, and the distance of the dragged position is enlarged or reduced. In response to the recognized enlargement or reduction, the image message When it is recognized that the image message is dragged at one point and rotated to draw a circular arc around a predetermined position based on the operation signal, the image message is displayed. Is rotated and displayed in accordance with the rotation operation .

According to one aspect of the present invention, there is provided a program for controlling an image processing apparatus having a display unit and an imaging unit, an operation step for generating an operation signal by a user's contact with the display unit, and an image on the display unit A bulletin board display control step for displaying a bulletin board for displaying a message and displaying a pre-generated image message on the bulletin board; and an imaging mode transition operator displayed in the bulletin board is operated, whereby the imaging unit A captured image display control step for displaying an image captured by the image board on the bulletin board , and an operation signal for instructing the start of recording of the image captured by the imaging unit and the end of recording are generated by the processing of the operation step. Then, the end of the recording is indicated from the timing when the start of the recording is instructed among the images captured by the imaging unit. An image message generation step of generating an image message by performing predetermined editing respect displayed the image consisting of moving image from the timing, with respect to the image message, and displayed superimposed on the play button, the When it is recognized that the playback button has been operated based on the operation signal, a process including a video playback step of playing a video corresponding to the image message is executed, and the process of the bulletin board display control step includes: The image message newly generated by the processing of the image message generation step is displayed on the bulletin board in addition to the image message generated in advance , and the image message has two points based on the operation signal. When it is dragged and it is recognized that the distance of the dragged position has been enlarged or reduced, In response to the recognized enlargement or reduction, the image message is enlarged or reduced and displayed, and the image message is dragged at one point based on the operation signal to draw an arc around a predetermined position. When it is recognized that the rotation operation is performed, the image message is rotated and displayed according to the rotation operation .

In one aspect of the present invention, there is provided an image processing apparatus having a display unit and an imaging unit, wherein a bulletin board that generates an operation signal by a user's contact with the display unit and displays an image message on the display unit is provided. An image message generated and displayed in advance is displayed on the bulletin board, and an image captured by the imaging unit is displayed on the bulletin board by operating the imaging mode transition operator displayed in the bulletin board. When an operation signal for instructing the start and end of the recording of the image captured by the image capturing unit is generated, the start of the recording is instructed among the images captured by the image capturing unit. from the timing, the recording termination consists moving up timing specified picture message is generated, with respect to the image message, play button is superimposed It is shown, based on the operation signal, when said that the playback button has been operated is recognized, the moving image corresponding to the image message is played, with predetermined editing is performed on the images displayed An image message is generated, and the newly generated image message is displayed on the bulletin board in addition to the previously generated image message, and the image message is dragged at two points based on the operation signal. When it is recognized that the distance of the dragged position is enlarged or reduced, the image message is enlarged or reduced and displayed in response to the recognized enlargement or reduction, and the operation signal is displayed. Based on this, when it is recognized that the image message is dragged at one point and rotated to draw an arc around a predetermined position, Image message is rotated according to the rotation operation.

  The image processing apparatus of the present invention may be an independent apparatus or a block that performs image processing.

  According to one aspect of the present invention, it is possible to generate an image message to be attached to an e-mail using an image captured in real time, and attach an image message to an electronic device using a touch panel. It is possible to easily transmit the e-mail. In addition, it is possible to enjoy editing an image while enjoying the editing of the image while viewing the image captured in real time, and then displaying the image message immediately generated.

It is a figure which shows the external appearance of the image processing apparatus to which this invention is applied. It is a figure which shows the structural example of the function implement | achieved by the image processing apparatus of FIG. It is a flowchart explaining a message management process. It is a figure explaining the example of a display of a main board. It is a flowchart explaining a setting process. It is a figure explaining a setting window. It is a flowchart explaining a new message creation process. It is a figure explaining a new message creation process. It is a flowchart explaining an edit process. It is a figure explaining an edit process. It is a figure explaining an edit process. It is a flowchart explaining a message display operation process. It is a figure explaining message display operation processing. It is a flowchart explaining a message reproduction | regeneration process. It is a figure explaining message reproduction processing. It is a figure explaining message transmission processing. It is a flowchart explaining a message transmission process. It is a figure explaining the message transmission process which transmits a some image message. It is a figure explaining a trash box management process. It is a flowchart explaining a trash box management process.

[Configuration example of image processing apparatus]
FIG. 1 shows a configuration example of an embodiment of hardware of an image processing apparatus to which the present invention is applied. The image processing apparatus 1 of FIG. 1 includes a touch panel and generates an image message based on an image captured in real time. In addition, the image processing apparatus transmits an e-mail with the generated image message attached. That is, the image processing apparatus 1 is, for example, a personal computer including a display including a touch panel and a display and a main body integrated. The image message referred to here is a still image message and a moving image message composed only of a still image and a moving image, and an edited still image message and an edited moving image message that have been further edited by characters, illustrations, effect processing, and the like. including. In the following, when it is not necessary to distinguish between a still image message, a moving image message, an edited still image message, and an edited moving image message, these are collectively referred to as an image message. Further, when it is not necessary to particularly distinguish the presence / absence of editing processing, both the edited still image message and the still image message are simply referred to as a still image message. Furthermore, both the edited moving image message and the moving image message are simply referred to as a moving image message unless it is necessary to distinguish between them.

  The image processing apparatus 1 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a bus 14, an input / output interface 15, an image signal input unit 16, an imaging unit 17, an operation input. A unit 18 and a voice input unit 19 are provided. The image processing apparatus 1 also includes an audio output unit 20, a display unit 21, a storage unit 22, a communication unit 23, and a drive 24.

  The CPU 11 controls the entire operation of the image processing apparatus 1, and expands a program stored in the ROM 12 or the storage unit 22 to the RAM 13 as necessary to execute various processes. The CPU 11 executes various programs based on signals input from various components connected via the bus 14 and the input / output interface 15, and executes various programs via the bus 14 and the input / output interface 15. The processing result is output from the configuration of.

  The image signal input unit 16 accepts various image signals typified by the NTSC (National Television Standards Committee) method, the HDMI (High-Definition Multimedia Interface) method, and the like, and the received image signal is stored in the CPU 11 or as necessary. Supplied to the unit 22. Further, the image signal input unit 16 supplies the input image signal to the display unit 21 including an LCD (Liquid Crystal Display), an organic EL (Electro-Luminescence), and the like for display.

  The imaging unit 17 includes an imaging device such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS), captures an image, and supplies the image to the CPU 11 or the storage unit 22. In addition, the imaging unit 17 supplies the captured image to the display unit 21 made of an LCD, an organic EL, or the like for display.

  The operation input unit 18 has an input function such as a keyboard and a mouse, for example, and generates an operation signal corresponding to the operation content of the user and supplies the operation signal to the CPU 11. In this embodiment, the operation input unit 18 and the display unit 21 are integrated, and a so-called touch panel 102 (see FIG. 2) is configured. That is, the touch panel displays necessary information and displays operation buttons and switches as a user interface, accepts an input operation by a user's contact with respect to the display position of the operation buttons and switches, and corresponds to the accepted input operation. Generate an operation signal. Further, the imaging unit 17 is provided on the upper part of the touch panel 102 and captures an image in a direction facing the touch panel 102. That is, when the user is at a position facing the touch panel 102 and is operating the touch panel 102, the user who is operating is imaged from the front.

The voice input unit 19 is, for example, a microphone and receives a voice input and outputs the received voice as a voice signal. The display unit 21 includes an LCD, an organic EL, and the like, and is controlled by the CPU 11 to display an image based on image signals supplied from the image signal input unit 16, the imaging unit 17, and the storage unit 22.

  The storage unit 22 includes, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), and the like, and is controlled by the CPU 11 and includes various programs, setting data, and image signals (still images and moving images). ), And the audio signal is stored and read out as necessary.

  The communication unit 23 includes an Ethernet board and the like, and is controlled by the CPU 11 to communicate with other electronic devices via a public telephone line (not shown) or the Internet, and exchange various data. The data exchanged by communication with other electronic devices includes e-mail data with other electronic devices.

  The drive 24 is controlled by the CPU 11, and includes a magnetic disk (including a flexible disk), an optical disk (including a CD-ROM (Compact Disc-Read Only Memory), a DVD (Digital Versatile Disc)), and a magneto-optical disk (MD (Mini Disc). )), Or read / write data to / from the removable medium 25 such as a semiconductor memory.

[Configuration example of functions realized by image processing apparatus]
Next, a configuration example of functions realized by the hardware of the image processing apparatus 1 in FIG. 1 will be described with reference to the functional block diagram in FIG. 2 is realized when the hardware of the image processing apparatus 1 in FIG. 1 executes various programs, the hardware having similar functions is configured. It may be made to do. Therefore, the functional block diagram of FIG. 2 may be regarded as a functional block diagram realized by a program, or may be regarded as a hardware block diagram having similar functions.

  The image processing apparatus 1 includes a control unit 101, a touch panel 102, an imaging unit 17, a storage unit 22, and a communication unit 23. The control unit 101 controls the entire operation of the image processing apparatus 1, generates an image message to be attached to an e-mail based on an operation signal supplied from the touch panel 102, and attaches the generated image message. Then, an electronic mail is transmitted from the communication unit 23.

  The control unit 101 includes a new message registration management unit 111, a setting information management unit 112, an operation content recognition unit 113, a message display management unit 114, a reproduction processing unit 115, and a mail transmission management unit 116.

  The new message registration management unit 111 generates a new image message, registers it as message data 202 in the storage unit 22, and manages the message data 202. The new message registration management unit 111 includes a captured image display control unit 121, a message editing processing unit 122, a still image message generation unit 123, a moving image message generation unit 124, and a deletion processing unit 125. The captured image display control unit 121 controls an image to be displayed on the display unit 21 based on the image captured by the imaging unit 17 when generating a new image message.

  When editing an image captured as a new image message, the message editing processing unit 122 edits the image based on the operation signal of the touch panel 102 and registers it as message data 202.

  The still image message generation unit 123 generates an image message based on the still image captured by the imaging unit 17, registers it as message data 202 in the storage unit 22, and displays it on the display unit 21. The moving image message generation unit 124 generates an image message based on the moving image captured by the imaging unit 17, registers it as message data 202 in the storage unit 22, and displays it on the display unit 21.

  The deletion processing unit 125 manages processing for deleting or restoring the message data 202 stored in the storage unit 22 by putting it in a trash box described later. More specifically, the deletion processing unit 125 includes a restoration processing unit 131 and a trash can management unit 132. When the generated message is put into the trash box (when dragged and dropped on the icon indicating the trash box), the trash box management unit 132 regards it as deleted, deletes it on the display unit 21, and stores the storage unit 22. Is registered in the trash box data 204. When the trash management process is instructed, the trash management unit 132 reads the trash data 204 in the storage unit 22 and displays a list of deleted messages. When deletion is instructed based on the operation of the operation input unit 18, the trash box management unit 132 deletes the message data registered in the trash box data 204, and completely deletes the message data. On the other hand, when the restoration is instructed, the restoration processing unit 131 reads the data of the message instructed to restore from the trash box data 204 and restores it, and displays it on the display unit 21 in the original state.

  The setting information management unit 112 registers and manages setting information for setting the type of the imaging unit 17 used for creating a new message, a folder in the storage unit 22 for storing message data, and the like in the storage unit 22. .

  The operation content recognition unit 113 recognizes the operation content based on the operation signal supplied from the operation input unit 18 configuring the touch panel 102 and outputs the recognized operation content. That is, the operation content recognizing unit 113 responds to a displayed button or icon from the correspondence between the position touched by the user on the display screen of the display unit 21 and the position on the display image, and the contact pressure. Recognize what operation is being performed. More specifically, the operation content recognition unit 113 recognizes, for example, a pressing operation, a drag operation, a drop operation, or the like for a button or icon on the display unit 21.

  The message display management unit 114 displays a bulletin board-like message board that is set as an image message display area (message board display area) among the display areas of the display unit 21, and displays image messages displayed on the message board. Manage display status. More specifically, the message display management unit 114 manages the display state of image messages generated in advance based on the message data 202 in the storage unit 22. Further, the message display management unit 114 displays the image message newly generated by the new message registration management unit 111 in addition to the image message display generated in advance, and manages the display state. More specifically, the message display management unit 114 includes a movement operation processing unit 151, an enlargement / reduction operation processing unit 152, and a rotation operation processing unit 153. The movement operation processing unit 151 moves and displays the display position of the image message in accordance with the operation content input to the operation input unit 18. The enlargement / reduction operation processing unit 152 causes the display size of the image message to be enlarged or reduced according to the operation content input to the operation input unit 18 and displayed. The rotation operation processing unit 153 rotates and displays the display angle of the image message according to the operation content input to the operation input unit 18.

  In response to the operation content input to the operation input unit 18, the reproduction processing unit 115 reproduces an image message composed of a still image and an image message composed of a moving image corresponding to each of the image messages.

  The mail transmission management unit 116 transmits an image message attached to an electronic mail according to the operation content input to the operation input unit 18. More specifically, the mail transmission management unit 116 includes a mail setting information registration unit 191, a message selection recognition unit 192, a mail editing processing unit 193, and a mail transmission processing unit 194. The mail setting information registration unit 191 registers e-mail setting information to which an image message is attached in the storage unit 22 as mail setting information 203. The mail setting information includes information on the destination address, subject, and text, and is managed in the storage unit 22 for each profile name. The message selection recognition unit 192 recognizes which image message has been selected according to the operation content of the operation input unit 18. The mail editing processing unit 193 edits the destination address, subject, and text information included in the mail setting information according to the operation content of the operation input unit 18. When an e-mail transmission is instructed according to the operation content of the operation input unit 18, the mail transmission processing unit 194 controls the communication unit 23 to attach the selected image message based on the mail setting information. And send an email.

[Message management processing]
Next, message management processing will be described with reference to the flowchart of FIG.

  In step S1, the message display management unit 114 determines whether or not the operation content recognition unit 113 has instructed the start of the message management process, and repeats the same process until the start of the message management process is instructed. In step S1, when an operation button that is displayed on the touch panel 102 and instructed to start a message management process (not shown) is operated by the user, in step S2, the operation input unit 18 generates a corresponding operation signal. To the control unit 101. At this time, when it is recognized that the start of the message management process has been instructed based on the operation signal, the operation content recognition unit 113 notifies the message display management unit 114 to that effect. The message display management unit 114 displays, for example, a message board 301 as shown in FIG. 4 according to the notification indicating that the operation instructing the start of the message management process has been recognized from the operation content recognition unit 113.

  FIG. 4 shows a state where the touch panel 102 displays the message board 301. The touch panel 102 includes an imaging unit 17 on the upper part of the casing. For this reason, the imaging part 17 images the range which opposes the surface which comprises the operation input part 18 and the display part 21 (all are the same thing in the figure). As a result, the imaging unit 17 effectively captures a user who operates or views the touch panel 102 of the image processing apparatus 1.

  In step S3, the message display management unit 114 reads the message data 202 stored in the storage unit 22, and displays image messages 321-1 to 321-3 on the main board 301, for example, as shown in FIG. Is displayed.

  In FIG. 4, the image message 321-1 displays an image that can be seen as three families, and is described as three months ago on the lower right side. It has been shown to have been generated months ago. The description “3 months ago” in the lower row is displayed by the message display manager 114 based on the date and time when the image message 321-1 was generated.

  In addition, the image message 321-2 displays an image that can be viewed as two parents and children, and is described as three months ago on the right side of the lower row, and an image message composed of images that can be viewed as two parents and children is generated three months ago. It is shown that Furthermore, in the image message 321-3, an image that can be seen as four family members is displayed, and 2009/5/19 15:06:02 is displayed on the lower right side. That is, it is shown that the image message 321-3 seen as four family members is an image message generated on May 19, 2009 at 15: 6: 02.

  In the following description, when it is not necessary to distinguish each of the image messages 321-1 to 321-3, they are simply referred to as an image message 321 and other configurations are also referred to similarly. Further, as will be described later, in the case of an image message composed of moving image data, the message display management unit 114 further displays a playback button having a triangular shape protruding to the right near the approximate center of the image message 321. By providing a difference in display as described above, it is possible to identify at a glance whether the image message 321 is based on still image data or moving image data.

  Further, buttons 311 to 314 are displayed on the upper right of the message board 301 from the left. The button 311 is a button that is pressed when displaying setting information. The button 312 is a button that is pressed when the message board 301 is displayed in a minimized manner. The button 313 is a button that is pressed when the message board 301 is maximized or returned to the original size. The button 314 is a button that is pressed when the process is ended. As described above, these buttons 311 to 314 are recognized as a region where the buttons 311 to 314 are displayed on the display unit 21 of the touch panel 102 as being pressed by the user touching with a fingertip or the like. In this embodiment, other buttons can be pressed by the same operation.

  Further, on the message board 301, an icon 331 for instructing generation of a new message, an icon 332 for instructing transmission of an e-mail attached with an image message, and an icon 333 for instructing input of the image message to the trash box are displayed. ing.

  The icon 331 is pressed by a pointer (not shown) when the user wants to create a new image message. The icon 332 drops the image message selected by the user when an instruction to send an e-mail with the image message selected by the user is attached. That is, in the case of FIG. 4, when any of the image messages 321-1 to 321-3 is dragged and dropped on the icon 332, transmission of an e-mail attached with the dropped image message 321 is instructed. .

  Further, the icon 333 drops the image message 321 selected by the user when trying to put the image message selected by the user into the trash box. That is, in the case of FIG. 4, when any of the image messages 321-1 to 321-3 is dragged and dropped on the icon 333, an instruction to put the dropped image message 321 into the trash box is given.

  Here, dragging is an operation of selecting one of the image messages 321 and moving it within the display area of the display unit 21, and dropping refers to releasing the selected state of the image message 321 and moving it. It is an operation that is ended and placed at a moving position in the display area.

  By displaying the message board 301 in this way, it is possible to display the image message 321 with an image as if a bulletin board was actually provided and a photograph was pasted there. Note that the position, size, and rotation angle of the image message 321 on the message board 301 are randomly set and displayed by the message display management unit 114. In FIG. 4, since there are only three image messages 321, there is room on the message board 301, and even if all are displayed, there is space, but when many image messages 321 are displayed, Some things overlap. In such a case, the message display management unit 114 displays the new message so that it is displayed in front of the old one based on the date and time information when the image message 321 is generated. That is, the latest one is displayed on the foreground, and the oldest one is displayed on the back. For this reason, it is possible to express the atmosphere of the actual bulletin board.

  Now, the description returns to the flowchart of FIG.

  In step S4, the setting information management unit 112 determines whether or not the operation content recognition unit 113 has instructed display of the setting information. That is, it is determined whether or not the button 311 on the message board 301 displayed on the display unit 21 of the touch panel 102 is operated by the operation input unit 18 to instruct display of the setting information.

  In step S4, for example, when the operation button 311 is operated to instruct display of setting information, the setting information management unit 112 executes setting processing in step S5.

[Setting process]
Here, the setting process will be described with reference to the flowchart of FIG.

  In step S21, the setting information management unit 112 displays a web camera setting window 401-1 as setting information, for example, as shown in the upper part of FIG. In the web camera setting window 401-1 in FIG. 6, a tab 411 that is operated when switching to the web camera setting screen and a tab 412 that is operated when switching to the media folder setting screen are displayed in the upper part. Also, a button 413 labeled “OK” that is operated when the input of the setting information is completed, a button 414 that is labeled “Cancel” that is operated when the input of the setting information is canceled, and an input of the setting information. A button 415 that is operated when the process is ended is displayed.

  Below the tabs 411 and 412, a video device setting field 421, a resolution setting field 422, a check box 423 for setting a video mirror display, and an update button 424 indicating updating to a setting state are displayed. Yes. In the video device setting field 421, by pressing a button on the right side, selectable video devices are displayed as a drop-down list and can be selected. Further below, a check box 431 for setting whether or not an image message with audio is set, an audio device setting field 432, and an audio input pin setting field 433 are displayed.

  FIG. 6 shows that “Visual Communication Camera” is selected as the video device. In the resolution setting field 422, by selecting a button on the right side, selectable resolutions are displayed as a drop-down list and can be selected. FIG. 6 shows that a resolution of 640 pixels × 480 pixels is selected. The check box 423 is displayed as “video mirror”, and a check mark is input when displaying a mirror-related image when displaying an image captured by the imaging unit 17. That is, as described above, the imaging unit 17 is provided on the upper part of the touch panel 102 and images the opposite direction. For this reason, when the user is present facing the front surface of the touch panel 102, when the image captured by the imaging unit 17 is displayed on the display unit 21 of the touch panel 102, the displayed user image is the actual user orientation. Is not a mirror relationship. Then, many users have a sense of incongruity by looking at their own images displayed on the touch panel 102. In other words, humans usually have many opportunities to confirm their appearance in the mirror, so even if they see their own images that are not mirror-related, it is difficult to imagine the change of the image due to their movements. A sense of incongruity is likely to occur with respect to the movement of Therefore, when the check box 423 is checked, the display unit 21 displays the image captured by the imaging unit 17 while being horizontally reversed. As a result, the user can edit an image made up of the user's own image in an editing process described later without feeling uncomfortable by seeing the user's own image having a mirror-like relationship.

  In the check box 431, a check mark is input when an image message has audio, and at this time, the audio device setting field 432 and the audio input pin setting field 433 are operable. In the audio device setting field 432, selectable audio devices are displayed and can be selected as a drop-down list by pressing the button on the right side. FIG. 6 shows that “MIC” is selected as the audio device. In the audio input pin setting field 433, selectable audio input pins are displayed and can be selected as a drop-down list by pressing a button on the right side. In FIG. 6, “master volume” is displayed, which indicates that sound is input according to the master volume of the image processing apparatus 1.

  In step S22, the setting information management unit 112 determines whether or not the setting information for setting the web camera has been input by operating the operation input unit 18 of the touch panel 102 based on the operation content recognized by the operation content recognition unit 113. Determine whether. In step S22, for example, when setting information for setting the web camera is input, in step S23, the setting information management unit 112 displays the setting information reflecting the input setting information on the display unit 21 of the touch panel 102. To display. On the other hand, when the setting information for setting the web camera is not input, the process of step S23 is skipped.

  In step S24, the setting information management unit 112 determines whether the media folder setting is instructed by operating the tab 412. In step S24, for example, when the tab 412 is operated and the setting of the media folder is instructed, in step S25, the setting information management unit 112 is displayed on the display unit 21, for example, in the lower part of FIG. In this way, the media folder setting window 401-2 is displayed.

  The media folder setting window 401-2 is provided with an image folder setting field 451 and a video folder setting field 452 from the top, and sets folders in the storage unit 22 for registering image messages composed of still images and moving images. A column to do is provided. The image folder setting field 451 and the video folder setting field 452 are respectively provided with reference buttons 451a and 452a on the right side. The reference buttons 451a and 452a are pressed when referring to selectable folders. When the reference buttons 451a and 452a are operated, the selectable folders are displayed and can be selected. In FIG. 6, “C: \ Users” is selected, and it is set to be registered in the folder specified by “\ Users” of the C drive.

  In step S <b> 26, the setting information management unit 112 determines whether or not the setting information for setting the media folder is input by operating the operation input unit 18 of the touch panel 102 based on the operation content recognized by the operation content recognition unit 113. Determine whether. In step S26, for example, when setting information for setting the media folder is input, in step S27, the setting information management unit 112 displays the setting information in a state in which the input setting information is reflected on the display unit 21 of the touch panel 102. To display. On the other hand, when the setting information for setting the media folder is not input, the process of step S27 is skipped.

  In step S24, if media folder setting is not instructed, the processes in steps S25 to S27 are skipped.

  In step S28, the setting information management unit 112 determines whether the setting of the web camera is instructed again by pressing the tab 411. For example, when the tab 411 is pressed and web camera setting is instructed, the process returns to step S21. On the other hand, if the tab 411 is not pressed in step S28 and the setting of the web camera is not instructed, in step S29, the setting information management unit 112 presses the button 413 to “OK”, that is, the setting content. It is determined whether or not the setting end is instructed. If the button 413 is operated in step S29, the process proceeds to step S30.

  In step S30, the setting information management unit 112 updates the setting information 201 based on the setting contents of the web camera setting window 401-1 and the media folder setting window 401-2, and stores the setting information 201 in the storage unit 22. The process ends.

  On the other hand, if the button 413 is not pressed in step S29 and the end of the setting process is not instructed, in step S31, the setting information management unit 112 has pressed the button 414 and instructed to cancel the setting process. Determine whether or not.

  In step S31, for example, when the button 414 is pressed and cancellation is instructed, the process ends. That is, the setting information management unit 112 ends the setting process without updating the setting information 201 regardless of the setting contents of the web camera setting window 401-1 and the media folder setting window 401-2. In step S31, whether or not the button 415 that is pressed when instructing the end is pressed is the same as that of the button 414.

  If the button 414 is not pressed in step S31, the setting information management unit 112 determines in step S32 whether the web camera setting window 401-1 is currently displayed. In step S31, if the web camera setting window 401-1 is displayed, the process returns to step S21. If the web camera setting window 401-1 is not displayed, the process returns to step S25. That is, as long as the buttons 413 to 415 are not operated, the processes of steps S21 to S29, S31, and S32 are repeated, the setting information is displayed, and the state in which the setting can be changed is maintained. When the button 413 is pressed, the setting information 201 is updated so that the setting state at that time is reflected and stored in the storage unit 22, and the setting process is completed, and when the buttons 414 and 415 are pressed. The setting process ends without the setting information 201 being updated.

  Through the above processing, setting information for generating an image message is updated and registered.

  Now, the description returns to the flowchart of FIG.

  When the setting process ends in step S5, the process proceeds to step S6. If the display of the setting information is not instructed in step S4, the process of step S5 is skipped.

  In step S <b> 6, the new message registration management unit 111 determines whether the creation of a new message is instructed based on the recognition result of the operation content of the operation input unit 18 of the touch panel 102 by the operation content recognition unit 113. In step S6, for example, when the icon 331 shown in FIG. 4 is pressed by a pointer (not shown) to instruct creation of a new message, the process proceeds to step S7.

  In step S <b> 7, the new message registration management unit 111 executes a new message creation process, creates a new message, and registers it in the message data 202 of the storage unit 22.

[New message creation process]
Here, the new message creation process will be described with reference to the flowchart of FIG.

  In step S41, the captured image display control unit 121 activates the imaging unit 17, and starts image capturing. With this processing, the imaging unit 17 starts imaging, and supplies an image captured via an optical system (not shown) to the control unit 101 as image data.

  In step S42, the captured image display control unit 121 acquires the image captured by the imaging unit 17, and displays it on the display unit 21 of the touch panel 102, for example, as shown by the uppermost window 501-1 in FIG. Let The window 501-1 includes a captured image display field 511, an edit tray display field 512, and an edit tool display field 513. The captured image display field 511 displays the image data itself supplied from the imaging unit 17 in real time. The captured image display field 511 is provided with a button 521 that is operated when instructing capture, a button 522 that is operated when instructing start and stop of moving image recording, and a button 523 that is operated when ending.

  In the captured image display field 511 of FIG. 8, the upper body of the person imaged by the imaging unit 17 is displayed. In FIG. 8, the message board 301-1 displaying the captured image display field 511 in a state where the editing process described later is not performed from the top, the message board 301-2 after editing, and after the capturing process described later. Each message board 301-3 is displayed.

In the edit tray display field 512, trays are switched and displayed according to the editing tool. Types of editing tools include animation editing, stamp editing, watercolor pen editing, eraser editing, pen editing, and frame editing (frame switching). In the uppermost row in FIG. 8, a tray corresponding to watercolor pen editing is displayed in the edit tray display field 512.

  In the editing tool display field 513, icon images indicating the types of editing that can be selected are displayed as editing tools, and the editing content is switched when each is selected. In the example of FIG. 8, a star-shaped icon that selects animation editing, a stamp-shaped icon that selects stamp editing, a brush-shaped icon that selects watercolor pen editing, and an eraser-shaped icon that selects eraser editing. An icon is displayed. In addition, a pen-shaped icon for selecting pen editing and a frame-shaped icon for selecting frame editing are displayed. Then, an editing tool is selected according to the type of the selected icon, and the tray displayed in the editing tray display field 512 is switched and displayed corresponding to the selected editing tool.

  In step S43, the message edit processing unit 122 determines whether the operation input unit 18 of the touch panel 102 is operated by the operation content recognition unit 113 and an editing operation is instructed. That is, for example, when the editing tool display column 513 or the edit tray display column 512 displayed on the display unit 21 constituting the touch panel 102 is operated by the operation input unit 18 and editing is instructed, the process is performed in step S44. Then, the editing process is executed.

[Edit processing]
Here, the editing process will be described with reference to the flowchart of FIG.

  In step S <b> 61, the message editing processing unit 122 determines whether animation editing has been selected by the operation content recognition unit 113 using a pointer (not shown). In step S61, for example, if the star-shaped icon in the editing tool display field 513 in FIG. 8 is selected and animation editing is selected, the process proceeds to step S62.

  In step S62, the message edit processing unit 122 switches the display contents of the edit tray display field 512 to, for example, the edit tray display field 512-1 indicating the animation edit mode as shown in the top row of FIG. . A list display field 561, a return button 562, and a feed button 563 are displayed in the edit tray display field 512-1 indicating animation editing. In FIG. 10, in the list display field 561, the reproduction start images of the animation images are displayed as a list for every seven types. Each time the feed button 563 is operated, the scroll button is scrolled to the left, and the return button 562 is displayed. Scrolls to the right when operated. When the user selects any playback start image with a pointer (not shown), the selected animation image is displayed in the captured image display field 511. The display position of the animation image can be changed by dragging and dropping using a pointer (not shown).

  In step S63, the message editing processing unit 122 determines whether any animation image has been selected by the operation content recognition unit 113 with a pointer (not shown). If any animation image is selected in step S63, the message editing processing unit 122 displays the selected animation image in the captured image display field 511 in step S64. The animation image is an animation image in which, for example, a large number of balloons are displayed and then each of them floats. Also, animation images such as how the brightness of stars disappears may be prepared and repeatedly displayed. That is, when a drag operation is performed on the display surface of the display unit 21 of the touch panel 102 with a pointer (not shown), the animation image is displayed for a while immediately after the drag for each dragged position, and then gradually disappears. It is what you want to do. By displaying in this way, it is possible to generate an animation image that makes it appear as if the star is shining in accordance with the drag movement.

If no animation editing is selected in step S61, or if no animation is selected in step S63, the process proceeds to step S65.

  In step S65, the message editing processing unit 122 determines whether or not stamp editing has been selected by the operation content recognition unit 113 using a pointer (not shown). In step S65, for example, when the stamp-like icon in the editing tool display field 513 in FIG. 8 is selected and stamp editing is selected, the process proceeds to step S66.

  In step S66, the message edit processing unit 122 switches the display content of the edit tray display field 512 to the edit tray display field 512-2 indicating the stamp edit mode as shown in the second row of FIG. 10, for example. To do. A list display field 571, a return button 572, a feed button 573, and a size selection field 574 are displayed in the edit tray display field 512-2 indicating stamp editing. In FIG. 10, in the list display field 571, stamp images are displayed as a list for every six types, and each time the feed button 573 is operated, the screen is scrolled to the left and the return button 572 is operated. Scroll to the right. The size selection field 574 is provided with select buttons for selecting “small”, “medium”, and “large” from the top. In FIG. 10, “medium” is selected. Yes. When the user selects any stamp image with a pointer (not shown), the selected stamp image is displayed in the captured image display field 511. The display position of the stamp image can be changed by dragging and dropping using a pointer (not shown).

  In step S <b> 67, the message editing processing unit 122 determines whether any stamp image has been selected by the operation content recognition unit 113 using a pointer (not shown). If any stamp image is selected in step S67, the message editing processing unit 122 displays the selected stamp image in the captured image display field 511 in step S68.

If stamp editing is not selected in step S65, or if no stamp image is selected in step S67, the process proceeds to step S69.

  In step S <b> 69, the message editing processing unit 122 determines whether or not the watercolor pen editing has been selected by the operation content recognition unit 113 using a pointer (not shown). In step S69, for example, when the brush icon in the editing tool display field 513 in FIG. 8 is selected and watercolor pen editing is selected, the process proceeds to step S70.

  In step S70, the message edit processing unit 122 switches the display content of the edit tray display field 512 to, for example, the edit tray display field 512-3 indicating the watercolor pen edit mode as shown in the third row of FIG. indicate. A list display field 581, a size field 582, and an opacity field 583 are displayed in the edit tray display field 512-3 indicating watercolor pen editing. In FIG. 10, the list display field 581 displays seven types of watercolor pen colors as a list. Further, the size column 582 is provided with a knob for selecting the size of the thickness, and can be set in units of pixels by moving left and right. Further, the opacity column 583 is provided with a knob for selecting the opacity, and can be set as a percentage by moving left and right. When a watercolor pen is selected with the pointer, the captured image is displayed as if it were drawn with the watercolor pen according to the movement of the pointer, depending on the thickness and opacity set in the size field 582 and the opacity field 583 respectively. An image can be drawn in the column 511.

  In step S <b> 71, the message editing processing unit 122 determines whether any watercolor pen is selected by the operation content recognition unit 113 with a pointer (not shown) and edited (whether an image is drawn). If one of the watercolor pens is selected in step S71 and the movement of the pointer is started, in step S72, the message editing processing unit 122 picks up the captured image as the pointer moves with the watercolor pen of the selected color. An image is drawn in the display field 511.

If the watercolor pen editing is not selected in step S69, or if the editing with the watercolor pen is not performed in step S71, the process proceeds to step S73.

  In step S <b> 73, the message editing processing unit 122 determines whether or not the operation content recognition unit 113 has selected the eraser editing with a pointer (not shown). In step S73, for example, when the eraser-like icon in the editing tool display field 513 in FIG. 8 is selected and eraser editing is selected, the process proceeds to step S74.

  In step S74, the message edit processing unit 122 switches the display content of the edit tray display field 512 to the edit tray display field 512-4 indicating the eraser edit mode as shown in the fourth row of FIG. 10, for example. To do. In the edit tray display column 512-4 indicating the eraser editing, a size column 591, an image column 592, and an all delete button 593 are displayed. In FIG. 10, the size column 591 is provided with a knob for setting a size to be deleted by editing, and the size is set in units of pixels by moving left and right. In the image column 592, a spot having a size equivalent to the size at this time is displayed. That is, the user can set the size while recognizing the range to be erased by looking at the spot size in the image field 592 while moving the knob in the size field 591 left and right. The delete all button 593 is a button operated when deleting all the contents drawn by editing. As a result, when the eraser edit is selected by the pointer, the edit drawn in the captured image display field 511 is as if it is erased by the eraser with the movement of the pointer at the size set in the size field 591. The contents can be erased. Also, when the delete all button 593 is operated, all edit contents are deleted.

  In step S <b> 75, the message editing processing unit 122 determines whether the operation content recognition unit 113 has performed deletion editing with a size set by a pointer (not shown) or whether the delete button 593 has been pressed. In step S75, when the movement of the pointer is started, or when the delete all button 593 is pressed, the process proceeds to step S76. That is, in step S76, the message editing processing unit 122 deletes the edited contents in the captured image display field 511 with the movement of the pointer at the size set in the selected size field 591 or deletes all edited contents. to erase.

  If no eraser editing is selected in step S73, and if no eraser editing is performed in step S75, the process proceeds to step S77.

  In step S77, the message editing processing unit 122 determines whether pen editing has been selected by the operation content recognition unit 113 using a pointer (not shown). In step S77, for example, when the pen-shaped icon in the editing tool display field 513 in FIG. 8 is selected and pen editing is selected, the process proceeds to step S78.

  In step S78, the message edit processing unit 122 switches the display content of the edit tray display field 512 to the edit tray display field 512-5 indicating the pen edit mode as shown in the fifth row of FIG. 10, for example. To do. A list display column 601, a size column 602, and an opacity column 603 are displayed in the edit tray display column 512-5 indicating pen editing. In FIG. 10, the list display field 601 displays seven types of pen colors as a list. Further, the size column 602 is provided with a knob for selecting the size of the thickness, and can be set in units of pixels by moving left and right. Further, the opacity column 603 is provided with a knob for selecting the opacity, and can be set as a percentage by moving left and right. When a pen is selected by the pointer, the picked-up image display field 511 as if drawn with a pen as the pointer moves due to the thickness and opacity set in the size field 602 and the opacity field 603 respectively. An image can be drawn inside.

In step S79, the message editing processing unit 122 determines whether any pen has been selected by the operation content recognition unit 113 with a pointer (not shown) and editing has been performed (whether an image has been drawn). If any pen is selected in step S79 and the movement of the pointer is started, in step S80, the message editing processing unit 122 displays the captured image display field as the pointer moves with the pen of the selected color. Draw an image at 511.

If pen editing is not selected in step S77, or if editing with a pen is not performed in step S79, the process proceeds to step S81.

  In step S81, the message editing processing unit 122 determines whether or not frame editing has been selected by the operation content recognition unit 113 using a pointer (not shown). In step S81, for example, when the frame-shaped icon in the editing tool display field 513 in FIG. 8 is selected and frame editing is selected, the process proceeds to step S82.

  In step S82, the message edit processing unit 122 switches the display content of the edit tray display field 512 to the edit tray display field 512-6 indicating the frame edit mode as shown in the sixth row of FIG. 10, for example. To do. In the edit tray display column 512-6 indicating frame editing, a list display column 611, a return button 612, and a feed button 613 are displayed. In FIG. 10, in the list display field 611, frame images are displayed as a list for every seven types, and each time the feed button 613 is operated, the screen is scrolled to the left and the return button 612 is operated. Scroll to the right. When the user selects any frame image with a pointer (not shown), the selected frame image is displayed in the captured image display field 511.

In step S <b> 83, the message editing processing unit 122 determines whether any frame image is selected by the operation content recognition unit 113 using a pointer (not shown). If any frame image is selected in step S83, the message editing processing unit 122 displays the selected frame image in the captured image display field 511 in step S84 .

If no frame editing is selected in step S81, or if no frame image is selected in step S83, the editing process ends.

  In other words, the animation image, stamp image, watercolor pen, eraser, pen, and frame image can be edited on the captured image display field 511 by the above processing. As a result, for example, it is possible to edit from the state of the captured image display field 511 of the window 501-1 in FIG. 8 to the state of the captured image display field 511 of the window 501-2. In the window 501-2 of FIG. 8, a stamp image such as a strawberry fruit is added to a stamp image resembling a square sticky note drawn with a pen as “Happy Congratulations”, and a birthday cake is added. And a stamp image imitating a present.

  That is, when a button 331 for instructing creation of a new message is pressed, the user who is imaged by the imaging unit 17 and is present in front of the touch panel 102 is displayed in the captured image display field 511 of the display unit 21. By operating the operation input unit 18 of the touch panel 102, the user can add an editing operation while looking at himself / herself displayed in the captured image display field 511, and is added to his / her image in real time. You can edit while enjoying watching the edited contents. In addition, a plurality of people gather in front of the touch panel 102 and the plurality of people are displayed in the captured image display field 511, so that all the gathered people can enjoy the editing operation while viewing the editing state. . Furthermore, it is possible to display and enjoy the image message finally created by all the gathered people on the message board 301.

  Now, the description returns to the flowchart of FIG.

  If the captured image display field 511 is edited by the processing in step S44, for example, as indicated by the window 501-2 in FIG. 8, the processing proceeds to step S45. If no editing operation is instructed in step S43, the process in step S44 is skipped.

  In step S <b> 45, the still image message generation unit 123 determines whether capture is instructed by the operation content recognition unit 113 using a pointer (not shown). In step S45, for example, as indicated by the window 501-3 in FIG. 8, when the operation input unit 18 is operated, the pointer 531 is operated, the button 521 is pressed, and capture is instructed. Advances to step S46. Note that the pointer 531 in the figure is represented as an icon for convenience, and in this example, is the touch panel 102, so the pointer 531 touches a contact portion that becomes the operation input unit 18 of the touch panel 102. It is the user's finger itself.

  In step S46, the still image message generating unit 123 reads the captured image currently displayed in the captured image display field 511, generates a new image message including a still image, and stores the message data 202 in the storage unit 22. The registration proceeds to step S54. That is, a still image message or an edited still image message is generated as an image message. At this time, the still image message generating unit 123 reads the setting information 201 stored in the storage unit 22, and, for example, in the folder set in the image folder setting field 451 of the media folder setting window 401-2 in FIG. Register an image message. If the above-described editing process has been added to the captured image display field 511, the captured image displayed in the captured image display field 511 including the editing process result is registered as an image message.

  In step S54, the message display management unit 114 randomly sets a display size, a display position, and a rotation angle for displaying a newly registered image message.

  In step S55, the message display management unit 114 reads the newly registered message data from the storage unit 22, displays the set display size, display position, and rotation angle in addition to the already displayed image message 321. To do. That is, the message display management unit 114 displays, for example, an image message 321-11 on the lowermost message board 301-3 in FIG.

  With the above processing, it is possible to generate an image message using the image currently captured by the imaging unit 17. At this time, it is possible to apply editing processing to the image being captured as described above. Furthermore, when an image message is generated, a new image message including an editing process can be generated only by an operation with a pointer. As a result, even in an electronic device that employs the touch panel 102 that is not suitable for so-called keyboard input, like the image processing apparatus 1 of the present embodiment, the image using the image can be edited easily. A new message can be created.

  In step S56, the new message registration management unit 111 determines whether or not the operation content recognition unit 113 has instructed the end of the new message creation process using a pointer (not shown). In step S56, for example, when a pointer (not shown) is operated by the operation input unit 18 and the button 523 is pressed to end the new message creation process, the process ends. If the end is not instructed in step S56, the process returns to step S42.

If capture is not instructed in step S45, the moving image message generating unit 124 determines in step S47 whether recording is instructed by the operation content recognizing unit 113 using a pointer (not shown). In step S47, the example, from the state of the window 501-11 in FIG. 11, when it is pressed by the button 522 is a pointer 531, as shown in the window 501-12 of the second stage 11, which recording is instructed The process proceeds to step S48. Note that the window 501-11 in FIG. 11 is in the same state as the window 501-1 in FIG.

  In step S48, the moving image message generation unit 124 starts recording the image captured by the imaging unit 17, and, for example, in the folder set in the video folder setting field 452 in the media folder setting window 401-2 in FIG. Record sequentially.

  In step S49, similar to the processing in step S43, the message editing processing unit 122 determines whether the operation input unit 18 of the touch panel 102 has been operated by the operation content recognition unit 113 and an editing operation has been instructed. That is, for example, when the editing tool display column 513 or the editing tray display column 512 displayed on the display unit 21 constituting the touch panel 102 is operated by the operation input unit 18 and editing is instructed, the process is performed in step S50. Then, the editing process is executed. Note that the editing process in step S50 is the same as the editing process in step S44, and a description thereof will be omitted.

  That is, by the processing in step S44, for example, as shown in the captured image display field 511-12 in FIG. 11, the editing is performed so that a frame image that looks like a coral on the seabed is attached, and then an instruction to start recording is given. Then, recording is started from the state where the frame image is attached. Then, as a result of the editing process in step S50, as shown in the captured image display field 511-13, when characters such as “I went to the aquarium today” are input by pen editing or the like, “I went to the aquarium today. The process of drawing the letters “Tayo” will be recorded.

In step S51, the moving image message generating unit 124 determines whether or not the operation content recognition unit 113 has instructed the end of recording with a pointer (not shown). If the end is not instructed, the process returns to step S48. . That is, steps S48 to S51 are repeated until the end of recording is instructed. In step S51, for example, when the button 522 is pressed by the pointer 531 as shown in the lowermost window 501-14 in FIG. 11 , it is considered that the end of the recording has been instructed, and the process proceeds to step S52. Proceed to

  In step S <b> 52, the moving image message generation unit 124 ends the recording of the image captured by the imaging unit 17.

  In step S <b> 53, the moving image message generating unit 124 generates an image message based on the recorded moving image data and registers it in the message data 202 of the storage unit 22. At this time, the moving image message generation unit 124 reads the setting information 201 stored in the storage unit 22 and stores, for example, images in the folder set in the video folder setting field 452 of the media folder setting window 401-2 in FIG. Register a message.

  As a result of the processing in steps S54 and S55, an image message 321-21 composed of a newly created moving image is displayed as shown by the message board 301-14 in FIG. At this time, the image message 321-21 displays, for example, the image of the last frame of the moving image data.

  In step S47, if recording is not instructed, the processes in steps S48 to S55 are skipped.

  With the above processing, it is possible to generate an image message using a moving image recorded while being imaged now by the imaging unit 17. At this time, it is possible to apply editing processing to the image being captured as described above. Furthermore, when an image message is generated, a new image message including an editing process can be generated only by an operation with a pointer (a touch operation on the operation input unit 18 of the touch panel 102). In addition, it is possible to generate an image message in which all editing processes from the start of recording to the end of recording are directly used as moving images. As a result, as in the image processing apparatus 1 of the present embodiment, even in an electronic device that employs the touch panel 102 that is not suitable for so-called keyboard input, the moving image is easily edited and edited. It is possible to create a new image message using. Note that, when an image message using a moving image is set with audio by setting the check box 431 in FIG. 6 described above, a moving image including audio data together with the moving image is generated as an image message. It becomes possible.

  Further, the image captured by the image capturing unit 17 in real time is used to perform desired editing processing while the user himself / herself is viewing the editing state of the user, and then an image message is generated, and the generated image message is displayed on the message board. It is possible to display on 301 and enjoy. Further, for example, the whole family sits at a position where it can be imaged by the imaging unit 17, and the family edits images and videos as desired to generate image messages, and the generated image messages are displayed on the message board. It can be displayed and enjoyed.

  Now, the description returns to the flowchart of FIG.

  If it is not instructed to create a new message in step S6, the process in step S7 is skipped.

  In step S8, the reproduction processing unit 115 determines whether or not the operation content recognition unit 113 has performed an operation for displaying the image message 321 with a pointer (not shown). In step S8, for example, when it is determined that the display operation of the image message 321 has been performed, the process proceeds to step S9, and the message display operation process is executed.

[Message display operation processing]
Here, the message display operation process will be described with reference to the flowchart of FIG.

  In step S <b> 101, the still image reproduction processing unit 172 of the reproduction processing unit 115 determines whether any of the image messages 321 is selected by the pointer 531 by the operation content recognition unit 113. For example, when the image message 321-31 is selected by the pointer 531, as illustrated in the uppermost message board 301-21 in FIG. 13, the still image reproduction processing unit 172 proceeds to step S <b> 102.

  In step S <b> 102, the still image reproduction processing unit 172 switches the display so that it is positioned in front of other image messages 321 as indicated by the image message 321-32 of the message board 301-22 in FIG. 13. In other words, the image message 321 is displayed so as to be superimposed on the foreground in order from the rearmost surface in time series, but by being selected in this way, the image message 321 is displayed on the foreground and is selected. The image message 321 is highlighted and displayed. By such processing, an image message composed of a still image is actually reproduced and displayed.

  In step S103, the movement operation processing unit 151 of the message display management unit 114 determines whether or not the image message 321 selected by the pointer 531 by the operation content recognition unit 113 is dragged and dropped. In step S103, for example, when the image message 321-33 is dragged by the pointer 531 and moved to the right in the figure and dropped as in the message board 301-23 in FIG. 13, the process proceeds to step S104. .

  In step S104, the movement operation processing unit 151 displays the image message 321-34 at the position where the movement is completed and dropped, as indicated by the message board 301-24 in FIG. That is, by such processing, the image message 321 can be freely moved on the message board 301 and displayed.

  If no drag and drop is performed in step S103, the process in step S104 is skipped.

  In step S105, the enlargement / reduction operation processing unit 152 of the message display management unit 114 drags the image message 321 selected by the pointer 531 by the operation content recognition unit 113 at two points, and the distance between the two points is increased. It is determined whether the image has been enlarged or reduced. In step S105, for example, when the pointers 531-1 and 531-2 are dragged at two points and the distance between the two points is expanded as in the message board 301-25 in FIG. 13, the process proceeds to step S106. move on.

  In step S106, the enlargement / reduction operation processing unit 152 enlarges the size according to the distance between the two points as shown by the image message 321-36 as shown by the message board 301-26 in FIG. indicate. When the operation is performed so that the distance between the two points is reduced, the image message 321-36 is displayed so that the size is reduced according to the distance between the two points. That is, by such processing, the image message 321 can be displayed on the message board 301 with the size freely changed.

  If it is not dragged at two points or the distance between the two points is neither enlarged nor reduced in step S105, the process of step S106 is skipped.

  In step S107, the rotation operation processing unit 153 of the message display management unit 114 causes the image message 321 selected by the pointer 531 by the operation content recognition unit 113 to be dragged at one point so as to draw an arc around a predetermined position. It is determined whether or not a rotation operation has been performed. That is, whether or not the image message 321 selected by the pointer 531 has been dragged at one point on the outer periphery of the display area of the image message 321 and rotated to draw an arc around the center of the display area of the image message 321. Is determined. In step S107, for example, as in the message board 301-37 in FIG. 13, when the pointer 531 is dragged at one point and an operation is performed to draw an arc around a predetermined position, the process proceeds to step S108. move on.

  In step S108, the rotation operation processing unit 153 rotates and displays the image message 321 according to the rotation angle of the dragged position, as indicated by the image message 321-38 in FIG. That is, by such processing, the image message 321 can be displayed on the message board 301 by freely changing the rotation angle.

  If it is dragged at one point and is not rotated so as to draw an arc in step S107, the process of step S108 is skipped.

  In step S109, the rotation operation processing unit 153 of the message display management unit 114 causes the operation content recognition unit 113 to drag the image message 321 at two points, and the distance between the two points is the diameter around the center position between the two points. It is determined whether or not the rotation operation is performed so that That is, it is determined whether or not a rotation operation has been performed so that the distance between the two points becomes the diameter with the center position between the two points as the axis by dragging at the two outer peripheral portions of the display area of the image message 321. . In step S109, for example, the message board 301-29 in FIG. 13 is dragged at two points by the pointers 531-1 and 531-2, and the distance between the two points is the diameter around the center position between the two points. If a rotation operation is performed, the process proceeds to step S110.

  In step S110, the rotation operation processing unit 153 rotates and displays the image message 321-39 according to the rotation angle of the dragged position, as indicated by the image message 321-40 in FIG. That is, by such processing, the image message 321 can be displayed on the message board 301 by freely changing the rotation angle.

  In step S109, when the drag operation is performed at two points and the rotation operation is not performed so as to draw an arc so that the distance between the two points is a diameter with the center position between the two points as an axis, the process of step S110 is performed. Processing is skipped.

  That is, by the above processing, in an electronic device employing the touch panel 102, the image message displayed on the touch panel 102 can be moved, enlarged, or reduced in the contact direction by simply touching the image message with the fingertip via the operation input unit 18. Or can be rotated. Note that the above-described image message operations may be performed by combining the operations, for example, may be enlarged or reduced while rotating.

  Now, the description returns to the flowchart of FIG.

  If no image message display operation is performed in step S8, the process in step S9 is skipped.

  In step S10, the moving image reproduction processing unit 171 of the reproduction processing unit 115 determines whether or not the operation content recognition unit 113 has instructed reproduction by pressing the reproduction button of the image message 321 with a pointer (not shown). In step S10, for example, if it is determined that the playback button of the image message 321 is pressed and playback is instructed, the process proceeds to step S11, and message playback processing is executed.

[Message playback process]
Here, the message reproduction process will be described with reference to the flowchart of FIG.

  In step S121, when the playback button is operated like the pointer 531-51 in the image message 321-51 of FIG. 15, the moving image playback processing unit 171 stores the message data 202 corresponding to the image message 321-51 in the storage unit. 22 to read.

  In step S122, the moving image reproduction processing unit 171 reproduces and displays the read moving image data in the image message 321 to 51 as shown in the lower part of FIG. At this time, as shown in the lower part of FIG. 15, the moving image reproduction processing unit 171 also displays a pause button in which two vertical bars are arranged horizontally.

  In step S123, the moving image reproduction processing unit 171 determines whether the operation content recognition unit 113 has pressed the pause button with a pointer (not shown) to instruct the pause of reproduction.

  In step S123, when the pause button is pressed, in step S124, the moving image reproduction processing unit 171 displays an image message 321-51 of a still image including a frame image at the time of reproduction stop, which is shown in the upper part of FIG. Display a play button as well.

  In step S125, the moving image playback processing unit 171 determines whether or not the playback button has been pressed. If the play button is not pressed in step S125, the process proceeds to step S128. In step S128, the moving image reproduction processing unit 171 determines whether or not an instruction to end reproduction of the moving image has been issued. In step S128, for example, when an instruction to end reproduction of the moving image is given, the process proceeds to step S127. On the other hand, when it is not instructed to end the reproduction of the moving image in step S128, the process returns to step S124. That is, the processes of steps S124, S125, and S128 are repeated until the end of the reproduction of the moving image is instructed or the reproduction button is pressed. If reproduction is instructed in step S125, the process proceeds to step S126.

  In step S126, the moving image reproduction processing unit 171 determines whether or not the end of reproduction of the moving image has been instructed or whether or not reproduction of the moving image has ended. In step S126, when there is no instruction to end the reproduction of the moving image and the reproduction of the moving image has not ended, the process returns to step S122. That is, the processes in steps S122 to S126 are repeated until the end of playback is instructed or until playback ends. In step S126, when playback is instructed or when playback ends, in step S127, the video playback processing unit 171 displays the image message 321 using the final frame of the video data and ends. To do.

  With the above processing, it is possible to reproduce, pause, or terminate the image message composed of moving image data. In the upper part of FIG. 15, the message boards 301-51 displayed in a state where the image messages 321-51 to 321-54 until the reproduction is instructed are operated by the pointers 531-51 to 531-54, respectively. It is shown. In the lower part of FIG. 15, a message board 301-52 on which the image messages 321-51 to 321-54 being reproduced are displayed is shown. In this way, a plurality of image messages may be reproduced simultaneously. Further, although the example using the last frame has been described for the image message 321 after the end, the image message 321 may be displayed using other frames.

  Now, the description returns to the flowchart of FIG.

  If reproduction is not instructed in step S10, the process in step S11 is skipped.

  In step S12, the message selection recognizing unit 192 of the mail transmission managing unit 116 drags any image message by the operation content recognizing unit 113 and drops it at the position of the icon 332, and transmits it as an attachment to the e-mail. It is determined whether or not an instruction has been given. For example, if the image message 321-3 is dragged by the pointer 531 and dropped at the position of the icon 332 as indicated by the arrow of the message board 301-61 in FIG. Then, the process proceeds to step S13.

  In step S13, the mail transmission management unit 116 executes a message transmission process, and transmits an e-mail with the dropped image message attached.

[Message transmission processing]
Here, the message transmission process will be described with reference to the flowchart of FIG.

  In step S141, the mail setting information registration unit 191 of the mail transmission management unit 116 reads the mail setting information from the storage unit 22, and determines whether the transmission destination address is unregistered. In step S141, for example, when the transmission destination address is unregistered, the process proceeds to step S144.

  In step S144, the mail setting information registration unit 191 displays a setting window 701, for example, as shown by the message board 301-63 in FIG. A setting window 701 in FIG. 16 is provided with a profile name column 711, a destination address column 712, a subject column 713, a content column 714, an address column 715, an account column 716, and a password column 717 from the top. Below that, a button 721 that is operated when saving as mail setting information with a profile name, a button 722 that is pressed when overwriting, and a button that is operated when canceling registration of mail setting information 723 is provided.

  The profile name column 711 is a column for displaying the profile name of the mail setting information. When the inverted triangle button 711a provided on the right side is pressed, a plurality of registered profile names are displayed as a drop-down list. And can be selected by the pointer. When the profile name is selected, the mail setting information registered in association with it is read, and the information in the transmission destination address column 712 to password column 717 is switched. The profile name is input when newly registered, and is used for identifying mail setting information.

  The transmission destination address column 712 is a column in which an address for specifying the transmission destination of the electronic mail is input. In the subject field 713, the subject of the e-mail to be transmitted is input. In the content column 714, the text of the e-mail to be transmitted is input. In the address field 715, an email address of the user of the image processing apparatus 1 serving as a transmission source is input. In the account column 716, for example, an account name of an Internet service provider for which the user of the image processing apparatus 1 as a transmission source receives service is input. In the password field 717, a password corresponding to the account name is input. In the profile name column 711 to the password column 717, when no mail setting information is registered, all are displayed in a blank state as shown by the message board 301-63 in FIG. If any of the mail setting information is registered, the contents of the mail setting information registered in association with the most recently set profile name are displayed in the profile name column 711 to the password column 717. Is done.

  In step S145, the mail editing processing unit 193 causes the operation content recognition unit 113 to operate the operation input unit 18 of the touch panel 102, the button 711a is pressed, a drop-down list is displayed, and any profile name is displayed. It is determined whether or not it has been selected. If it is determined in step S145 that the button 711a has not been pressed and no profile name has been selected, the process proceeds to step S146.

  In step S146, the mail editing processing unit 193 determines whether or not the operation setting unit 18 of the touch panel 102 is operated by the operation content recognition unit 113 and mail setting information is input to the transmission destination address field 712 to the password field 717. judge. In step S146, for example, when mail setting information is input in the transmission destination address field 712 to password field 717, the process proceeds to step S147.

  In step S147, the mail editing processing unit 193 causes the input mail setting information to be displayed on the display unit 21 of the touch panel 102 as indicated by, for example, the message board 301-64 in FIG. In the message board 301-64 of FIG. 16, “aaa@ee.com” is input in the destination address column 712, “ccc” is input in the subject column 713, and the electronic column is input in the content column 714. It has been entered as "Hello" as the body of the e-mail. Also, “ddd@ee.com” is entered in the address field 715 as the address of the user who is the sender, “ddd” is entered in the account field 716, and the entered characters are entered in the password field 717. Is displayed as "********" so that it cannot be seen by a third party.

  If it is determined in step S146 that no mail setting information has been input, the process in step S147 is skipped.

  In step S <b> 148, whether the mail setting information registration unit 191 has been instructed by the operation content recognition unit 113 to operate the operation input unit 18 of the touch panel 102 to save the input mail setting information with a name. Determine whether or not. In step S148, for example, when the button 721 of the setting window 701 is pressed by the pointer 531 as shown by the message board 301-64 in FIG. The process proceeds to step S149.

  In step S149, the mail setting information registration unit 191 controls the display unit 21 to display a profile name input window 741 as shown by the message board 301-65 in FIG. The profile name input window 741 is provided with a profile name input field 751, below which a button 761 operated when instructing the completion of input and a button 762 operated when canceling the input of the profile name are displayed. Is provided.

  In step S150, the mail editing processing unit 193 determines whether or not the profile name has been input by the operation content recognition unit 113 by causing the operation input unit 18 to function as, for example, a software keyboard. In step S150, for example, when a profile name is input, in step S151, the mail editing processing unit 193 controls the display unit 21, and for example, the input profile name is displayed in the profile name input field 751 in FIG. Display as shown. In the profile name input field 751 in the message board 301-65 in FIG. 16, “AAA” is displayed, indicating that “AAA” has been input as the profile name. If it is determined in step S150 that no profile name has been input, the process in step S151 is skipped.

  In step S152, the mail setting information registration unit 191 determines whether the operation content recognition unit 113 has operated the operation input unit 18 to press the button 761 labeled “OK” to instruct the completion of the input of the profile name. Determine whether or not. In step S151, for example, as shown in the profile name input window 741 in FIG. 16, when the button 761 is pressed by the pointer 531, it is considered that the input of the profile name has been instructed, and the process proceeds to step S153. move on.

  In step S153, the mail setting information registration unit 191 associates the information input in the setting window 701 with the profile name in the profile name input field 751 of the current profile name input window 741, and stores the mail setting information 203 in the storage unit 22. Register as

  In step S154, the mail transmission processing unit 194 reads out the mail setting information 203 stored in the storage unit 22, and the message data 202 corresponding to the image message 321 dropped at the position of the icon 332 in the process of step S12. read out. Further, the mail transmission processing unit 194 creates an electronic mail based on the mail setting information 203, and then controls the communication unit 23 to attach the read message data 202 and transmit it to the transmission destination address. At this time, the mail transmission processing unit 194 executes an authentication process on the server of the Internet service provider based on the account name and password included in the mail setting information 203, and then creates the created e-mail. Send an image message attached.

In step S155, the mail transmission processing unit 194 controls the display unit 21 of the touch panel 102 to display an e-mail transmission confirmation window 791, for example, as shown by the message board 301-66 in FIG. In the transmission confirmation window 791 of FIG. 16, “E-mail has been sent” is written, indicating that the transmission of the e-mail has been completed. The transmission confirmation window 791 is provided with a button 792 labeled “OK” that is operated to indicate confirmation.

  In step S156, the mail transmission processing unit 194 determines whether or not the operation content recognition unit 113 has operated the operation input unit 18 to perform processing indicating that the user has confirmed transmission confirmation (a button indicating “OK”). Whether or not 792 has been pressed is determined. If it is not indicated in step S156 that the user has confirmed the transmission confirmation, the process returns to step S155. That is, the processing of steps S155 and S156 is repeated and the transmission confirmation window 791 is continuously displayed until it is shown that the user confirms the transmission confirmation.

  In step S156, for example, when the button 792 shown on the message board 301-66 in FIG. 16 is pressed by a pointer (not shown), it is considered that the transmission confirmation is shown by the user, and the processing is performed. Ends.

  On the other hand, if the button 761 labeled “OK” is not pressed in step S152, the process proceeds to step S157.

  In step S157, the mail setting information registration unit 191 determines whether or not the operation content recognition unit 113 has operated the operation input unit 18 to press the button 762 to instruct cancellation. In step S157, when the button 762 is not pressed and cancellation is not instructed, the process returns to step S149. That is, steps S149 to S152 and S157 are repeated, and the profile name input window 741 is continuously displayed.

  In step S157, when the button 762 is pressed and cancellation is instructed, in step S158, the mail setting information registration unit 191 closes the display of the profile name input window 741, and the process returns to step S147. . That is, it is considered that the registration of the profile name has been canceled by pressing the button 762, and the processing returns to the state before the instruction to save with the profile name is given.

  Furthermore, if it is not instructed to name and save in step S148, the process proceeds to step S159. In step S159, the mail setting information registration unit 191 determines whether or not the operation content recognition unit 113 has operated the operation input unit 18 to press the button 722 and instructed overwriting of the mail setting information. judge. In step S159, for example, when the button 722 is operated and overwriting is instructed, the process proceeds to step S162.

  In step S162, the mail setting information registration unit 191 associates the information input in the setting window 701 with the registered profile name, and overwrites and saves the information as the mail setting information 203 in the storage unit 22, and the processing is performed in step S154. Proceed to Note that since it cannot be overwritten and saved in association with an unregistered profile name, processing is performed so that new mail setting information cannot be overwritten.

  On the other hand, in step S159, when the button 722 is not operated and overwriting is not instructed, the process proceeds to step S160. In step S160, the mail setting information registration unit 191 cancels the transmission processing of the e-mail attached with the image message when the operation input unit 18 is operated by the operation content recognition unit 113 and the button 723 is pressed. It is determined whether or not an instruction has been given. In step S160, when an instruction to cancel the button 723 is issued, the mail setting information registration unit 191 closes the display of the setting window 701 and ends the message transmission process in step S161. That is, in this case, since the mail setting information is not registered, the mail transmission process is terminated because the mail cannot be transmitted.

  In step S160, when the button 723 is not pressed and cancellation is not instructed, the process returns to step S144. That is, the setting window 701 continues to be displayed, and input of mail setting information is accepted.

  Furthermore, since the mail setting information 203 is registered once the above processing is performed, the transmission destination is registered. Henceforth, in the processing of step S141, the transmission destination is not registered. The process proceeds to step S142.

  In step S142, the mail setting information registration unit 191 reads the mail setting information 203 stored in the storage unit 22, and confirms the setting based on the mail setting information 203 as shown by the message boards 301-62 in FIG. A window 771 is displayed. In the setting confirmation window 771, information registered in the mail setting information 203 set by the series of processes described above is displayed. That is, in the setting confirmation window 771, “aaa@ee.com” is displayed as the destination address from the top, “ccc” is displayed as the subject below, and further below the e-mail address. it is listed as "Hello" as the content that is the body. Thus, by displaying the setting confirmation window 771, the user can confirm the contents registered in the mail setting information.

  In addition, a button 781 indicating “E-mail setting” which is pressed when resetting the mail setting information is provided at the bottom of the setting confirmation window 771. Further, on the right side of the button 781, a button 782 labeled “OK” that is pressed when instructing transmission is provided. Further, on the right side, a button 783 labeled “CANCEL” that is pressed when canceling message transmission is provided.

  In step S143, the mail setting information registration unit 191 determines whether or not the operation content recognition unit 113 has operated the operation input unit 18 to press the button 781 to instruct re-setting of the mail setting information. In step S143, for example, when the button 781 is pressed with a pointer (not shown), it is considered that the resetting of the mail setting information is instructed, and the process proceeds to step S144.

  On the other hand, if it is determined in step S143 that the button 781 has not been pressed and resetting of the mail setting information has not been instructed, the process proceeds to step S163.

In step S <b> 163, the mail transmission processing unit 194 determines whether or not the operation content recognition unit 113 has pressed the button 782 displayed as “OK” to instruct mail transmission. In step S163 , for example, when the button 782 is pressed to instruct transmission of mail, the process proceeds to step S154. That is, an e-mail attached with the selected image message is transmitted with the contents registered in the current mail setting information 203.

  On the other hand, if the button 782 is not pressed in step S163, that is, if mail transmission is not instructed, the process proceeds to step S160.

  Furthermore, in step S145, when the button 711a is pressed to display a drop-down list, and any profile name is selected, and when the button 722 is pressed to instruct overwriting, the process is performed. The process proceeds to step S164.

  In step S164, the mail setting information registration unit 191 reads the mail setting information 203 registered in association with the selected profile name from the mail setting information 203 in the storage unit 22, and uses the read information to set the setting confirmation window. The information in 771 is switched. Then, the process returns to step S143.

  The above processing is summarized as follows. For example, when the image message 321-3 is dropped at the position of the icon 332 as in the message board 301-61 in FIG. 16, if the mail setting information is not registered, the setting window 701 is displayed as in the message board 301-63. Will prompt you to register your email settings information. When the destination address field 712 to password field 717 in the setting window 701 are input as shown in the message board 301-64 of FIG. 16 and the button 721 is pressed, the profile name is input as shown in the message board 301-65. A window 741 is displayed.

  Further, when a profile name is input in the profile name input field 751 of the profile name input window 741 of the message board 301-65, new mail setting information is registered in the storage unit 22 in association with the profile name. At this time, message data registered in association with the image message 321-3 is read out and attached to an e-mail generated based on the new mail setting information. As a result, a transmission confirmation window 791 is displayed as shown by the message boards 301-66.

  As described above, when the mail setting information is not registered, the process is completed by sequentially shifting from the display state of the message board 301-61 in FIG. 16 to the state of the message boards 301-63 to 301-66.

  On the other hand, when the mail setting information has already been registered, for example, when the image message 321-3 is dropped at the position of the icon 332 as in the message board 301-61 in FIG. A setting confirmation window 771 is displayed. With this process, the contents of the mail setting information now are presented to the user. Furthermore, when the mail setting information may be transmitted, when the button 782 is pressed, the message data 202 registered in association with the image message 321-3 is read out, and is based on the mail setting information. Attached to the generated e-mail and sent. As a result, a transmission confirmation window 791 is displayed as shown by the message boards 301-66.

  That is, once the mail setting information is registered, it is possible to send an e-mail attached with an image message simply by dragging the image message on the message board 301 and dropping it on the icon 332. . For this reason, it is possible to omit processing that requires key input, such as a destination address, a subject, and a text. As a result, even when the input function by the user is limited to the touch panel 102, such as the image processing apparatus 1, the key input can be omitted, so that it is possible to easily transmit an e-mail attached with an image message. Become.

  Now, the description returns to the flowchart of FIG.

  If it is determined in step S12 that transmission of an e-mail attached with an image message is not instructed, the message transmission process in step S13 is skipped.

  In step S12, an example in which one image message 321 is dragged and dropped onto the icon 332 has been described for an instruction to send an e-mail attached with an image message. It may be. That is, in the case of the touch panel 102, as shown by the message board 301-71 in FIG. May be. In the message board 301-71 of FIG. 18, a plurality of pointers 531-1 to 531-3 are set to each of the image messages 321-1 to 321-3 by multi-touch, whereby a plurality of image messages 321-1. To 321-3 are dragged. Further, as indicated by the arrows, the pointers 531-1 to 531-3 are moved to the position of the icon 332, and the image messages 321-1 to 321-3 are dropped. As described above, it is assumed that the transmission of the e-mail attached with the image messages 321-1 to 321-3 is instructed, and the image messages 321-1 to 321-3 are attached by the same processing as when the image message 321 is dropped. You may make it transmit the e-mail which was made.

  In addition, even an electronic device that employs the touch panel 102 may not be provided with a multi-touch function. In such a case, as shown in the lower part of FIG. 18, a plurality of image messages 321 may be dropped into one folder, and this folder may be dragged and dropped onto the icon 332. That is, in the message board 301-72 of FIG. 18, the image message 321-1 is dragged by the pointer 531-1 and dropped into a folder 795 provided for newly collecting a plurality of image messages. Thereafter, in the same manner, the image message 321-2 is dragged with the pointer 531-2 and dropped into the folder 795, and the image message 321-3 is dragged with the pointer 531-3 and dropped into the folder 795. . As a result of this processing, the folder 795 stores the image messages 321-1 to 321-3. Then, the folder 795 is dragged by the pointer 531-4 and dropped on the icon 332. By this processing, an instruction to send an e-mail attached with the image messages 321-1 to 321-3 is given, and an e-mail attached with the image messages 321-1 to 321-3 is processed by the same processing as described above. May be transmitted.

  In step S14, the trash box management unit 132 of the deletion management unit 125 determines whether any image message 321 is dragged, dropped on the icon 333, and put in the trash box. In step S14, for example, as shown by the message board 301-81 in FIG. 19, when the image message 321-3 is dragged by the pointer 531 and dropped on the icon 333, it is regarded as being put in the trash box. The process proceeds to step S15.

  In step S <b> 15, the trash can management unit 132 registers the image message 321-3 as the trash can data 204 and updates the information of the message data 202 stored in the storage unit 22. Then, the trash can management unit 132 deletes the display of the image message 321-3 as indicated by the message board 301-82 in FIG. In other words, the image message 321-3 pasted on the message board 301 is peeled off, and the display is changed to an image displayed in a trash can. In this case, however, the still image or moving image information corresponding to the image message 321-3 is left in the message data 202, and the still image or moving image message is registered in the trash can data 204 for recording. It will be in a state to be. As for the image message to be dragged to the icon 333, a plurality of image messages may be simultaneously dropped on the icon 333 as in the message transmission process.

  If any image message 321 is dragged and not dropped on the icon 333 in step S14, the process in step S15 is skipped.

  In step S <b> 16, the trash box management unit 132 manages (organizes) image messages registered as trash box data when the operation input unit 18 is operated by the operation content recognition unit 113 and the icon 333 indicating the trash box is pressed. Whether or not is instructed is determined. In step S16, for example, as indicated by the message board 301-82 in FIG. 19, when the pointer 531 is moved to the position of the icon 333 and pressed, and management (organization) of the trash box data 204 is instructed, processing is performed. Advances to step S17.

  In step S17, the trash can management unit 132 executes a trash can management process.

[Trash management]
Here, the trash can management process will be described with reference to the flowchart of FIG.

  In step S <b> 181, the trash can management unit 132 accesses the message data 202 of the storage unit 22 and reads an image message registered in the trash box data 204.

In step S182, the trash can management unit 132 displays a trash can window 801 as a list of the read image messages 321 as shown by message boards 301-83 in FIG. 19, for example. In the trash box window 801, an image message 321-3 registered in the trash box data 204 is displayed. Also, the trash can window 801 is provided with a button 811 that is operated when all image messages 321 in the trash can window 801 are selected. In addition, a button 812 that is operated when it is desired to restore the selected image message 321 to the original message board 301 and a button 813 that is operated when it is desired to completely delete the selected image message 321 are provided. Yes. The buttons 812 and 813 are labeled “ restore ” and “ delete ”, respectively. Also, a button 814 that is operated when the user wants to exit is provided at the upper right of the trash can window 801.

  In step S183, the trash can management unit 132 determines whether any image message 321 has been selected by a pointer (not shown) or the button 811. For example, when the image message 321-3 shown in FIG. 19 is selected in step S183, the process proceeds to, for example, a bold line so as to indicate that the selected image message 321-3 is selected in step S184. Select and display it by surrounding it with a frame.

  In step S185, the trash box management unit 132 determines whether or not the operation content recognition unit 113 has operated the operation input unit 18 to press the button 813 and delete the image message registered as trash box data. Determine whether. In step S185, for example, as shown by the message board 301-83 in FIG. 19, when the button 813 is pressed by the pointer 531 and deletion is selected, the process proceeds to step S186.

In step S186, the trash box management unit 132 deletes and updates the data of the selected image message 321 from the trash box data 204, and displays the trash box window 801 as shown by the message board 301-85 in FIG. Erase. As a result of this process, the information of the still image or moving image corresponding to the image message 321 selected to be deleted is deleted from the storage unit 22 and cannot be restored. That is, in this case, the image message 321-3 is completely deleted so that it cannot be restored.

  On the other hand, if deletion of the image message registered as the trash box data 204 is not selected in step S185, the process proceeds to step S187. In step S187, the restoration processing unit 131 determines whether restoration of an image message registered in the trash box data 204 is selected. In step S187, for example, when the button 812 is pressed by the pointer 531 and the restoration of the image message registered in the trash box data 204 is selected as in the message board 301-84 in FIG. The process proceeds to step S188.

  In step S188, the restoration processing unit 131 deletes the image message 321 registered in the selected trash box data 204 from the trash box data 204 and re-registers it in the message data 202. Further, the restoration processing unit 131 displays the restored image message 321-3 on the message board 301 as shown by the message board 301-86 in FIG.

  By this processing, even if the image message 321 is once deleted from the message board 301, it can be restored to the original state when it is desired to restore it again. As a result, management of image messages on the message board 301 can be facilitated.

  If no image message is selected in step S183, or if restoration of the image message registered in the trash box data is not selected in step S187, the process proceeds to step S189.

  In step S189, the trash can management unit 132 determines whether or not the operation content recognition unit 113 has operated the operation input unit 18 to press the button 814 to instruct the end of the trash can management process. If the end of the trash can management process is not instructed in step S189, the process returns to step S182. In other words, the processing in steps S182 to S189 is repeated until the deletion, restoration, or termination instruction is given, and the trash box window 801 is continuously displayed. In step S189, when the button 814 is pressed and an end instruction is given, the trash box management process ends.

  With the above processing, management of the image message 321 on the main board can be realized as if the photograph is physically managed on the bulletin board. In addition, unless complete deletion processing is performed, it is possible to restore an image message that has been registered in the trash box data. Can be used. In the above description, an example in which one image message 321 is selected and deleted or restored has been described. However, a plurality of image messages 321 can be selected and deleted or restored at the same time.

  Now, the description returns to the flowchart of FIG.

  In step S16, if the trash management process is not instructed, the process in step S17 is skipped.

  In step S <b> 18, the message display management unit 114 determines whether or not the operation content recognition unit 113 has operated the operation input unit 18 to instruct termination of the message management process. In step S18, for example, when the button 314 in FIG. 4 is pressed by an unillustrated pointer to instruct termination, the message management process ends. On the other hand, if the end of the message management process is not instructed in step S18, the process returns to step S3, and the subsequent processes are repeated.

  Through the above processing, even an electronic device having only an input function such as a touch panel can easily create an image message from a still image or a moving image captured in real time. Further, even an electronic device having only an input function, such as a touch panel, can easily transmit an electronic mail with an image message attached. Furthermore, with these functions, even in electronic devices where key input operations such as touch panels are difficult, it is possible to easily and quickly send e-mails attached with image messages consisting of still images and moving images captured in real time. It becomes possible.

  In addition, the user can easily generate and edit image messages while checking the images of himself / herself and the surrounding people in real time while enjoying the operation of the touch panel 102 with the surrounding people as well as the user. It becomes. Further, since the generated image messages are sequentially posted on the message board 301, it is possible to enjoy watching the posted image messages. Furthermore, it is possible to manage the image message as if the message board 301 displayed on the touch panel 102 is managed by pasting a photograph on a real bulletin board. In addition, it is possible to easily send a favorite image message by dragging and dropping it on the touch panel 102 at a position where a button for instructing to send an e-mail is displayed, as if using a real bulletin board. Become. Further, when the same message board 301 is shared by a plurality of persons, for example, by sharing the message board 301 with each other, image messages created by each family can be shown on the message board 301 for communication. It becomes possible.

  By the way, the above-described series of processing can be executed by hardware or can be executed by software. When a series of processing is executed by software, a program constituting the software is installed in the computer. Here, the computer includes, for example, a general-purpose personal computer capable of executing various functions by installing various programs by installing a computer incorporated in dedicated hardware.

  That is, the image processing apparatus 1 in FIG. 1 may be configured by a general-purpose personal computer.

  In the personal computer configured as described above, the CPU 11 loads, for example, a program stored in the storage unit 22 to the RAM 13 via the input / output interface 15 and the bus 14 and executes the program. Is performed.

  The program executed by the computer (CPU 11) can be provided by being recorded on the removable medium 25 as a package medium, for example. The program can be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.

  In the computer, the program can be installed in the storage unit 22 via the input / output interface 15 by attaching the removable medium 25 to the drive 24. The program can be received by the communication unit 23 via a wired or wireless transmission medium and installed in the storage unit 22. In addition, the program can be installed in the ROM 12 or the storage unit 22 in advance.

  In this specification, the step of describing the program recorded on the recording medium is not limited to the processing performed in time series in the order described, but of course, it is not necessarily performed in time series. Or the process performed separately is included.

  DESCRIPTION OF SYMBOLS 1 Image processing apparatus, 17 Imaging part, 18 Operation input part, 21 Display part, 22 Storage part, 101 Control part, 102 Touch panel, 111 New message registration management part, 112 Setting information management part, 113 Operation content recognition part, 114 Message Display management unit, 115 playback processing unit, 116 mail transmission management unit, 121 captured image display control unit, 122 message editing processing unit, 123 still image message generation unit, 124 video message generation unit, 125 deletion processing unit, 131 restoration processing unit , 132 Trash management section, 191 mail setting information registration section, 192 message selection recognition section, 193 mail editing processing section, 194 mail transmission processing section

Claims (11)

In an image processing apparatus having a display unit and an imaging unit,
An operation means for generating an operation signal by a user's contact with the display unit;
A bulletin board display control means for displaying a bulletin board for posting an image message on the display unit and displaying a pre-generated image message on the bulletin board;
A captured image display control unit that displays an image captured by the imaging unit on the bulletin board by operating an imaging mode transition operator displayed in the bulletin board;
When an operation signal for instructing the start of recording and the end of recording of the image captured by the imaging unit is generated by the operation means, the start of the recording is instructed among the images captured by the imaging unit. Image message generating means for generating an image message consisting of a moving image from the recorded timing to the timing when the end of recording is instructed ;
A moving image reproduction means for displaying a reproduction button superimposed on the image message, and reproducing a moving image corresponding to the image message when it is recognized that the reproduction button is operated based on the operation signal. including the door,
The bulletin board display control means displays the image message newly generated by the image message generation means on the bulletin board in addition to the previously generated image message ,
Based on the operation signal, when it is recognized that the image message is dragged at two points and the distance of the dragged position is enlarged or reduced, in response to the recognized enlargement or reduction, The image message is enlarged or reduced and displayed.
Based on the operation signal, when it is recognized that the image message is dragged at one point and rotated to draw an arc around a predetermined position, the image message is rotated according to the rotation operation. Image processing apparatus to be displayed .   The moving image playback means displays the playback button for each of the plurality of image messages, and when it is recognized that the playback button has been pressed based on the operation signal, Each of the corresponding videos is played back and displayed, and a pause button is superimposed and displayed.When it is recognized that the pause button has been pressed, the video being played is paused and displayed, Display the playback button superimposed
  The image processing apparatus according to claim 1. The image processing apparatus according to claim 1, wherein the image message generation unit performs predetermined editing on the displayed image and generates an image message attached to an e-mail. The image processing according to claim 1, wherein the image message generation unit generates an image message including a still image when an operation signal instructing capturing of an image captured by the imaging unit is generated by the operation unit. apparatus. The image message generating means includes at least an animation image, a stamp image, a watercolor pen-like paint, a pen-like paint, and a frame on the image captured by the imaging unit based on an operation signal generated by the operation means. The image message is generated from the edited image captured by the imaging unit, based on an operation signal generated by the operation unit, by adding or editing an illustration by a process including the processing. Image processing device. 2. The image according to claim 1, wherein the bulletin board display control unit displays the image messages generated by the image message generation unit sequentially at random positions in a random size so as to overlap with the generated time series. Processing equipment. The bulletin board display control means, when one of the plurality of displayed image messages is selected based on an operation signal corresponding to an operation by the user's contact, the selected image message is The image processing apparatus according to claim 6. The bulletin board display control means displays a corresponding image message on the display unit when any drag of the displayed image message is recognized based on an operation signal corresponding to an operation by the user's contact. The image processing apparatus according to claim 1, wherein the image processing apparatus is moved and displayed in accordance with the movement in the plane. Based on an operation signal corresponding to an operation by the user's contact, any drag of the displayed image message is recognized by the operation means and moved to a predetermined position in the display surface of the display unit. The image processing apparatus according to claim 8, further comprising: a transmission unit that, when dropped, attaches a corresponding image message and transmits an e-mail. In an image processing method of an image processing apparatus having a display unit and an imaging unit,
An operation step of generating an operation signal due to a user's contact with the display unit;
A bulletin board display control step for displaying a bulletin board for posting an image message on the display unit, and displaying a pre-generated image message on the bulletin board;
A captured image display control step of displaying an image captured by the imaging unit on the bulletin board by operating the imaging mode transition operator displayed in the bulletin board;
When an operation signal instructing the start of recording of the image captured by the imaging unit and the end of recording is generated by the processing of the operation step, the start of recording of the image captured by the imaging unit is started. An image message generating step for generating an image message composed of a video from a timing instructed to a timing instructed to end the recording ;
A video playback step of displaying a playback button superimposed on the image message, and playing back a video corresponding to the image message when it is recognized that the playback button has been operated based on the operation signal. including the door,
In the processing of the bulletin board display control step, the image message newly generated by the processing of the image message generation step is displayed on the bulletin board in addition to the image message generated in advance .
Based on the operation signal, when it is recognized that the image message is dragged at two points and the distance of the dragged position is enlarged or reduced, in response to the recognized enlargement or reduction, Displaying the image message in an enlarged or reduced manner,
Based on the operation signal, when it is recognized that the image message is dragged at one point and rotated to draw an arc around a predetermined position, the image message is rotated according to the rotation operation. image processing method for to display. In a computer that controls an image processing apparatus having a display unit and an imaging unit,
An operation step of generating an operation signal due to a user's contact with the display unit;
A bulletin board display control step for displaying a bulletin board for posting an image message on the display unit, and displaying a pre-generated image message on the bulletin board;
A captured image display control step of displaying an image captured by the imaging unit on the bulletin board by operating the imaging mode transition operator displayed in the bulletin board;
When an operation signal instructing the start of recording of the image captured by the imaging unit and the end of recording is generated by the processing of the operation step, the start of recording of the image captured by the imaging unit is started. An image message generating step for generating an image message composed of a video from a timing instructed to a timing instructed to end the recording ;
A video playback step of displaying a playback button superimposed on the image message, and playing back a video corresponding to the image message when it is recognized that the playback button has been operated based on the operation signal. to execute the processing including bets,
In the processing of the bulletin board display control step, the image message newly generated by the processing of the image message generation step is displayed on the bulletin board in addition to the image message generated in advance .
Based on the operation signal, when it is recognized that the image message is dragged at two points and the distance of the dragged position is enlarged or reduced, in response to the recognized enlargement or reduction, Displaying the image message in an enlarged or reduced manner,
Based on the operation signal, when it is recognized that the image message is dragged at one point and rotated to draw an arc around a predetermined position, the image message is rotated according to the rotation operation. Program to display .

Images