JP2006338604A - Communication terminal and image communication method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To perform work processing of an image transmitted by a terminal by simple operations and processing. <P>SOLUTION: When this communication terminal has a photography part, a display part which displays an image photographed by the photography part and a function for transmitting the image photographed by the photography part and displayed on the display part, a range of a color approximated to a display color of a part specified by a specifying operation is extracted from the photographed image by specifying a specific position in the image displayed on the display part, image processing for replacing the extracted range with another prepared image is performed and the image generated by the image processing is transmitted. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a communication terminal device suitable for application to a mobile phone terminal having a so-called videophone function, and an image communication method applied to processing in the terminal device.

  Mobile phone terminals with a built-in shooting function for shooting still images or moving images are widely used. In this case, not only the captured image is simply displayed on the display unit of the terminal, but also the communication using the mobile phone terminal is performed to transmit the captured image to the other party, so that the call can be made while looking at the other party's face. There is one that realizes a so-called videophone function.

  When transmitting an image captured by the terminal to the other party using the videophone function, not only the image captured by the capturing unit is transmitted as it is, but also the image may be transmitted after some processing. For example, a terminal capable of synthesizing a decoration image prepared in advance and transmitting the synthesized image to a partner has been developed. As a transmission to the other party's terminal, in addition to using the videophone function, for example, an image may be sent with an image attached to an e-mail. is there.

Further, as more advanced image processing, for example, as described in Patent Document 1, an image processing for designating a range as a background of a captured image and replacing the designated background range with another image There is disclosure about doing the processing.
JP 2000-261646 A

  However, the operation for performing the image processing as described in Patent Document 1 is a relatively complicated operation, and is applied to the rapid operation at the terminal when making a call using the videophone function. Is impossible. In particular, the operation for setting which region of the image is to be processed is not easy and has a problem.

  In addition, a communication terminal such as a mobile phone terminal has a camera function and an image processing function attached to the original communication function, and is equipped with a very advanced image processing function due to restrictions on cost and circuit mounting space. It is difficult to provide a function for processing a captured image by the above-described image processing.

  The present invention has been made in view of such a point, and an object thereof is to enable processing of an image transmitted from a terminal by a simple operation or process.

  The present invention is displayed on the display unit when it has a photographing unit, a display unit for displaying an image photographed by the photographing unit, and a function for transmitting an image photographed by the photographing unit and displayed on the display unit. By specifying a specific position in the captured image, a color range that approximates the display color specified by the operation is extracted from the photographed image, and the extracted range is extracted from another image prepared in advance. An image process to be replaced with an image is performed, and an image generated by the image process is transmitted.

  By doing so, simply by specifying an arbitrary position in the image, a replacement range is determined from the color of the specified position, and a processing process for replacing the range with another image is performed.

  According to the present invention, advanced image processing for transmission can be performed with simple processing and operation. For example, when a videophone is started, the processing operation can be performed quickly, which is suitable as an image processing function provided in a communication terminal. is there.

  Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings.

  FIG. 1 is a diagram illustrating a configuration example of the mobile phone terminal device 10 of the present example. The cellular phone terminal device 10 of this example includes a radio telephone communication antenna 11 for performing radio communication with a radio telephone base station, and the antenna 11 is connected to a communication circuit 12 for radio telephone communication. Wireless communication with the base station is performed under the control of the control unit 20. When the communication circuit 12 performs communication for a call, the received audio data is supplied to the speaker 14 and output. When outputting from the speaker 14, processing for speaker output is performed by a sound processing unit (not shown). Also, the audio data obtained by the microphone 13 is supplied to the sound communication circuit 12 and transmitted. Also at the time of this transmission, an audio processing unit (not shown) performs audio processing using the output of the microphone 13 as audio data for transmission.

  The cellular phone terminal device 10 includes a display unit 40 configured with a liquid crystal display or the like, and an operation unit 16 configured with operation keys or the like. The display unit 40 displays an e-mail message, a screen for accessing the web, and an image taken by the camera unit 32 described later. The operation unit 16 can perform input operations such as telephone numbers and e-mail texts, various mode settings, and camera shooting operations. An operation of image processing to be described later is also displayed on the display unit 16 while being displayed on the display unit 40.

  Each block in the terminal device 10 is connected to the control unit 19 and the like via the control line 23 and is configured to be able to transfer data via the data line 24, and stores necessary data in the memory 15. I am trying to make it. The memory 15 stores not only data necessary for the mobile phone terminal but also data for applications provided in the terminal such as image data. Replacement image data used in image editing, which will be described later, is also stored in the memory 15 in advance. This replacement image data may be prepared in advance as a preset in the terminal. However, for example, an image captured by the camera unit 32 by user operation can be registered as replacement image data, or accessed via the Internet or the like. An arbitrary image may be downloaded and registered. In addition to the memory 15, a hard disk drive device that is a large-capacity data storage means may be provided.

  In addition, the mobile phone terminal 10 of this example includes a camera unit 32 and an imaging control unit 31 that controls imaging with the camera unit 32, and is configured to be able to capture still images and moving images. The captured image data can be stored in the memory 15 and can be sent to the communication circuit 12 to be transmitted to the other party with which the terminal 10 communicates. As for transmission, when images are sent simultaneously during voice calls, moving images are displayed on the other party's terminal to display moving images, videophone transmission processing, and when sending e-mail, In some cases, still image or moving image data is attached.

  With respect to the image data to be transmitted, the terminal of this example can perform image editing (processing) processing under the control of the control unit 20. When the editing process is performed, it is displayed on the display unit 40 and performed based on an operation on the operation unit 16. In the case of this example, the editing is performed by replacing a range of almost the same color as the color displayed at the location designated by the user with another image prepared in advance. The specific editing process will be described later.

  Next, an example of image processing when the mobile phone terminal 10 of this example makes a call using the videophone function will be described with reference to the flowchart of FIG. First, when there is an operation for starting a videophone call (step S11), the user is caused to perform an operation for selecting a call method based on a screen display or the like (step S12). The start of a telephone call by a videophone here includes both a case of starting a call from its own terminal and a case of starting by an incoming call from a partner terminal. Here, as a call method, when it is selected by a user operation that a videophone call is performed without image editing (image processing), the process proceeds to a process of starting the call by the selected process.

  If the user selects to perform image editing as the calling method, the operation for selecting the replacement image as the operation on the mobile phone terminal 10 is the display on the display unit 40 and the operation on the operation unit 16. It is executed (step S13). At the same time, an image stored in advance in the memory 15 (or an image stored in the memory 15 selected at the time of processing) is read from the memory 15 (step S14), sent to the communication circuit 12, and the image (hereinafter referred to as “image”). This image is referred to as image A) instead of the image captured by the camera unit 32 of the terminal itself (step S15). This image A is basically transmitted repeatedly until a composite image is generated (until step S21 described later). The image A may be an image that displays, for example, a text message or the like, “Image editing is in progress. Please wait for a while until the camera image is displayed.” Alternatively, the face image (such as a still image) of the owner of the terminal 10 may be registered in advance, and the registered face image data of the owner may be sent.

  Then, while these images A are being transmitted, the images currently captured by the camera unit 32 are displayed on the display unit 40 (step S16), and when changing the contents of the image processing settings, the user operation Then, the change work is executed (step S17). Next, the color to be replaced is selected using the pointer displayed on the display unit 40 (step S18). For this color selection, a pointer is displayed in the displayed captured image, for example, as in a display example to be described later, and the color at the location indicated by the pointer is selected. Alternatively, a large number of colors may be arranged on the side of the display screen and displayed like a palette, and the color indicated by the pointer may be selected from the displayed colors. If there is a color selection, a color range close to the color of the selected location is calculated (step S19). The details of this range calculation process will be described later. However, when the image data is in the YUV format or RGB format, it can be easily realized by carrying out the process of giving each element signal a numerical value range.

  Then, a range other than the color within the calculated range is extracted from the currently captured image (step S20). The extracted image is overwritten on the arrangement image selected in step S13, and the captured image and the selected image are combined (step S21). The composite image obtained in this way is transmitted to the call partner. (Step S22). Thereafter, it is determined whether there is a change in the composition content (step S23). If there is a change, the processing is repeated from steps S13 and S14. Further, since the photographing by the camera unit 32 is continuously performed at a predetermined frame period, it is determined whether or not the call using the videophone function is finished (step S24). Returning to S20, the processes in steps S20 to S22 are repeatedly executed in the frame period.

  FIG. 3 is a diagram showing a display example at the start of a call on the mobile phone terminal 10 when performing a videophone call by such an operation. In this example, a display example on the terminal 10a used by the caller A and a display example on the terminal 10b used by the caller B are shown. In this example, there is an incoming videophone call from the caller B to the caller B, and when the incoming call operation is performed, it is operated to perform image editing on the terminal 10a. In this information, the display 41a of the caller A (its own image) captured by the camera unit 32 of the terminal 10a is displayed on the display unit 40 of the terminal 10a in a large display, and the image of the caller B is displayed. The display 42a of (an opponent image) is performed in a small display. In the image display 41 a of the caller A, there is a display of the pointer 43 by an arrow or the like, and the display position of the pointer 43 is changed by an operation of a cursor key or the like prepared as the operation unit 16. The display in FIG. 3 is a display at the time of image editing. When the image editing operation is finished and the display is performed on a normal videophone, the partner's image is displayed on the large display 41a, and the own display is displayed on the small display 42a. An image is displayed. However, the own image by the small display 42a is an image after the editing process, and is the same image as that displayed on the partner terminal.

  As a display on the terminal 10b used by the caller B, the display 42b of the caller B image (its own image) taken by the camera unit 32 provided in the terminal 10b is displayed in a small display and sent from the other party. As the large display 41b by the displayed image, since the other terminal is editing, an image set to be transmitted during editing (during color selection) at the other terminal is received and displayed. . When the color selection operation on the partner terminal 10a is completed, an image based on the selection is sent and displayed on the display 41b.

  FIG. 4 shows an example of an image editing operation and an example of a composite image based on the operation. For example, as shown in FIG. 4A, in a state where a camera image is displayed, a specific position in the image is indicated by a pointer (indicated by an arrow), whereby the color at the indicated position is replaced. Specified as a color. When a color is designated, a range other than the color range that matches the designated color is extracted as shown in the upper side of FIG. Then, as shown in the lower side of FIG. 4B, a replacement image is prepared, and the extracted image is pasted on the replacement image as shown in FIG. 4C. Both images are combined.

  FIG. 5 shows another example of the image editing operation and another example of the composite image based on the operation. In the example of FIG. 4, the background is selected, but in the example of FIG. 5, a specific part in the image as the subject is designated with a pointer, and the part having the same color as that part is replaced. This is an example in which an image is processed. That is, as shown in FIG. 5A, in a state where a camera image is displayed, a specific position (here, a hair portion) of a subject in the image is indicated by a pointer (arrow display). When such a color is specified, a range other than the color range that matches the specified color is extracted as shown in the upper side of FIG. Then, as shown in the lower side of FIG. 5B, a replacement image is prepared, and as shown in FIG. 5C, the extracted image is pasted on the replacement image. Thus, both images are combined, and image processing is performed in which a portion substantially the same as the color of the hair pointed by the pointer becomes another image.

  FIG. 6 is a diagram showing an example of setting the range indicated by the pointer and the color range that is actually replaced. The horizontal axis represents the chromaticity coordinate x, the vertical axis represents the color stimulus y, and the visible light range. Is shown. Here, if the display color of the pixel at the point designated by the pointer is α, the range β indicated by a circle centered on α is the color range calculated in step S19 of the flowchart of FIG. A color within this range is a replacement color.

  More specifically, assuming that the data of each pixel of the image is composed of the primary color signals RGB, it is assumed that the pixel values at the point designated by the pointer are R0, G0, and B0, respectively. At this time, when the value of each pixel is R0 ± a, G0 ± a, B0 ± a, it is determined as a replacement color. The value a of [± a] that determines the range may be set freely by a user operation. When the value a is small, a location that almost completely matches the designated color is replaced, and when the value a is large, a color location that approximates the specified color to some extent is replaced.

  As described above, according to the mobile phone terminal of this example, since the range of colors to be replaced from the image captured by the camera unit to another image is selected, the image is captured by the camera at the time of the video phone call with an intuitive and easy-to-understand operation. A part of the image can be replaced. For example, as in the example shown in FIG. 4, when a part of the background is to be hidden, by designating the color of that part, the process of replacing the background to be hidden can be performed quickly, and the user can easily Can be operated. In addition, the processing process is relatively simple, the burden on the control unit 20 and the memory 15 that controls image processing is small, and the load on the terminal configuration can be small. A terminal with a configuration capable of general videophone calls can be realized with almost no additional hardware. Further, by widening the range of colors to be replaced to some extent, even a portion where there is some color unevenness can be replaced at the same time, and good image processing can be performed.

  Further, as in the example shown in FIG. 5, not only the background but also images of various parts such as lips and hair can be replaced, and an image having various effects can be obtained. Further, by allowing the user to select a range to be determined as an approximate color to some extent, the replacement range can be freely set, and processing according to the user's preference can be performed.

  In the above description, a color is specified, and the range that matches the specified color is simply replaced with another image. For example, when a specific position in the image is specified with a pointer. In addition, only a range that matches the color at that position and that is continuous with the position specified by the pointer may be replaced. Specifically, for example, when the hair is indicated with a pointer in the example of FIG. 5, the range replaced with the composite image of FIG. 5C is the same as that of a part of clothes such as the neck as well as the hair. Even if it is a part of the correct color, it will be the range replaced with another image, but only the part of the hair that is the same color range that is continuous with the position specified by the pointer will be extracted and only that part will be replaced It may be.

  In the above-described embodiment, an example in which a videophone call is performed has been described. However, for example, an image to be transmitted attached to an e-mail may be edited (processed) by the same process.

  In the embodiment described so far, the example is applied to a mobile phone terminal. However, when an image shot in a terminal is edited by the same operation and processing in the terminal and transmitted outside the terminal. It is applicable to various configurations. For example, when a data processing terminal called PDA (Personal Digital Assistance) is equipped with a camera unit and a communication function capable of accessing a wireless LAN (Local Area Network), etc., and configured to transmit captured images In addition, a program for performing the processing shown in the flowchart of FIG. 2 may be installed to function as a terminal that performs similar image processing under the control of the program.

It is a block diagram which shows the example of a terminal structure by one embodiment of this invention. It is the flowchart which showed the example of the control processing by one embodiment of this invention. It is explanatory drawing which showed the example of the color selection process at the time of the incoming call by one embodiment of this invention. It is explanatory drawing which shows the example (example 1) of the image by operation of one embodiment of this invention. It is explanatory drawing which shows the example (example 2) of the image by operation of one embodiment of this invention. It is a characteristic view which shows the image of the color range calculation example by one embodiment of this invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 ... Mobile phone terminal, 11 ... Antenna, 12 ... Communication circuit, 13 ... Microphone, 14 ... Speaker, 15 ... Memory, 16 ... Operation part, 20 ... Control part, 21 ... Control line, 22 ... Data line, 31 ... Imaging Control unit, 32 ... camera unit, 40 ... display unit

Claims (5)

A shooting section;
A display unit for displaying an image photographed by the photographing unit;
An operation unit for designating a specific position in an image photographed by the photographing unit and displayed on the display unit;
An image processing unit that extracts a color range that approximates the display color of the location specified by the operation of the operation unit from a captured image, and replaces the extracted range with another image prepared in advance;
A communication terminal device comprising: a transmission unit that transmits an image generated by the processing in the image processing unit. The communication terminal device according to claim 1,
The communication terminal apparatus is characterized in that a range extracted with a color approximate to the specific display color is only a range continuous with the designated portion. The communication terminal device according to claim 1,
A communication terminal apparatus, wherein a color range approximate to the specific display color can be set by an operation of the operation unit. The communication terminal device according to claim 1,
An image prepared in advance or an image immediately before performing an operation is transmitted from the transmission unit while the operation unit performs an operation for replacement in the image processing unit. Terminal device. Perform processing to display the captured image,
When a specific position in the image displayed in the process is specified,
An extraction process for extracting from the captured image a color range that approximates the display color of the designated location;
A replacement process for replacing the image in the range extracted by the extraction process with another image prepared in advance;
An image communication method comprising: performing transmission processing for transmitting an image generated by the replacement processing.