JP4603249B2 - Image processing apparatus for mobile phone with camera - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image processing apparatus in a camera-equipped mobile phone.
[0002]
[Prior art]
As a conventional image processing apparatus, there is an apparatus that pastes an image captured by an imaging apparatus to a virtual three-dimensional object in a virtual space. According to this, the image information is captured by the image capturing means, and from this captured image information, using a clipping range designation frame provided with a gauge provided corresponding to the feature position of the virtual three-dimensional object, The image cutout means cuts out a predetermined range of image information. Moreover, the image information of the predetermined range cut out in this way is pasted on the virtual three-dimensional object (for example, refer to Patent Document 1).
[0003]
In this image processing apparatus, the frame for a frame is displayed on a two-dimensional image captured and captured in advance, the frame is moved and enlarged by the user, and the cutout position is determined. The work of cutting out the part and pasting it on the virtual three-dimensional object is required.
[0004]
[Patent Document 1]
Japanese Patent Laid-Open No. 2000-235656
[Problems to be solved by the invention]
However, in the conventional image processing apparatus, at the time of image capturing by the image capturing apparatus, it is difficult to grasp the image image after pasting the captured image on the virtual three-dimensional object, and a composite image as expected cannot be obtained. There was a problem that there was.
In addition, there is a problem that the user needs to move and cut out the frame of the cutout range for the captured image, and this work is troublesome.
[0006]
The problem to be solved by the present invention is to solve the problem that the screen image after pasting the virtual three-dimensional object of the image at the time of imaging cannot be grasped, or the user moves and cuts out the frame of the cutout range One example is the solution to the problem that it is necessary and time-consuming.
[0007]
[Means for Solving the Problems]
An image processing apparatus for a camera-equipped cellular phone according to claim 1 comprises: a communication unit that transmits and receives communication / call information and image information via a cellular phone communication network; an image storage unit that stores image information; and a key input unit. An image processing apparatus for a camera-equipped mobile phone including a frame image in which a captured image is transmitted and displayed, and a transmission region in which a character's feature position and shape are matched is set in the image storage unit in advance A display means for displaying the frame image as a preview image, and an imaging lens arranged on the same plane as the display means so that the image can be taken while viewing the preview image, and the photographer's own image is received through the imaging lens. The image data formed on the surface and photoelectrically converted is continuously output to the display means and synthesized with the frame image to form the transmission region. An imaging unit to display the image of the photographer himself, capturing a still image of the key input unit the transmission region the photographer is displayed in its own the release key is confirmed the moment pressed in to by the image pickup means The captured still image data in the transparent area is converted into texture to obtain captured image texture data, and the texture data to be replaced by pasting of the captured image texture data and coordinate settings are matched in advance, and then the character Image processing means for pasting the captured image texture data at the feature position, and the frame image is provided with a mark indicating the position of the imaging position in the transmissive area in the transmissive area , when the image is taken by the imaging means, especially that marks the transmissive region are not synthesized in the still image captured To.
[0008]
[0009]
[0010]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
The image processing apparatus according to the embodiment of the present invention is used in , for example, a camera-equipped mobile phone. Figure 1 is a block diagram showing a camera-equipped mobile phone configuration is an example of using the image processing equipment according to the embodiment of the present invention.
As shown in FIG. 1, the camera-equipped mobile phone 10 is configured by connecting an imaging unit 12, a CPU 13, a memory unit 14, a key input unit 15, a display unit 16, and a communication unit 17 via a data bus 11. .
[0011]
The imaging unit 12 functions as an imaging unit, forms an image of a subject on a light receiving surface, performs photoelectric conversion, and outputs the image data. The imaging unit 12 may be provided with, for example, a strobe device (not shown), a focus light adjusting unit that focuses the subject on the light receiving surface, a zoom adjusting unit, a diaphragm adjusting unit, and the like.
[0012]
The CPU 13 functions as an information processing unit and an image processing unit that control the overall operation of the camera-equipped mobile phone 10. This CPU 13 cuts out the image information obtained by the image pickup unit 12 and sends it to a memory, which will be described later, where the character registration unit is located. In addition to controlling the image to be pasted on a character (object), the CPU 13 controls the sampling timing of image data, Control such as control, exposure control, focus control, zoom control, image data recording control, communication control, display control, and various conversion processing of image data is performed.
[0013]
The memory unit 14 functions as image storage means for storing image information such as character vertex information, polygon configuration information, texture coordinate information, and deformation weighting information in addition to the virtual three-dimensional shape data of the character.
In addition, the memory unit 14 includes a mobile phone operation program, an image processing program, various constants, parameters indicating imaging conditions, parameters indicating image properties, and the like, as well as work during program execution. It is composed of an area RAM or the like.
[0014]
The key input unit 15 includes data keys and control keys, and is used for applications such as input of corresponding data and a release key (button for releasing a shutter at the time of imaging) used for imaging of the imaging unit 12.
The display unit 16 functions as a display unit that displays image data in addition to various text data, and includes, for example, a liquid crystal display or an organic EL display.
The communication unit 17 transmits and receives communication / call information, the image information, and the like via a mobile phone communication network.
[0015]
Then, like an example of the appearance of a camera-equipped mobile phone 10 to use the image processing equipment according to the present embodiment will be described with reference to FIG.
FIG. 2 is a front view of the camera-equipped mobile phone. The camera-equipped mobile phone 10 has a vertically long outer casing, and a display unit 16 including a liquid crystal display or an organic EL display panel is provided on the front.
[0016]
Above the display unit 16, a speaker 18 serving as a receiver of the camera-equipped mobile phone 10 is disposed, and on the other hand, an imaging lens 19 of the imaging unit 12 is disposed below the display unit 16. In addition, an antenna 23 of the communication unit 17 protrudes from the upper surface of the exterior body.
[0017]
Further, a control key group 24 for selecting various functions including a release key 21 and a cursor movement key 22 and a data key group 25 including a dial button are disposed below the imaging lens 19.
Further, a microphone 26 is disposed below the data key group 25. Each key of the data key group 25 and the control key group 24 is provided in the key input unit 15.
[0018]
Next, a processing method of the image processing apparatus according to the present embodiment will be described with reference to the drawings.
First, a processing procedure from capturing a captured image to texturing will be described with reference to FIGS. 4 and 5 along the flowchart of FIG.
[0019]
First, a frame image 27 as a guide is prepared in advance as shown in FIG. 4 (step S1). The frame image 27 is set with a transmissive area 27 a in which the captured image is displayed when the preview image is displayed on the imaging unit 12. The design is such that the image when the captured image is pasted is as easy to understand as possible. The data of the frame image 27 is registered in advance in a predetermined area in the memory unit 14.
[0020]
Next, when the digital camera function is started (step S2), the imaging unit 12 captures a preview image (for example, the user's own face) through the imaging lens 19 (step S3).
[0021]
The frame image 27 obtained in step S1 is combined with the captured preview image except for the transparent region 27a (step S4). The synthesized image is displayed on the display unit 16 (step S5).
[0022]
Then, as shown in FIG. 5, the user looks at the display image on the display unit 16 while holding the camera-equipped mobile phone 10, and is the subject so that the user's own face enters the transparent area 27 a of the frame image 27. While adjusting the positional relationship between the user's own face and the imaging lens 19 of the camera-equipped mobile phone 10 by moving the hand , the cursor movement key 22 is further operated to display the user's own face displayed in the transparent area 27a. a fine adjustment of the zoom and position Te.
[0023]
Next, it is determined whether or not the user has decided to shoot and pressed the release key 21 (step S6) . If not pressed (NO in step S6) , the processes in and after step S3 are repeated. As a result, it is possible to adjust the preview image that changes every moment in real time so that the user's own face enters the transparent region 27a .
[0024]
When the user finally decides and the release key 21 is pressed (YES in step S6), the still image displayed in the transparent area 27a is determined at the moment when the release key 21 is pressed , and this still image is displayed. the encoded as a general image information (step S7).
[0025]
If the image information generated in step S7 and the format of the data that can be handled as a texture are different, the format is converted to a texture format (step S8). Thus, the captured image texture data said determined still image is textured is obtained.
[0026]
Subsequently, the captured image texture data, wherein the position of the character (for example, a face) the processing procedure up displays stuck on along the flows charts of FIG. 6 will be described with reference to FIGS. 7-11.
First, the virtual three-dimensional shape data (for example, the whole stuffed animal) of the character 28 registered in advance in the memory unit 14 as shown in FIG. 7 is read (step S11).
[0027]
Next, the virtual three-dimensional motion data of the character 28 is read as continuous image data for moving image formation (step S12).
[0028]
Next, the virtual three-dimensional texture data 28A of the character 28 shown in FIG. 8 (for example, the whole body with the stuffed face removed) is read (step S13).
[0029]
Here, the virtual three-dimensional texture data 28A refers to a texture (for example, a stuffed toy whole body without a face) that cannot be replaced by pasting captured image texture data (for example, a user's face). Here, the presence / absence of captured image texture data created by the procedure described in the flowchart of FIG. 3 is determined (step S14).
[0030]
If it is determined that there is no captured image texture data, virtual three-dimensional texture data 28B (for example, a stuffed face) of the character 28 as shown in FIG. 9 is read (step S15).
[0031]
Here, the virtual three-dimensional texture data 28B of the character refers to a texture that is replaced by pasting captured image texture data (for example, a user's face). This is pasted as a default image before the image is taken or when the user deletes the image data.
[0032]
On the other hand, when there is captured image texture data, captured image texture data 29 as shown in FIG. 10 is read (step S16).
[0033]
At the time of creating character data, texture coordinates are set on the premise that an image of the virtual three-dimensional texture data 28B of the character 28 is pasted. However, the character data is taken according to the virtual three-dimensional texture data 28B and the frame image 27. By matching the texture coordinate relationship of the image data in advance, the captured image can be pasted at an appropriate position simply by replacing the image without requiring position adjustment after shooting.
[0034]
In this way, as shown in FIG. 11, an image (for example, a stuffed animal with a user's face pasted) is created by pasting the captured image texture data 29 on the virtual three-dimensional texture data 28A.
[0035]
Furthermore, if a 3D simulation is performed on the virtual 3D wire frame of the character 28 as shown in FIG. 12 using 3D polygon display, texture mapping technology, etc., a virtual 3D character animation (for example, the user's A moving image of a stuffed toy with a face attached can be generated (step S17).
[0036]
Finally, the virtual three-dimensional character moving image generated in this way is displayed on the display unit 16 (step S18).
[0037]
When the frame image 27 is combined with the captured image at the time of the preview (step S4), the frame image 27 serves as a guide for setting the pasting position. Can also be used.
For example, the image data stored in the memory unit 14 in advance is used to synthesize an image of the edge of the glasses represented as a frame or an image such as a red nose (for example, glasses or a red nose are synthesized on the user's face). Can be pasted on the character 28.
[0038]
Also, when capturing an image, it is not always necessary to combine the captured image and the frame image, so that the captured image is overwritten by an image written only as a guide for the imaging position. Can be prevented.
For example, when taking an image of a user's face and using an image with a mark or the like as a guide for the imaging position of the eyes or mouth as a frame, if the image is combined as it is, the above mark or the like is overwritten on the face. , The image becomes unsuitable for use in pasting. Since such a problem does not occur unless they are combined, a frame image having a finer guide function (for example, a mark or the like as a guide for the imaging position) can be used for imaging.
[0039]
In the present embodiment, the virtual three-dimensional character moving image is displayed on the display unit 16, but the image displayed by pasting the captured image texture data on the character may be a two-dimensional moving image or a still image. .
[0040]
Note that the above virtual three-dimensional character video, two-dimensional video, still image, etc. do not necessarily have to be displayed on the display unit 16, and those data are transmitted to other mobile phones, terminals, etc. wirelessly or by wire. Also good.
[0041]
[0042]
As described above in detail, the camera-equipped cellular phone 10 that is the image processing apparatus in the present embodiment is
A frame image 27 in which a captured image is transmitted and displayed and a transparent region 27a in which the shape of the character 28 is matched with the feature position (for example, a face portion) is set in the memory unit 14 (image storage unit) in advance. A display unit 16 (display means) that is prepared and displays the frame image 27 as a preview image ;
An imaging lens 19 is arranged on the same surface as the display unit 16 so that the user can take an image while viewing the preview image. The image of the photographer himself is formed on the light receiving surface through the imaging lens 19 and the photoelectrically converted image data is displayed. The image is continuously output to the unit 16 and is combined with the frame image 27 to display an image of the photographer himself / herself in the transmissive area 27a, and the release key 21 that functions as the shutter button of the key input unit 15 is pressed by the photographer. An image pickup unit 12 (image pickup means) for picking up a still image displayed in the transmission region determined at a certain moment ;
The still image data in the transparent area captured by the image capturing unit 12 is converted into texture to be captured image texture data 29, and texture data (virtual three-dimensional texture data 28B replaced by pasting of the captured image texture data 29 is added. ) And the coordinate setting in advance, and then the CPU 13 (image processing means) for pasting the captured image texture data 29 to the characteristic position of the character 28 (for example, the position of the face) .
As a result, the user can easily grasp the pasting position of the captured image on the character 28 at the time of shooting, and an optimum still image can be obtained simply by pressing the release key 21 when the position becomes appropriate.
Further, it is not necessary for the user to move and cut out the frame of the cutout range for the picked up image, and the picked-up image texture data 29 can be pasted on the character 28 without taking time and effort.
[Brief description of the drawings]
1 is a block diagram showing a camera-equipped mobile phone configuration is an example of using the image processing equipment according to the embodiment of the present invention.
2 is a front view of a camera-equipped mobile phone as an example of using the image processing equipment according to the embodiment of the present invention.
FIG. 3 is a flowchart showing processing procedure processing from capturing a captured image to texturing of the image processing method in the image processing apparatus according to the embodiment of the present invention;
FIG. 4 is a diagram illustrating a frame image.
FIG. 5 is an explanatory diagram showing that a user is capturing a captured image while holding a camera-equipped mobile phone;
6 is a flow chart showing the processing procedure up displays paste captured image texture data to the feature position of the character.
FIG. 7 is a diagram showing characters 28 registered in advance.
FIG. 8 is a diagram showing character virtual three-dimensional texture data 28A (for example, a whole body with a stuffed animal face removed);
FIG. 9 is a view showing virtual three-dimensional texture data 28B (for example, a stuffed face) of a character.
FIG. 10 is a diagram illustrating captured image texture data 29 (for example, a user's face).
FIG. 11 is a diagram illustrating an image (for example, a stuffed animal with a user's face pasted) on which the captured image texture data 29 is pasted on the virtual three-dimensional texture data 28A.
FIG. 12 shows a virtual three-dimensional wire frame of a character.
[Explanation of symbols]
10 Mobile phone (image processing device)
12 Imaging unit (imaging means)
13 CPU (image processing means)
14 Memory unit (image storage means)
16 Display section (display means)
27 Image for frame 27a Transparent area 28 Character

Claims (1)

An image processing apparatus for a camera-equipped mobile phone including a communication unit that transmits and receives communication / call information and image information via a mobile phone communication network, an image storage unit that stores image information, and a key input unit,
A display in which a captured image is transmitted and displayed, and a frame image in which a transparent region in which the character's characteristic position and shape are matched is set in the image storage means, and the frame image is displayed as a preview image Means,
An imaging lens is arranged on the same surface as the display means so that the image can be taken while viewing the preview image, and an image of the photographer is formed on a light receiving surface through the imaging lens, and photoelectrically converted image data is displayed on the display means Are continuously output and combined with the frame image to display an image of the photographer himself / herself in the transmissive area, and displayed in the transmissive area determined at the moment when the release key of the key input unit is pressed. An imaging means for capturing a still image of the photographer himself / herself ,
Still image data in the transparent area captured by the imaging means is converted into texture to obtain captured image texture data, and the texture data to be replaced by pasting the captured image texture data and coordinate settings are matched in advance. From the image processing means for pasting the captured image texture data to the feature position of the character ,
Have
The frame image is provided with a mark in the transmissive area indicating a position that serves as a guide for an imaging position in the transmissive area .
An image processing apparatus in a camera-equipped cellular phone , wherein a mark in the transmission region is not combined with the captured still image when imaging is performed by the imaging means .

Images