JP4351023B2 - Image processing method and apparatus - Google Patents

Description

  The present invention relates to an image processing apparatus incorporated in a communication apparatus using an image (for example, an apparatus for videophone, videoconference, video mail or video chat, a door phone, etc.), and more particularly, a person image (for example, a user) Is related to an image processing apparatus.

  There is an amusement machine that prints an image of a user on various print media and outputs the image. This is designed to superimpose a frame of a decoration (for example, a pattern or a pattern) on a photographed person image and print it on a print medium (for example, a sticker) for output.

  Such a game machine itself does not have a function of adjusting the positional relationship between the person image and the decoration. Therefore, even if the user arranges the person image so that it fits within the decoration frame, the decoration overlaps with the person image and the face part, head part, etc. may be hidden and the person image may become unclear. It was.

  Patent Document 1 (Japanese Patent Laid-Open No. 2000-22929) considers this point, and decoration is applied to a body part of a human image, particularly a body part region set corresponding to the face and head. We have proposed a technique to add decorations while avoiding overlapping.

On the other hand, a camera-equipped mobile phone having a function of superimposing a photographed person image on a decorative frame and combining the image and transmitting the combined image has been put into practical use. When the sender uses such a camera-equipped mobile phone, the receiver can enjoy the pleasure of seeing the image edited by the sender.
JP 2000-22929 A (page 3-7)

  In Patent Document 1, it is assumed that the decoration does not move. In the technique described in this document, the decoration is added so that the decoration does not overlap the body part of the human image, particularly the body part region set corresponding to the face and head.

  In this document, printing on a print medium is used as an output, and it is not normally possible to edit a once printed image after printing. Therefore, this idea can be said to be appropriate.

  However, the above-described camera-equipped mobile phones have become widespread, and here, in a normal case, the composite image is displayed on a display without being printed on a print medium. And in the future, users will want to move the decoration to get more fun.

  However, the prior art only considers adding a decoration that does not move to a person image that does not move, and does not consider adding a decoration that moves. For this reason, when the conventional technology is simply applied to a moving person image, the person image and the decoration interfere with each other, and the person image becomes unclear or cannot be used to enhance playability.

  SUMMARY OF THE INVENTION An object of the present invention is to provide an image processing method and an inter-related technology capable of clearly displaying a person image and improving playability.

The image processing method according to claim 1, comprising: an image input unit that obtains an input image; a control unit that controls image processing of the input image; and an image output unit that outputs the input image subjected to image processing. A method of detecting a position of a body part region from an input image including a human image, and detecting a rotation amount of the body part region within a plane of the input image and / or a rotation amount with respect to the plane ; , decoration with moving trajectory in accordance with the detected position of the body part area coordinate system determines the position of the origin of the steps of determining the amount of rotation of the coordinate system of the decoration in accordance with the rotation amount detected, set the origin comprising the steps of position and defined amount of rotation from the movement locus determining the location information of the decoration, based on the arrangement information, and generating an output image by placing the decoration, and outputting an output image.

According to this configuration, when the person subject moves in the input image, the decoration can be moved according to the movement. According to this configuration, when the face is tilted to the side, the decoration can be tilted and the movement path of the decoration can be tilted. Therefore, even when the rotation direction changes in the plane of the human input image, , The decoration can appear to follow. According to this configuration, when the person is “turning right”, the decoration and the movement path can be deformed according to the rotation of the person. Accordingly, it is possible to make it appear that the decoration follows the change in the orientation of the input image with respect to the plane.

  In the image processing method according to the second aspect, in the outputting step, image data in which decoration is drawn on the input image is output.

  With this configuration, it is not necessary to combine the input image and the decoration at the transfer destination of the output image.

  In the image processing method according to the third aspect, in the outputting step, image data in which the input image is associated with the decoration is output.

  With this configuration, the destination of the output image is convenient to restore the input image.

  In the image processing method according to the fourth aspect, the input image and the decoration are associated by the metafile.

  With this configuration, the input image and the decoration can be simply associated with each other.

  With this configuration, the input image can be easily associated with the decoration.

In the image processing method according to claim 5 , the opacity of the decoration is set in a region in front of the virtual surface corresponding to the projection surface of the input image of the body part region and behind the virtual surface corresponding to the projection surface of the input image. In the step of determining differently in a certain area and the step of generating, an output image is generated by arranging decorations based on the arrangement information and the determined opacity.

  According to this configuration, the decoration can be erased when the decoration movement path comes to the back of the body part area, so that the decoration surrounds the body part area and moves in three dimensions around it. You can make it look like you are.

In the image processing method according to the sixth aspect, when the decoration collides with the body part region, the moving direction of the decoration is changed to a direction away from the body part region.

  According to this configuration, it is possible to give the decoration a motion that reflects when it hits the body part region.

  According to the present invention, even when both the person image and the decoration are moving, the decoration can be prevented from interfering with the person image, the person image can be clearly displayed, and the movement of the decoration can be changed to the movement of the person image. By making them cooperate, playability can be improved.

  Embodiments of the present invention will be described below with reference to the drawings. First, important terms are explained prior to the description of the specific configuration.

  A “person image” is an image that includes a part or all of an image of a person. Therefore, it may be an image of the entire person, only the face or back of the person, or only the upper body. Moreover, the image in which several persons are reflected may be sufficient. Of course, the background may have any pattern other than a person such as a landscape or a pattern.

  "Body part" means a part of a person's body, and it can be recognized as a part of a person's body even if the skin wearing clothes, hats or shoes is not visible on the body part It is included in the body part. Therefore, the face is a body part, and the head is a body part. In addition, eyes, nose, mouth, ears, eyebrows, hair, neck, dressed upper body, hands, arms, legs, shoes legs, head with hat, eyes with glasses, etc. are also body parts. is there.

  The “body part area” is an area defined as an area occupied by a body part in a human image, and the body part area may include a part that is not a body part inside the body part area. It may be a region located in the area.

  That is, if the body part is a face, the body part region may include a region in which the body part region corresponding to the face is around the face, or the smallest size surrounding both eyes, mouth, and nose. It may be a rectangle that does not include a forehead or ears.

  “Decoration” refers to an image pattern to be added. This image pattern may be an image pattern stored in advance, or may be an image pattern generated by a computer graphics technique. The image pattern may be a character or symbol pattern or a graphic pattern.

  FIG. 1 is a functional block diagram of an image processing apparatus according to an embodiment of the present invention. As shown in FIG. 1, the image processing apparatus according to the present embodiment includes the following components.

  The image input unit 1 obtains an input image. In this embodiment, a person image is included in the input image, and the person image is allowed to move, and the input image usually means an image for one frame of the moving image.

  However, this input image may be the one immediately after being shot by the camera, or after the shooting, the data encoded according to the encoding method such as MPEG (the one recorded on the recording medium may be loaded, It may also be obtained by decoding (received from the transmitter).

  The control part 2 controls each element shown in FIG. 1 according to the flowchart of FIG.

  The image output unit 3 outputs an image in which decoration is combined with the input image as an output image. In addition to being displayed on the display device, the output image can be encoded according to the encoding method and stored in a recording medium, or can be transmitted by another communication apparatus.

  The input image storage unit 4 stores the input image input by the image input unit 1.

  The template memory | storage part 5 memorize | stores the template of a body part area | region. Hereinafter, in this embodiment, the face area is taken as a body part area.

  As shown in FIGS. 6 (a), 6 (b), and 6 (c), templates are prepared for various sizes that model the contours of facial parts (head, eyes, nose, mouth). use.

  Further, as shown in FIGS. 7B and 7C, in addition to the template based on the basic posture shown in FIG. 7A, a template having a rotation amount in the plane of the input image is used in combination. Thereby, the inclination by the rotation of the face can be detected.

  Further, as shown in FIGS. 8B and 8C, in addition to the template based on the basic posture shown in FIG. 8A, a template having a rotation amount with respect to the plane of the input image is used in combination. Thereby, when the face orientation is different from the basic posture (when the normal vector passing through the center of the face is not orthogonal to the plane of the input image), the face orientation can be detected.

  In FIG. 1, the detection unit 6 detects a body part region from the input image stored in the input image storage unit 4 using the template in the template storage unit 5.

  Here, as illustrated in FIG. 9, the detection unit 6 applies a differential filter to the input image of the input image storage unit 4 to extract an edge component. In addition, the detection unit 6 selects the template 53 from the template storage unit 5, and executes pattern match matching using the selected template 53 and the edge component.

  In this pattern matching, the template 53 (Nx * Ny pixels) is moved on the search range (Mx−Nx + 1) (My−Ny + 1) in the larger input image (Mx * My pixels), and the residual shown in the following equation is used. This is a process for obtaining the upper left position (a, b) of the template image in which R is minimized.

  However, in (Equation 1), I (a, b) (mx, my) is a partial image of the input image, and T (mx, my) is an image of the template 53.

  Further, instead of (Equation 1), a cross-correlation coefficient C represented by the following equation may be obtained, and the upper left position (a, b) of the template image where the cross-correlation coefficient C is maximized may be obtained.

  The detection unit 6 uses various templates shown in FIGS. 6, 7, and 8. When using (Equation 1), the template that minimizes the residual R is used, and when using (Equation 2), A template that maximizes the relationship number C is obtained. Then, the detection unit 6 sets the obtained position, size, inclination, and orientation of the template as the respective values of the face image in the input image.

  In the case of a face image, the same processing is performed every frame (or every several frames), whereby the transition of the position, size, inclination, and orientation of the face can be obtained.

  In addition, when a plurality of faces are present in the image, the detection unit 6 has a position where the residual R is low or a position where the correlation coefficient C is high in the area excluding the already detected first face area. Is detected as the face partial region of the second person.

  Similarly, for the third and subsequent faces, the detection unit 6 sets a position where the residual R is low and a position where the correlation coefficient R is high in the area excluding the already detected face partial area as the face partial area. However, the detection is repeated until the residual R becomes larger than a predetermined threshold or until the correlation coefficient C becomes smaller. Thereby, the face partial areas of a plurality of persons can be detected.

  Of course, it is not always necessary to carry out all of the above detections, and they may be appropriately selected except for detection of the position and size of the face area.

  In FIG. 1, a decoration information storage unit 7 stores decoration information. Next, the decoration information will be described with reference to FIG.

  In the example (decoration information (part 2)) illustrated in FIG. 10A, the decoration information is configured by associating decoration image data and transform properties.

  In the “decoration image data” field, the decoration image data itself or an ID or pointer indicating the decoration image data itself is stored. As shown in FIG. 10A, the decoration image data may be composed only of image data or may have additional attributes (such as behavior at the time of collision). .

  The field of “transform property” is information on how to move the decoration with respect to the decoration layer and how to overlay the decoration layer.

  Specifically, the decoration is defined by a transform property and a local coordinate system different from the input image (an anchor point is the origin). Furthermore, this coordinate system is defined by position, scale (scale rate). , Rotation and opacity attributes.

  More specifically, the anchor point indicates the position and orientation of the coordinate origin of the local coordinate system related to the decoration. The values set in the “Position”, “Scale”, and “Rotation” fields are the actual image position, scale (enlargement / reduction ratio), and rotation with respect to the coordinate system (local coordinate system) of the anchor point. Indicates the degree.

  In the example shown in FIG. 10B (decoration information (part 1)), the decoration information is configured by associating the decoration image data with the mask data.

  At this time, the “decorative image data” is data of an image displayed as a background pattern, and the “mask data” is a data portion of the background pattern (hatched portion) and a portion where an image of the face area appears (non-hatched line). Part) shows the size and positional relationship of the area.

  In FIG. 1, the decoration arrangement unit 8 determines decoration arrangement information so as to synchronize with the change in the body part region detected by the detection unit 6.

  If the maximum matching is obtained with the template shown in FIG. 7, the detection unit 6 detects the amount of rotation of the body part region in the plane of the input image, and the decoration placement unit 8 is detected by the detection unit 6. The arrangement information of the decoration is determined so as to synchronize with the change of the rotation amount.

  If the maximum matching is obtained with the template shown in FIG. 8, the detection unit 6 detects the rotation amount of the body part region with respect to the plane of the input image, and the decoration arrangement unit 8 detects the rotation amount detected by the detection unit 6. The arrangement information of the decoration is determined so as to synchronize with the change of.

  The detailed processing of the decoration placement unit 8 will be described in detail later with several examples.

  In FIG. 1, the combining unit 10 outputs an image in which decorations are arranged in the input image based on the arrangement information determined by the decoration arranging unit 8. This image is temporarily stored in the output image storage unit 11 and then output from the image output unit 3.

Note that the combining unit 10 performs display on the display device.
(Data Form 1) Image data in which decorations are directly drawn on an input image is generated, but the combining unit 10 is used as image data to be transferred to a recording medium or another communication device.
(Data format 2) Image data in which an input image is associated with parameters for drawing decoration may be generated and transferred. The data format 2 is advantageous because the input image can be easily restored. The data form 2 will be described later.

  Further, in FIG. 1, the operation unit 12 is for a user to input operation information to the control unit 2.

  Next, a specific configuration example of the image processing apparatus shown in FIG. 1 and its peripheral part will be described with reference to FIG. FIG. 2 is a block diagram of the image processing apparatus.

  In the example of FIG. 2, the image processing apparatus of FIG. 1 is incorporated in a camera-equipped mobile phone. As shown in FIG. 2, the camera-equipped mobile phone has the following elements.

  The CPU 21 controls each element in FIG. 2 via the bus 20, and executes a control program recorded in the ROM 23 and according to the flowchart in FIG.

  In the RAM 22, a temporary storage area required for processing by the CPU 21 is secured. The flash memory 24 is a device corresponding to a recording medium.

  In addition, the communication processing unit 26 transmits / receives data to / from an external communication device via the antenna 25.

  The image processing unit 27 includes an encoder / decoder or the like related to an encoding method such as MPEG, and processes an image captured by the camera 28 or based on image data instructed by the CPU 21, an LCD (an example of a display device) 29. Control the display state of.

  The audio processing unit 30 controls input from the microphone 31 and audio output via the speaker 32.

  An interface 33 is connected to the bus 20, and the user can input operation information through the key set 34 via the interface 33, and another device is connected via the port 35. It is configured so that it can be connected.

  Here, the image input unit 1 in FIG. 1 is realized by the CPU 21 or the image processing unit 27 processing data recorded in the flash memory 24 or data taken by the camera 28.

  The control unit 2, the detection unit 6, the decoration arrangement unit 8, and the synthesis unit 10 are realized by the CPU 21 performing calculations while exchanging data with the RAM 22, the flash memory 24, and the like.

  Further, the input image storage unit 4, the template storage unit 5, the decoration information storage unit 7, the arrangement information storage unit 9, and the output image storage unit 11 correspond to areas secured in any of the RAM 22, the ROM 23, and the flash memory 24. . The key set 34 in FIG. 2 corresponds to the operation unit 12 in FIG.

  The CPU 21 recognizes an operation performed by the user on the key set 34, acquires an image from the camera 28, compresses the camera image and stores it in the flash memory 24, reads and decompresses the stored image, combines the image, Playback and display on the LCD 29 are performed, but some of these processes may be performed by the image processing unit 27.

  FIG. 3 shows the appearance of the image processing apparatus of this embodiment. As described above, the image processing apparatus of this embodiment is incorporated in a camera-equipped mobile phone. The camera-equipped mobile phone of FIG. 3 is of the double fold type in which an upper housing 42 and a lower housing 41 are foldably coupled by a coupling portion 43. Of course, the external shape of the image processing apparatus is arbitrary, and the present invention can be applied to apparatuses other than mobile phones such as PDAs.

  Next, a supplementary explanation will be given for the data format 2 with reference to FIG. Here, there is an input image (file name “face.rm”) shown in FIG. 4B, and this is associated with the decoration (file name “star.swf”) shown in FIG. 4C.

  At this time, it is preferable to use a metafile in the SMIL (Synchronized Multimedia Integration Language) format shown in FIG. In FIG. 4A, a line <video src = “face.rm”> and a line <animation src = “star.swf”> sandwiched between <par> and </ par> It is executed at the same time. That is, at the time of display, the input image in FIG. 4B and the decoration in FIG. 4C are combined.

  When the association as shown in FIG. 4 is performed, it is not necessary to create an image itself that combines the input image of FIG. 4B and the decoration of FIG. 4C.

  Also, instead of the above SMIL metafile, an XML (Extensible Markup Language) format file, MP4, or the like may be used. That is, when the data form 2 is used, the input image and the decoration need only be associated with each other in some form.

  Next, the outline of the flow of processing by the image processing apparatus of this embodiment will be described with reference to FIGS. 1, 5, and 12. Here, it is assumed that the decoration information (part 1) shown in FIG.

  First, in step 1, the control unit 2 checks whether or not the processing has been completed for all the frames constituting the moving image. If completed, the process ends. If not completed, the process proceeds to step 2.

  In step 2, the image input unit 1 obtains an input image as shown in FIG. 12A, and the control unit 2 stores this in the input image storage unit 4.

  Next, in step 3, the control unit 2 instructs the detection unit 6 to detect a face area. Thereby, the detection unit 6 detects a face area using the template (FIG. 12B) in the template storage unit 5.

  Next, in step 4, the control unit 2 commands the decorative arrangement unit 8 to determine the arrangement information. Thus, the decoration placement unit 8 uses the decoration information in the decoration information storage unit 7 to determine the placement information and stores it in the placement information storage unit 9. As a result, a decoration image on the decoration layer as shown in FIG.

  Next, in step 5, the control unit 2 instructs the combining unit 10 to combine the input image and the decoration. Thus, the combining unit 10 combines the decoration with the input image according to the decoration information stored in the decoration information storage unit 7 and stores the result in the output image storage unit 11 as shown in FIG.

  Next, in step 6, the control unit 2 outputs the composite image in the output image storage unit 11 to the outside via the image output unit 3.

  The control unit 2 repeats the processing from step 1 onward until the processing for all frames is completed.

  Next, a configuration example of the user interface will be described with reference to FIG. Here, it is assumed that the application selected by the user is image editing software and is an image composition mode in which the person image photographed by the user and the decoration are combined.

  First, as shown in FIG. 11A, a person image to be edited is displayed on the LCD 29. An edit button 40 and a return button 41 are displayed along with the person image.

  Here, when the user selects the edit button 40 using the key set 34, a list of insertable decorations is displayed as shown in FIG.

  Then, when the user selects the item “tears” and presses the enter button 43, the face area of the person image is detected, and decoration arrangement information corresponding to the face area is obtained, and the decoration “tears” are determined according to the decoration arrangement information. The result is synthesized with the input image, and the result is displayed on the LCD 29 as shown in FIG. The tear decorations 50 and 51 are added to the input image. Here, tears are inserted as moving animations in addition to still images.

  Now, in the prior art, tears do not move while placed. Further, even if an animation is inserted, the movement of this animation moves independently of the movement of the person image.

  For example, tears are not automatically inserted under the eyes and are not automatically adjusted so that the tears stop around the cheeks of the forehead. End up. Also, when the face tilts or moves, the movement of tears cannot be corrected to match the face.

  On the other hand, in this embodiment, before the tear is inserted, the position and size of the body part region (for example, eyes in the face region in this embodiment) are detected to automatically synchronize with the movement of the body part region. Tears are inserted under the eyes. Moreover, the decoration can follow the movement of the face.

  Hereinafter, the decoration arrangement information will be further described with reference to application examples of the present embodiment. FIG. 13 is an explanatory diagram of the positional relationship between the input image and the coordinate system of decoration.

  Here, as a supplement prior to the description of each application example, a procedure for creating a general animation will be briefly described. Basically, the animation that moves the decoration is created by moving the decoration layer associated with the transform property.

  In other words, to create an animation, set the transform attributes (position, rotation, etc.) for each key frame (present on the time axis), and change each attribute value as time passes. Let

  Here, the key frame defines values such as the position of the layer at the reference time point. If there are two key frames with different values (of course, their positions on the time axis are different), for each frame located between these key frames on the time axis, each value of the attribute is calculated by calculating an intermediate value. The value can be determined. In other words, an animation can be created by repeating the operations of (1) creation of key frames and (2) movement trajectory setting by their interpolation.

  Next, taking the animation shown in FIG. 14 as an example, a procedure for creating an animation according to the above-described procedure will be described. Here, as shown in FIGS. 14A to 14D, it is assumed that an animation in which an airplane flies from the front left side to the back upper right side is generated by changing the position, rotation, and scale values of the layers.

  First, in order to define a coordinate system for arranging the decoration layer, an anchor point of this coordinate system is set. In this example, as shown in FIG. 13, the anchor point is set to (X0, Y0) which is the center of the original image, and the direction of the coordinate axis is also the same as that of the original image coordinate system.

  This anchor point is fixed until the end of the animation. As a result, the position and rotation are all values according to the coordinate system of the anchor point.

  Next, on the time axis, the positions of the key frames and the movement trajectory connecting them are set. For example, as shown in FIG. 15, it is assumed that the plane moves to point A in the first frame, to point B after 2 seconds, to point C after 4 seconds, and is connected by a curve between them. Similarly, the rotation and the enlargement / reduction ratio are set for the point frame. Then, on the time axis, for a frame located between these key frames, a movement locus is set by interpolation of adjacent key frames. As a result of the above, it is possible to create an animation in which the rocket flies in the upper right corner as shown in FIG.

(Application 1)
In this example, after the position and range of the face partial area are set by template matching, the origin (anchor point) of the coordinate system that expresses decoration is set at a location corresponding to the position.

  FIG. 16A shows the decoration image data, and FIG. 16C shows the result of setting the anchor point, which is the origin of the decoration coordinate system that locally draws an elliptical orbit, at the head of the face area. Is shown.

  Next, when the face partial area of the person is detected, as shown in FIG. 16B, the anchor point is moved to the head of the face area accordingly. Further, when the size of the face area changes, the scale of the coordinate system is changed in proportion to the size of the face area.

  Further, when the orientation changes, the direction of the axis of the coordinate system is changed so as to face the same direction as the initially set direction.

  As a result, even when a human moves relative to the photographing apparatus, the decoration that always draws an elliptical orbit on the head follows the human movement.

  In addition, in multiple face images taken in different scenes, the decoration can always be moved in a trajectory proportional to the head and head size according to each face position, range, tilt, orientation, Playability will be further enhanced.

(Application example 2)
In this example, decoration information (part 2) is used to change the opacity of the composite frame decoration following the movement of the face area.

  In this example, the decoration information (part 2) shown in FIG. 10B is used. Here, the processing from step 1 to step 3 in FIG. 5 is the same as the case of using the decoration information (part 1) shown in FIG.

  However, in Step 4 to Step 6 of FIG. 5, the still image shown in FIG. 12F is arranged so as to be visible in the hatched portion of the mask of FIG. The image of the portion detected as is pasted with the size adjusted to the non-hatched portion and output.

  That is, a composite mask with a still image as a background is used as decoration arrangement information, and the center of the composite mask is arranged so as to be the center of the face according to the result of face detection. Controls the opacity setting when compositing with the input image.

  As a result, it is possible to prevent a still image from being applied to the face area when combining with another image.

(Application 3)
In this example, as shown in FIG. 17, depth information (Z axis) for the viewpoint is added to both the input image and the decoration.

  When the decoration is cut at the virtual plane 60 corresponding to the projection plane of the input image, the decoration is divided into a foreground area 61 on the foreground side of the virtual plane 60 and a background area 62 on the background side. .

  Further, the opacity of the foreground area 61 and the background area 62 is changed to be synthesized. Thereby, as shown in FIG. 18, it is possible to create an animation in which stars are turning around the face.

  For this purpose, an anchor point may be intentionally set so that the trajectory passes through the face. Then, if the background region 62 passing through the back of the face is made transparent, it is possible to add a decoration that moves in a three-dimensional manner to the input image.

(Application 4)
In this example, as shown in FIG. 19, the attribute of the decoration in the next frame is estimated from the position and orientation series for the past several frames, the current position and orientation, the past several frames, and the like. For example, if it is assumed that the decoration moves at a constant velocity, the decoration attribute can be estimated by a simple expression.

  Furthermore, when it overlaps with the body part region, the trajectory may be corrected. When a plurality of decorations are made to appear, each decoration may be moved independently of each other.

  In this example, as shown in FIG. 20, when the decoration collides with the body part region or the outline of the input image, the moving direction of the decoration is changed to a direction away from the body part region. If it does in this way, a motion which reflects when it hits a body part field can be given to decoration, and play nature can be improved.

  The image processing apparatus according to the present invention can be suitably used by being mounted on, for example, a camera-equipped mobile phone.

1 is a functional block diagram of an image processing apparatus according to an embodiment of the present invention. 1 is a block diagram of an image processing apparatus according to an embodiment of the present invention. 1 is an external view of an image processing apparatus according to an embodiment of the present invention. (A) Illustrated diagram of metafile in one embodiment of the present invention (b) Illustrated diagram of input image file in one embodiment of the present invention (c) Illustrated diagram of decoration file in one embodiment of the present invention 1 is a flowchart of an image processing apparatus according to an embodiment of the present invention. (A) Template illustration in one embodiment of the present invention (b) Template illustration in one embodiment of the present invention (c) Template illustration in one embodiment of the present invention (A) Template illustration in one embodiment of the present invention (b) Template illustration in one embodiment of the present invention (c) Template illustration in one embodiment of the present invention (A) Template illustration in one embodiment of the present invention (b) Template illustration in one embodiment of the present invention (c) Template illustration in one embodiment of the present invention Explanatory drawing of the pattern matching in one embodiment of this invention (A) Configuration diagram of decoration information in one embodiment of the present invention (b) Configuration diagram of decoration information in one embodiment of the present invention (A) User interface display example in one embodiment of the present invention (b) User interface display example in one embodiment of the present invention (c) User interface display example in one embodiment of the present invention Figure (A) Processing process explanatory diagram of the image processing apparatus in one embodiment of the present invention (b) Processing process explanatory diagram of the image processing apparatus in one embodiment of the present invention (c) Image in one embodiment of the present invention (D) Processing process explanatory diagram of the image processing apparatus according to the embodiment of the present invention (e) Processing process explanatory diagram of the image processing apparatus according to the embodiment of the present invention (f) (G) Processing process explanatory diagram of the image processing apparatus in one embodiment of the present invention Explanatory drawing of the coordinate system in one embodiment of the present invention (A) An explanatory diagram of an animation in one embodiment of the present invention (b) An explanatory diagram of an animation in an embodiment of the present invention (c) An explanatory diagram of an animation in an embodiment of the present invention (d) The present invention Explanatory drawing of animation in one embodiment Explanatory drawing of animation in one embodiment of the present invention (A) Illustration of decoration in one embodiment of the present invention (b) Illustration of anchor point in one embodiment of the present invention (c) Illustration of composite image in one embodiment of the present invention The illustration of the decoration in one embodiment of the present invention Illustration of composite image in one embodiment of the present invention The illustration of the decoration in one embodiment of the present invention Illustration of composite image in one embodiment of the present invention

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image input part 2 Control part 3 Image output part 4 Input image memory | storage part 5 Template memory | storage part 6 Detection part 7 Decoration information storage part 8 Decoration arrangement | positioning part 9 Arrangement information storage part 10 Synthesis | combination part 11 Output image storage part 12 Operation part

Claims (10)

An image input unit for obtaining an input image;
A control unit that controls image processing of the input image;
An image processing method by an apparatus comprising: an image output unit that outputs the input image subjected to image processing,
Detecting a position of a body part region from the input image including a person image, and detecting a rotation amount of the body part region in a plane of the input image and / or a rotation amount with respect to the plane ;
A step in accordance with the detected position of the body part area defining a position of the origin of the coordinate system of the decoration with moving trajectory, determining the amount of rotation of the coordinate system of the combined decoration detected rotation amount,
Determining the decoration arrangement information from the determined origin position , the determined rotation amount and the movement trajectory;
A step of based on said location information, and generates an output image by placing the decoration,
Outputting the output image. An image processing method comprising:   The image processing method according to claim 1, wherein in the outputting step, image data in which a decoration is drawn on an input image is output.   The image processing method according to claim 1, wherein in the outputting step, image data in which an input image is associated with a decoration is output.   The image processing method according to claim 3, wherein the input image and the decoration are associated by a metafile. The opacity of the decorative, a region in front of the virtual surface corresponding to the projection surface of the input image of the body part area, and the area behind the virtual surface corresponding to the projection surface of the input image in, Differently defined steps,
The image processing method according to claim 1, wherein, in the generating step, an output image is generated by arranging the decoration based on the arrangement information and the determined opacity. The decoration, when impinging on the body part area, the moving direction of the decoration, in a direction away from said body part area, to change, the image processing method according to claim 1, wherein. An input image storage unit for storing an input image including a person image;
A template storage unit for storing a template of a human body part region;
A body part region of a person is detected from an input image stored in the input image storage unit using a template of the template storage unit , and the amount of rotation of the body part region in the plane of the input image and / or Or a detection unit for detecting the amount of rotation with respect to the plane ;
A decoration information storage unit for storing decoration image information;
The origin of the decoration coordinate system having a movement locus is determined according to the detected position of the body part region, the amount of rotation of the decoration coordinate system is determined according to the detected amount of rotation, and the rotation determined as the position of the determined origin A decoration placement unit that determines the placement information of the decoration from the amount and the movement trajectory ;
And a synthesis unit which generates an output image by arranging the decoration on the basis of the arrangement information, the image processing apparatus. The image processing apparatus according to claim 7 , wherein the decoration arrangement unit determines decoration arrangement information so as to synchronize with a change in the rotation amount detected by the detection unit. The image processing apparatus according to claim 7 , wherein the decoration information is configured by associating decoration image data and a transform property. The decorative information includes decoration image data, and the mask data, association with configured, the image processing apparatus according to any of claims 7 9.

Images