JP3050209B2 - Image processing apparatus, image processing method, and recording medium storing image processing program - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention records an image processing apparatus, an image processing method, and an image processing program for drawing and outputting an object on a predetermined image based on a photographed image of a subject (particularly a person). It relates to a recording medium.

[0002]

2. Description of the Related Art There has been an amusement apparatus called a seal vending machine (so-called Print Club (registered trademark) (Purikura (registered trademark))). The sticker vending machine shoots a user who is a player with a camera, synthesizes a shot image of the user with a background image (frame image) selected in advance by the user, and issues a sticker on which the synthesized image is printed. Device.

In recent years, there have been proposed seal vending machines which not only combine a photographed image and a background image but also can draw objects (patterns and scribbles) on the combined image. In this model, the image printed on the sticker is further characterized by the object, so that the user can feel more satisfied with the issued sticker.

[0004]

Incidentally, the drawing position, shape, size, etc. of the object to be drawn need to be matched with the image of the synthesized image (captured image + background image). If there is no image, the printed image will be blurred by the object.) For this reason, conventionally, after a photographed image and a background image are combined, an object is drawn by the user himself on the combined image.

[0005] However, many users who play games at the seal vending machine enjoy the process of taking pictures, not the process of drawing objects. That is, the conventional device that allows the user to draw an object does not effectively increase the enjoyment given to the player, but has a problem in that the user is troublesome.

[0006] Further, since this process increases the time required for one play, the operating efficiency of the device is reduced,
As a result, there was a problem that sales decreased.

An object of the present invention is to detect a specific part of a subject from a photographed image of the subject and draw an object on a predetermined image based on the detection result, thereby ensuring the characterized predetermined image. An image processing apparatus, an image processing method, and a recording medium on which an image processing program can be output.

[0008]

An image processing apparatus according to claim 1, comprising: a photographing device such as a camera; and a detecting device for detecting a specific portion of the subject in a photographed image of the subject photographed by the photographing device. Image drawing means for drawing an object on a predetermined image based on the detection result by the detection means, output means for outputting an image in which the object is drawn by the image drawing means, and shooting by the shooting device
Posing selecting means for selecting the pose of the object, it is selected
Specific part determination for determining the specific part based on the pose
Setting means .

In this configuration, the detection unit detects a specific portion of the subject in the image captured by the image capturing device. The image drawing means draws an object on a predetermined image based on the detection result. Thereby, the object of the image obtained from the captured image can be drawn on the captured image. The output means outputs an image in which the object is drawn. That is, it is possible to easily obtain an image in which an image felt from a captured image is drawn as an object. In addition, the identification based on the pose of the subject
Since the part is determined, the object that fits the pose of the subject
Project can be drawn.

The image processing apparatus according to claim 2, wherein said detecting means detects said specific part a plurality of times, and a specific part corresponding to said specific part detected this time even in previous detection. Is a means for determining whether or not is detected, and if not detected the last time, processing the specific portion detected this time as noise.

[0011] In this configuration, the detection of the specific portion performed a plurality of times, if not the detected specific region corresponding to a specific site detected this time in the previous detection, processing detected this time the specific site as noise Therefore, erroneous detection of the specific portion due to the influence of disturbance light or the like can be prevented in consideration of the moving speed of the specific portion. Specifically, when a specific part (for example, a finger) is repeatedly detected at intervals of about several ms and the specific part is not detected in the vicinity of the specific part detected this time, this is simply noise. It is. Therefore, detection accuracy can be improved by processing such a specific part as noise.

In the image processing apparatus according to the third aspect, the subject is a person.

With this configuration, for example, it is possible to draw an object corresponding to the person who has become the subject.

In the image processing apparatus according to the present invention, the predetermined image is a composite image obtained by compositing the photographed image with another image.

In this configuration, for example, an appropriate background image and an image of a photographed subject are synthesized, and an object is drawn on the synthesized image.

In the image processing apparatus according to the present invention, the detecting means detects a characteristic amount such as a position of a specific part, a vertical direction, a horizontal direction, a center of gravity position, and the like in the photographed image, and based on the characteristic amount, detects the image. This is a means for determining the arrangement, shape, number, size, color, etc. of the objects to be drawn by the drawing means.

With this configuration, the object to be drawn can be changed in accordance with the image of the photographed subject.

In the image processing apparatus according to a sixth aspect, the detecting means is means for extracting a flesh color area in the photographed image and detecting the specific part from the flesh color area.

With this configuration, since the skin color region is extracted and the specific portion is detected from the extracted skin color region, the detection accuracy when the specific portion is a skin color can be improved (errors due to noise in portions other than the skin color region). No detection occurs.). For example, when the subject is a person, it is possible to improve the detection accuracy when a part (a finger, a face, or the like) where the skin of the person is exposed is a specific part.

In the image processing apparatus according to the present invention, the detecting means is means for extracting a region of the subject in the photographed image and detecting the specific part from the subject region.

In this configuration, the area of the subject is extracted and the specific part is detected from the extracted subject area, so that the detection accuracy of the specific part can be improved .

[0022]

The invention image processing apparatus according to claim 8,
A detection region setting unit configured to set an image region in which the detection unit detects the specific region with respect to the captured image.

In this configuration, since a specific portion is not detected in the entire captured image, the processing time required for detecting the position of the specific portion can be reduced.

According to a ninth aspect of the present invention, there is provided an image processing apparatus comprising a selection unit for selecting an object of an image pattern output from the image output unit, so that an image pattern to be drawn can be selected.

[0026] In the image processing apparatus according to claim 1 0, wherein the output means comprises a display device for displaying a plurality of objects during selection in the selection means.

Therefore, it is possible to select an image pattern to be drawn while looking at the screen.

The image processing apparatus according to claim 1 1, has a display device for displaying an image drawn object in said output means.

Therefore, the user can see the image on which the object is drawn.

[0030] In the image processing apparatus according to claim 1 2, wherein the output means, because it has a printing apparatus can print an image in which the object is drawn.

[0031] In the image processing apparatus according to claim 1 3,
Issue a sticker that prints the image on which the object is drawn.

[0032]

FIG. 1 is a diagram showing an outline of the present invention. A subject is photographed by the photographing device 1 (a camera or the like), and a photographed image is input to the position detection unit 2. The subject may be a person or a pet such as a dog or a cat. The position detection unit 2 is instructed on a specific part of the subject to be detected from the captured image. The specific part is a part of the subject, for example, a fingertip, a palm, a face, and the like. The image drawing means 3 receives the detection result (shape, position, direction, center of gravity, etc.) of the specific part from the position detecting means 2 and also outputs the objects (for example, stars, rings,
A ball, a user's face image photographed by the photographing device 1, and a drawing image (a predetermined image in the present invention) for drawing the object are input. Note that the object may be a part of a captured image of the imaging device 1 (for example, the above-described captured facial image of the user), or may be an appropriate image (for example, a star, Ring, ball, etc.). Similarly, the image to be drawn may be a photographed image of the photographing device 1 or may be a combined image obtained by combining the photographed image with another image.
The image may not be related to the image captured by the image capturing device 1 (for example, a display screen of a personal computer). The image drawing means 3 draws an object on an image to be drawn based on the detection result of the detection means 2. The image output means 4 outputs an image in which the object is drawn. The image output means 4 may be provided with a display device or a printing device.

Here, assuming that the predetermined image is an image of the screen of the personal computer and the object is a face image of the user photographed by the photographing device 1, the screen of the display device (image output means 4) of the personal computer displays the image of the user. A face image can be displayed. Furthermore, if the specific part is the fingertip of the user, and the position detecting means 2 outputs the amount of change in the position of the fingertip (the amount of movement of the fingertip) as the detection result,
The image drawing unit 3 converts the change amount of the fingertip as the detection result into the operation amount of the mouse (this conversion is performed by the detection unit 2
And outputs the result as a detection result. If an object (user's face) is drawn based on the result, the user's face image can be displayed as a cursor.

Further, if the present invention is applied to a seal vending machine, the object to be drawn can be changed according to the pose of the user. That is, an object having an image that matches the pose of the user can be easily drawn, and the image printed on the sticker does not become a blurred image. Further, since the object is not drawn by the user as in the related art, there is no problem of lowering the operation efficiency of the apparatus main body (a decrease in sales can be prevented).

Next, an embodiment of the present invention will be described with reference to FIG. In this embodiment, the present invention is applied to a seal vending machine (so-called print club (Purikura)).
It is applied to FIG. 2 is a block diagram showing the configuration of the seal vending machine according to this embodiment. The seal vending machine 11 is a photographing apparatus 1 for photographing a user as a subject.
2, an operation unit 13 for performing an input operation, a display device 14 for displaying operation guidance and an output image, a seal issuing machine 15 for issuing a seal on which the output image is printed, and a control device 16 for controlling the operation of the main unit. Have. Control device 1
6 is provided with the functions of the position detecting means 2 and the image drawing means 3 in FIG. The image output unit 4 in FIG. 1 corresponds to the display device 14 and the sticker issuing machine 15. The imaging device 12 is provided with a camera, a lighting device, and the like (not shown). The operation unit 13 is provided with a button for performing an input operation, a joystick, and the like. An image processing program for executing the following processing is supplied to the control device 16 by a recording medium (CD-ROM, hard disk, or the like).

The operation of the seal vending machine 11 according to this embodiment will be described below. FIG. 3 is a flowchart showing the operation of the seal vending machine according to this embodiment. The seal vending machine 11 starts the processing shown in FIG. 3 when the power is turned on. This process is repeatedly executed. When the power is turned on, the seal vending machine 11 executes a play (game) start waiting process (n1). This process is a process of waiting for insertion of a coin serving as a game price.

FIG. 4 is a flowchart showing a play start waiting process according to n1. The seal issuing machine 11 waits for a coin to be inserted into a coin insertion unit (not shown) provided in the operation unit 13 while performing the demonstration display (see FIG. 5A) on the display device 14 (n21). , N2
2). The demo display on the display device 14 may be a moving image or a still image. As shown in FIG. 5A, the number of coins currently inserted is displayed at the lower right of the demo display (0 coins in this figure). When a coin is inserted, the number of inserted coins is displayed. (Fig. 5
(Refer to (B)) (n23), it is determined whether or not the number of inserted sheets is a predetermined number (three in this embodiment) (n2).
3, n24). If it is determined in n24 that the number of inserted sheets is the predetermined number, this processing ends, and the flow proceeds to n2.

Returning to FIG. 3, after finishing the above-mentioned waiting for play start process (when three coins are inserted), the seal vending machine 11 executes a pose pattern selection process for allowing the user to select a pose pattern (n2). ). This pose pattern selection processing is substantially the same as the frame selection processing (processing of selecting a background image) in a conventional seal vending machine. Note that the seal vending machine 11 of this embodiment does not allow the user to select a frame image as in the related art, but allows the user to select a pose to be taken at the time of shooting. For this reason, the display device 14 displays a basic pattern (hereinafter, referred to as a pause pattern) of an image (an image in which an object described later is printed) to be printed on a sticker issued at the end of this processing (see FIG. 6). .

The user selects one favorite pose pattern from the pose patterns displayed on the display device 14. A pose pattern can be selected by operating a joystick provided on the operation unit 13 to position a cursor displayed on the screen at a favorite pose pattern, and operating buttons provided on the operation unit 13. . By changing the screen of the display device 14, a pose pattern other than that shown in FIG. 6 can be displayed. The screen can be changed by operating the joystick.

When the pause pattern is selected, the control device 16 performs a confirmation process for confirming the selection result (n3).
This confirmation processing is processing for confirming whether the user has erroneously selected a pose pattern. As shown in FIG. 7, the pose pattern selected at n2 is enlarged and displayed on the display device 14, and a part of the human body to be recognized (hereinafter, referred to as a recognition site (fingertip in this figure)) ( Guidance of a specific part (in the present invention) and a pose to be taken at the time of photographing is displayed. The user inputs confirmation of the pause pattern (this is fine) or cancellation (redo). When the cancellation is input in n3, the control device 16 returns to n2 and repeats the above processing, and executes the pre-shooting processing when the confirmation is input (n4).

The pre-shooting process according to n4 is a process of reading parameters preset for the pause pattern determined at n3 from a recording medium such as a CD-ROM or a hard disk and storing the parameters in a memory (parameters are: It is recorded for each pause pattern.) The parameters include, for example, a frame image to be synthesized as a background of a photographed image (person image), and data for determining the type, number, size, drawing position, and the like of an object to be drawn based on a detection result of a recognized part. It is included.

When the control device 16 completes the pre-shooting process,
Proceeding to n5, an image creation process is executed. This image creation process is a process of actually photographing a user with the photographing device 12 and creating an image to be printed on a sticker. Also, the control device 1
Reference numeral 6 turns on the illumination device provided in the photographing device 12 at the start of the image creation process, and turns off the light at the start of the later-described print process (at the start of the process of n11).

FIG. 8 is a flowchart showing the image creation processing according to n5. The control device 16 controls the imaging device 12
The captured image obtained by capturing the user is read (n31), and an image area of the user (hereinafter, referred to as a subject area) is extracted from the captured image (n32). Normally, seal vending machine 11
Has a curtain arranged behind the user. Therefore, the subject area can be extracted by removing pixels having a color substantially uniform with the color of this curtain (for example, blue) from the captured image. Thus, the subject area shown in FIG. 9B can be extracted from the captured image (original image) shown in FIG. 9A. Specifically, the above-described subject area is extracted by removing pixels satisfying the following [Equation 1] as background pixels.

[0044]

(Equation 1) Note that if α in [Equation 1] is set to be relatively small (large), the amount of the background pixel to be removed included in the extracted subject area increases, and conversely, α becomes tighter (smaller).
, The number of pixels in the subject area removed as background pixels increases. The subject area extracted in this process is
As will be described later, α is set slightly looser so as to be used as a photographed image of the user (pixels in the subject area are not removed as background pixels). The control device 16 creates a composite image in which the subject region extracted here is superimposed on the base image (frame image) read into the memory at n4 (n33).

The control device 16 executes a recognition part detection process for detecting the recognition part by processing the photographed image (n3).
4). The content of the recognition site detection processing according to n34 differs depending on the pose pattern determined at n3. This is because the recognition part to be recognized from the captured image differs depending on the selected pose pattern. n34
The details of the recognition part detection processing according to the above will be described later.
When detecting the feature amount such as the position, orientation, and center of gravity of the recognition part in n34, the control device 16 executes an object drawing process of drawing the object on the composite image created in n33 based on the detection result (n35). . The image on which the object is drawn (hereinafter, referred to as a print candidate image) is displayed on the display device 14 (n36).

Hereinafter, the recognition part is a fingertip (see FIG. 10),
The butted fingers (see FIG. 11) and the palm (FIG. 12)
Palms facing each other (see FIG. 13), grasping hands (see FIG. 14), between faces (see FIG. 15),
In crossed hand (see FIG. 16), a is seven modes, will be described with the above recognition site detection process and the object drawing process (n34, processing n35).

[Mode A] In the mode A shown in FIG. 10, the recognition site is the fingertip of the user. The control device 16 captures a captured image and executes a skin color region extraction process of extracting a skin color region from the image. FIG. 17 is a flowchart showing the skin color region extraction processing, and FIG. 18 is a view for explaining this processing. The control device 16 sets a face presence area where the user's face is photographed (see FIG. 18B) for the captured photographed image (see FIG. 18A) (n41).
As described above, since the pose at the time of photographing is specified by the pose pattern determined at n3, the control device 16 can estimate the area where the user's face is photographed substantially accurately.

The control device 16 performs a flesh color sample extraction process for extracting flesh color pixels with a certain width in the face existence region estimated at n41 (n42) (FIG. 18).
(C)). In this process, the RGB values of the background color pixels in the above [Equation 1] are converted into the RGB values of the general human skin color.
This is a process of setting a value and α to be slightly larger, and extracting a pixel satisfying the condition of [Equation 1]. The control device 16 performs statistical processing on the skin color pixels extracted in n42, and recognizes the most frequent color as the skin color of the user (n43).

Next, the control device 16 sets a recognition part detection area based on a parameter indicating an area where the user's hand is photographed in the photographed image (n44). As described above, according to the pose pattern determined at n3,
Since the pose is specified for the user at the time of shooting, not only the above-mentioned face but also the area where the user's hand is shot can be estimated substantially accurately. The control device 16 stores this parameter in the memory in the process of n4. More specifically, FIGS. 19A and 19B show the captured image for each pose pattern.
Are set (recognized region detection areas indicated by hatching) (set for each selected pose pattern). FIG.
FIG. 9A shows an example in which the substantially center of the captured image is a recognition site detection area, and FIG. 19B shows an example in which the lower side of the captured image is a recognition site detection area. Note that an area other than that shown in FIG. 19 may be set as the recognition site detection area. The control device 16 extracts the skin color region in the recognition site detection region (n45) (see FIG. 18D). In this process, the RGB value of the reference color in the above [Equation 1] is set to n43.
This is a process for extracting a region of a pixel that satisfies the condition of [Equation 1] by setting the skin color recognized in the above. The above processing is the skin color area detection processing.

By the above-described skin color region detection processing, FIG.
The skin color area shown in FIG. 20B is extracted from the captured image shown in FIG. The control device 16 detects a finger, which is a recognition part, from the skin color region extracted here. The detection of the finger area is performed by detecting a skin color whose width W is equal to or less than Wf (a preset maximum value of the finger thickness) and whose length L is equal to or greater than Lf (the minimum value of the preset finger length). All the regions are detected, and furthermore, among the detected flesh color regions, a region where one is connected to a flesh color region having a width of Wd or more is detected, and this region is detected as a finger (see FIG. 20C). .

Through the above processing, the region where the finger, which is the recognition site, is located in the photographed image is detected. Here, the skin color differs depending on the user who is the subject, and also changes depending on the shooting environment (brightness of lighting and the like). Therefore,
If a skin color region is extracted from a captured image using a preset RGB value of a skin color reference value, detection accuracy is poor, and there is a possibility that a recognition part cannot be detected from the extracted skin color region. For example, if the user's skin color is a preset skin color reference RG
If the value deviates from the B value, a region where the user's skin is exposed cannot be accurately extracted, and a problem that a finger as a recognition site cannot be detected occurs.

On the other hand, the apparatus of the present embodiment
Since the skin color of the user is first recognized and the skin color area is extracted based on the recognition in the processing of steps 43 to 43, the area where the user's skin is exposed can be reliably extracted. Therefore, the user's finger (recognition site) in the captured image can be reliably detected. In addition, the recognition time is not detected for the entire captured image, but a recognition area detection area is set in n44, and the recognition area is detected in this area, so that the processing time is reduced.

Also, instead of the above processing, the area of the finger can be detected from the photographed image using the subject area extraction processing described in n32. Note that, in this case, α in the above [Equation 1] is set to be tight (slightly smaller). As a result, the extracted subject area does not include the background area. Then, in the extracted subject region, the width W is equal to or smaller than Wf (the preset maximum value of the finger thickness) and the length L is Lf (the minimum value of the preset finger length). ) All of the above areas are detected, and if one of the detected areas is connected to a subject area having a width of Wf or more, this area is detected as a finger area. If α is set to a small value, a part of the background region is extracted as the subject region, and the subject region becomes thicker than the actual one. For this reason, the width of the finger region is equal to or larger than Wf, and the possibility that the finger region cannot be detected from the subject region increases. In the above description, the subject area is extracted from the entire captured image.
The processing time can be shortened by setting the recognition region detection region and detecting the finger region in the recognition region setting region as described in 4.

Further, the finger area in the photographed image may be detected by the above two processes, and the finger area detected only on one side may be removed as noise. By doing so, the detection accuracy of the finger region from the captured image can be improved.

Further, the detection accuracy can be improved by using the previous detection result. In the seal vending machine 11 of this embodiment, the detection of a specific part (for example, a finger) is repeatedly performed at intervals of about several ms. Therefore, in the vicinity of the currently detected finger area, the finger area is detected at the previous detection. If not, this is considered to be noise affected by disturbance light or the like. That is, the finger does not actually move significantly within a few ms, and if no noise is detected, the finger is detected at the previous detection. That is, by removing noise using the previous detection result, it is possible to improve the detection accuracy of the recognition site. For example, when finger regions are detected at three places indicated by hatching in FIG. 21A, if the previous detection result is FIG. 21B, the upper right recognition is performed as shown in FIG. 21C. It is assumed that the area is processed as noise, and finally two areas indicated by hatching in FIG. 21C are areas of the finger detected this time. Thereby, the detection accuracy of the recognition part can be further improved.

When the detection of the finger in the photographed image is completed in any of the above-described processes, the control device 16 determines a valid finger from the detected fingers. This is because even if the subject is one person, if both hands are spread, the above processing can be up to 10
When three fingers are detected (10 fingers are detected) and an object is drawn for each finger, n11 described later is used.
There are many objects in the image that has been subjected to the print processing, and this is not a preferable image. Therefore, in the apparatus of this embodiment, which of the detected fingers is to be made effective is determined based on the position on the captured image. For example, if two detected recognition sites are valid, the rightmost and topmost position is
Conditions such as enabling the two at the leftmost and uppermost positions are determined. The control device 16 executes the following processing for the ones made effective in the above processing.

The control unit 16 determines that 2
The characteristic amount of one finger is detected. The type of the feature amount detected here is determined in advance. In the mode A, the position of the fingertip, the direction of the finger, and the like are detected as feature amounts. The process of detecting the direction of the finger will be described with reference to FIG. With respect to the finger detected in the above-described processing, an intermediate point in the width direction of the finger is detected at predetermined intervals in the length direction (see FIGS. 22A and 22B). Then, the inclination is obtained from the shift between the adjacent intermediate points (see FIG. 22C), and the average of the obtained inclination is detected as the direction of the finger. The tip of the finger is detected as a fingertip (FIG. 22A).
reference).

Next, the control section 16 executes an object drawing process for drawing a light beam as an object on the composite image created in the process of n33. At this time, the start point of the object is determined based on the detected position of the fingertip, and the end point (the direction in which the light rays extend) is determined according to the direction of the finger. In addition, color distribution and the like in the captured image are detected,
The color (transparency) of the object is also detected. FIG. 23 is a flowchart showing the details of this process. With the above processing, the print candidate image is displayed on the display device 14 in n36.

[Mode B] Mode B shown in FIG. 11 is a mode in which the recognition tip is set to the fingertip (or the fingertip facing). The control device 16 detects the finger in the captured image by executing the skin color region extraction processing as in the case of the mode A described above. Then, of the extracted finger regions, a region whose length is equal to or longer than Ld (a region detected when two fingers are in contact with each other, and Ld is a general human finger on the captured image). (About 1.5 times the length of a fingertip) or a region of two finger regions where the fingertips face each other (a region detected when there is a small gap between the fingertips) (FIG. 24).
(See (A) and (B)).

Note that the position of the recognized part can be detected by the above-described subject region extraction processing. Further, as described in mode A, both processes may be executed to improve the detection accuracy. Further, the detection accuracy may be improved using the past information shown in FIG. In this mode B, one of the recognition parts detected in the above processing is made valid.

If the finger area is a continuous finger area, the control section 16 detects an intermediate point of the finger area as a contact point of the fingertip, and if the fingertip faces each other (if the fingertip faces), The intermediate point of the gap (if any) is set as the detection position. In addition, the length L of the two fingers touching (or facing) with each other is also detected as a feature value (FIG. 2).
4 (C)). The flame is drawn as an object above the contact point (or the middle point) of the fingertip according to the detected feature amount. As a result, the print candidate image is displayed on the display device 14. FIG. 25 is a flowchart showing the details of the object drawing process according to the mode B.

[Mode C] Mode C shown in FIG. 12 is a mode in which the recognition part is a palm. The control unit 16
The palm in the photographed image is extracted by the above-described skin color extraction processing or subject area extraction processing (see FIG. 26), and the effective palm is determined from the palm detected here.

The control device 16 detects the effective palm features (position, palm size, orientation, etc.) and, based on the detected features, determines the object to be drawn on the composite image formed in n33. The size, number, color, arrangement, etc. are determined. The object is drawn based on this. As a result, the print candidate image is displayed on the display device 14. FIG. 27 is a flowchart showing details of the object drawing process according to the mode C.

[Mode D] Mode D shown in FIG. 13 is a mode in which recognition portions are palms facing each other. The control unit 16 detects two opposing palms in the captured image by the above-described skin color extraction processing or subject area extraction processing (see FIGS. 28A and 28B). Then, the characteristic amounts of the facing palms (position, palm size, orientation, gaps of the facing hands, the center of gravity of the gaps, etc.) are detected. The size, color, and the like of the object to be drawn are determined. Based on this, drawing of the object on the composite image is executed, and the print candidate image is displayed on the display device 14. FIG. 29 is a flowchart showing details of the object drawing process according to the mode D.

[Mode E] Mode E shown in FIG. 14 is a mode in which the recognition site is a gripped hand. Control device 1
6 corresponds to the current image as shown in FIG.
(A)), performing both the above-described skin color area extraction processing (FIG. 30 (B)) and subject area extraction processing (FIG. 30 (C)).
The difference between the regions extracted by these two processes is calculated, and in the difference image (FIG. 30 (D)), the lost region is detected as the hand holding the region where the corresponding region does not remain at the top. It should be noted that, for the face region that is detected together in the above two processes and disappears in the difference image (FIG. 30D), a corresponding region remains above (effect of hair).

The control device 16 detects the position of the center of gravity and the like as the characteristic amount of the grasped hand, and determines the size, color, and the like of the object to be drawn on the composite image formed in n33 based on the characteristic amount detected here. It is determined. Based on this, drawing of the object on the composite image is executed, and the print candidate image is displayed on the display device 14. This mode D
For details of the object drawing process according to FIG.
1 shows the flowchart.

[Mode F] Mode F shown in FIG. 15 is a mode in which the recognition site is between two faces. In this mode F, the processing shown in FIG. 30 is performed in the same manner as in the mode E described above, and in the difference image (FIG. 30 (D)), an area where a corresponding area remains above the lost area is detected as a face. I do. Then, the control device 16 detects the distance between the two faces, the position of the center of gravity, the size of the face, and the like as the feature amounts, and draws them on the composite image formed in n33 based on the detected feature amounts. The size, color, number, etc. of the objects are determined. Based on this, drawing of the object on the composite image is executed, and the print candidate image is displayed on the display device 14. FIG. 32 is a flowchart showing the details of the object drawing process according to the mode D. In this mode F, the number of objects (heart marks) increases as the distance between the detected two faces becomes narrower.
Is drawn.

[Mode G] Mode G shown in FIG. 16 is a hand in which the recognition part is assembled (prayer in a prayer). The control device 16 executes the above-described skin region extraction processing,
The upper skin color region is detected as a face, and a hand forming a skin color region located immediately below the face (see FIG. 33). Then, the control device 16 detects the position, the position of the center of gravity, and the like as the detected feature amount of the assembled hand, and based on the detected feature amount, determines the size of the object to be drawn on the composite image formed in n33. Then, the color, the number, etc. are determined, and the object is drawn based on the determined color, number, and the like. As a result, the print candidate image is displayed on the display device 14. FIG. 34 is a flowchart showing the details of the object drawing process according to the mode G.

As described above, the control device 16 recognizes a part of the human body such as the fingertip, the butted finger, the palm, the facing palm, the grasped hand, the face, and the crossed hands in the captured image. The object is drawn on the composite image formed in n33 based on the feature amount of a part of the human body recognized here. The image on which the object is drawn is displayed on the display device 14 as a print candidate image.

Returning to FIG. 3, when the print candidate image is displayed on the display device 14 in the above processing, the control device 16 determines whether or not the OK button has been operated (n6). Return. That is, the above image creation processing is repeatedly executed until the user presses the OK button. This allows the user to change the pose while looking at the display unit 14 so as to obtain a favorite print candidate image.

When the control device 16 determines that the OK button has been pressed in n6, it counts down and displays a print confirmation screen shown in FIG.
n8).

Here, when the user inputs a redo,
The control device 16 determines whether or not the object is drawn (n9), and if the object is drawn, n
Returning to step 5, the above processing is repeated. On the other hand, if the object is not drawn, the pose pattern determined in n3 is displayed (see FIG. 7), and the user is guided in the pose at the time of shooting (n10). Thereafter, the process returns to n5 and the above processing is repeated. Note that the object is not drawn because a pose different from the pose pattern selected by the user was taken, so that the recognition site could not be detected.

When the user presses the enter button at n8, the seal vending machine 11 displays the display device 1 on the seal issuing machine 15.
Then, a sticker on which the image displayed in No. 4 is printed is issued, and the process ends (n11). Note that the image shown in FIG. 36 is displayed on the display device 14 during printing of the sticker.

As described above, in the seal vending machine 11 according to this embodiment, when the user changes the pose, not only the user's pose but also the drawn object changes in the output image. Can be performed, and the user is not emptied even if the player repeatedly plays. In addition, since the object is automatically drawn instead of being drawn by the user, the time required for one play does not become long, and a decrease in the operation rate of the apparatus can be prevented.

[0075]

As described above, according to the present invention, the position of a specific part of a subject is detected in a photographed image photographed by a photographing device, and an appropriate image is formed on a predetermined image based on the detected position. The object is drawn. Then, an image in which this object is drawn is output. Therefore, an image in which an appropriate object is drawn can be output without causing the operator (user) to draw the object. Moreover, it is detected by the pose of the subject
The specific part to be determined
You can draw objects that fit the pose of the object
Wear. In addition, the detection of the specific part is performed plural times,
The characteristic corresponding to the specific part detected this time in the detection of
If the fixed site has not been detected, the identification
Since the part is processed as noise, the specific part
Considering the moving speed of
Erroneous detection can be prevented, and the detection accuracy is improved.

Further, if the present invention is applied to a sticker vending machine, it is possible to prevent a play time from being lengthened and to print an image characterized by an object on a sticker. Therefore, it is possible to prevent a decrease in the operation rate of the apparatus and improve sales.

[Brief description of the drawings]

FIG. 1 is a block diagram showing an outline of the present invention.

FIG. 2 is a block diagram showing a configuration of a seal vending machine according to the embodiment of the present invention.

FIG. 3 is a flowchart showing processing of the seal vending machine according to the embodiment of the present invention.

FIG. 4 is a flowchart showing a play start waiting process.

FIG. 5 is a diagram showing a demonstration display screen on the display device.

FIG. 6 is a diagram showing a pose pattern selection screen.

FIG. 7 is a view showing a confirmation screen of a pose pattern.

FIG. 8 is a flowchart illustrating an image creation process.

FIG. 9 is a diagram showing a subject area extraction process.

FIG. 10 is a diagram showing a pause pattern in mode A.

FIG. 11 is a diagram showing a pause pattern in mode B.

FIG. 12 is a diagram showing a pause pattern in mode C.

FIG. 13 is a diagram showing a pause pattern in mode D.

FIG. 14 is a diagram showing a pause pattern in a mode E.

FIG. 15 is a diagram showing a pose pattern in mode F.

FIG. 16 is a diagram showing a pause pattern in mode G.

FIG. 17 is a flowchart showing a skin color region extraction process.

FIG. 18 is a diagram showing a skin color region extraction process.

FIG. 19 is a diagram showing a recognition site detection region for recognizing a specific site.

FIG. 20 is a diagram illustrating a process of detecting a finger from a captured image.

FIG. 21 is a diagram illustrating a process of removing noise using a previous detection result.

FIG. 22 is a diagram illustrating a process of detecting a finger direction as a feature amount.

FIG. 23 is a flowchart showing an object drawing process when a pose pattern in mode A is selected.

FIG. 24 is a diagram illustrating a process of detecting a butted finger from a captured image.

FIG. 25 is a flowchart showing an object drawing process when a pose pattern in mode B is selected.

FIG. 26 is a diagram illustrating processing for detecting a palm from a captured image.

FIG. 27 is a flowchart showing an object drawing process when a pose pattern in mode C is selected.

FIG. 28 is a diagram illustrating a process of detecting a facing palm from a captured image.

FIG. 29 is a flowchart showing an object drawing process when a pose pattern in mode D is selected.

FIG. 30 is a diagram illustrating a process of detecting a hand gripped and a space between two faces from a captured image.

FIG. 31 is a flowchart showing an object drawing process when a pose pattern in mode E is selected.

FIG. 32 is a flowchart showing an object drawing process when a pose pattern in mode E is selected.

FIG. 33 is a diagram illustrating a process of detecting a crossed hand from a captured image.

FIG. 34 is a flowchart showing an object drawing process when a pose pattern in mode F is selected.

FIG. 35 is a diagram showing a print confirmation screen.

FIG. 36 is a diagram illustrating a display image during a printing process.

[Explanation of symbols]

 Reference Signs List 1-Photographing device 2-Position detecting means 3-Image drawing means 4-Image output means 11-Seal vending machine 12-Photographing device 13-Operation unit 14-Display device 15-Seal issuing machine 16-Control device

────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Toshiyuki Kozuru 10 Okado Todocho, Ukyo-ku, Kyoto-shi, Kyoto Omron Corporation (56) References JP-A-5-328213 (JP, A) JP-A-8- 153197 (JP, A) JP-A-7-123403 (JP, A) JP-A-9-33217 (JP, A) JP-A-63-17411 (JP, A) Registered utility model 3048141 (JP, U) (58) ) Surveyed field (Int.Cl. ⁷ , DB name) H04N 7/18 G06T 1/00

Claims (21)

(57) [Claims] A photographing device such as a camera; a detecting means for detecting a specific part of the subject in a photographed image of the subject photographed by the photographing device; Image drawing means for drawing an object, output means for outputting an image in which the image drawing means draws the object, and a port for selecting a pose of a subject to be photographed by the photographing device.
Feature selection means for determining the specific part based on the selected pose.
An image processing apparatus comprising: a fixed part determining unit . 2. The method according to claim 1, wherein the detecting unit performs the detection of the specific part a plurality of times to determine whether a specific part corresponding to the specific part detected this time has been detected in the previous detection. if not, claim 1 is means for processing the detected this time the specific site as noise
An image processing apparatus according to claim 1. 3. The image processing apparatus according to claim 1, wherein the subject is a person. Wherein said predetermined image, the image processing apparatus according to claim 1 wherein a captured image and the other image and the synthesized composite image. 5. The detecting means detects a characteristic amount such as a position of a specific part, a vertical direction, a horizontal direction, a center of gravity position and the like in a captured image, and arranges an object to be rendered by the image rendering means based on the characteristic amount. shape, number, size, image processing apparatus according to any one of claims 1 to 4 which is a means for determining the color or the like. Wherein said detecting means, said skin color region extracted in the captured image, the image processing according to any one of claims 1 to 5 which is a means for detecting said specific part from the skin color region extracted here apparatus. Wherein said detecting means, said extracting region of the subject in the captured image, <br/> claim 1-5 is means for detecting said specific site from the extracted object area, where An image processing apparatus according to claim 1. 8. A detection area setting means for setting an image area in which the detection means detects the specific part.
8. The image processing apparatus according to any one of claims 7 to 7 . 9. The image processing apparatus according to any one of claims 1 to 8 with a selection means for selecting the type of object that renders an image pattern output from the image output unit. 10. The image processing apparatus according to claim 9, wherein said output means has a display device for displaying a plurality of types of said objects when selected by said selection means. Wherein said output means, an image processing apparatus according to any one of claims 1 to 1 0 having a display device for displaying an image in which the object is drawn in the image drawing means. 12. The method of claim 11, wherein output means, an image processing apparatus according to any one of claims 1 to 1 1 having a printing device. Wherein said printing apparatus, an image processing apparatus according to claim 1 2 is an apparatus that prints an image on the seal. 14. A method for detecting a specific part of a subject in a photographed image obtained by photographing the subject with a photographing device such as a camera, and drawing an object on a predetermined image based on the detected specific part of the subject; an image processing method for outputting an image drawing objects, based on the pose of the object is selected, the specific site
Image processing method to be determined. 15. The method according to claim 15, wherein the detection of the specific portion is performed a plurality of times.
15. The method according to claim 14, wherein it is determined whether a specific part corresponding to the specific part detected this time is also detected in the previous detection, and if not detected last time, the specific part detected this time is processed as noise . Image processing method. 16. The object according to claim 14 , wherein the subject is a person.
Or the image processing method according to 15 . 17. The image processing method according to claim 14 , wherein the predetermined image is a combined image obtained by combining the photographed image and another image. 18. A first method for detecting a specific part of a subject in a photographed image obtained by photographing the subject with a photographing device such as a camera.
A second step of drawing an object in a predetermined image based on the position of the specific part of the subject detected here; a third step of outputting an image in which the object is drawn ; A fourth step of selecting a pose, and detecting in the first step based on the selected pose
And a fifth step of determining a specific part to be processed. 19. The method according to claim 19, wherein the first step is performed a plurality of times.
A fourth step of determining whether or not a specific part corresponding to the specific part detected this time is also detected in the previous detection, and if not detected last time, processing the specific part detected this time as noise; Claim with
19. A recording medium recording the image processing program according to 18 . 20. The subject according to claim 18 , wherein the subject is a person.
Or a recording medium on which the image processing program according to 19 is recorded. 21. The recording medium according to claim 18 , wherein the predetermined image is a combined image obtained by combining the photographed image and another image.