JP2013186838A - Generation device, generation program, and generation method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable behavior of a photographed person to be easily grasped while suppressing violation of the person's portrait right and privacy.SOLUTION: A generation device 10 comprises an extraction unit 15b, a change unit 15c, and a generation unit 15d. The extraction unit 15b extracts a feature point from an image area of a person in an image. The change unit 15c changes the image area of the person to another image. The generation unit 15d overlaps the other image and the feature point to generate an image.

Description

  The present invention relates to a generation device, a generation program, and a generation method.

  Cameras are installed in various places such as offices, shopping centers, and shopping streets, and images taken with the cameras are used for various purposes such as crime prevention, marketing, and services. For example, an image captured by a camera installed in a shopping center, a shopping street, or the like is used for analysis of what route the user has moved and what product has been picked up. The analysis result is used to improve the display method of the product.

  Here, infringement of portrait rights and privacy of a person photographed by a camera may be a problem. Therefore, conventionally, there is a technique of extracting a face image of a person included in a photographed image and applying a mosaic to the extracted face image.

  Conventionally, there is a technique for extracting an entire area of a person included in a photographed image and applying a mosaic to the extracted entire area of the person.

JP 2000-000216 A JP 2004-062560 A

  However, the above-described conventional technology has a problem that it is difficult to grasp the behavior of a person while suppressing infringement of portrait rights and privacy of the photographed person.

  For example, in the conventional technique for applying a mosaic to a face image, there may be a case where an individual can be identified from the clothes and body shape of the person when the mosaic is applied to the face image. For this reason, the conventional technique for applying a mosaic to a face image may infringe the portrait rights or privacy of the photographed person.

  In addition, in the conventional technique of applying a mosaic to the entire area of the person, it is difficult to grasp the action of the person because the mosaic is applied to the entire area of the person.

  The disclosed technology has been made in view of the above, and a generation device, a generation program, and a generation that can easily grasp a person's behavior while suppressing infringement of portrait rights and privacy of a photographed person It aims to provide a method.

  The generation device disclosed in the present application includes an extraction unit, a change unit, and a generation unit. The extraction unit extracts feature points from an image area of a person in the image. The changing unit changes the image area of the person to another image. The generation unit generates an image by superimposing other images and feature points.

  According to one aspect of the generating device disclosed in the present application, it is possible to easily grasp the behavior of a person while suppressing infringement of portrait rights and privacy of the photographed person.

FIG. 1 is a diagram illustrating an example of a system configuration of a system including a generation apparatus according to the first embodiment. FIG. 2A is a diagram illustrating an example in which an image in which a person and a background are painted black and a person's feature points are superimposed on the person is displayed in time series. FIG. 2B is a diagram illustrating an example in which an image in which a person and a background are painted black and a person's feature points are superimposed on the person is displayed in time series. FIG. 3 is a diagram illustrating an example of a frame included in the image data. FIG. 4 is a diagram for explaining an example of a person region detection method. FIG. 5 is a diagram for explaining an example of processing executed by the generation apparatus according to the first embodiment. FIG. 6 is a schematic diagram illustrating an example of a process executed by the generation apparatus according to the first embodiment. FIG. 7 is a diagram illustrating an example of processing executed by the generation apparatus according to the first embodiment. FIG. 8 is a flowchart illustrating the procedure of the generation process according to the first embodiment. FIG. 9 is a flowchart illustrating a procedure of another generation process according to the first embodiment. FIG. 10 is a diagram illustrating an example of a system configuration of a system including the generation apparatus according to the second embodiment. FIG. 11 is a schematic diagram illustrating an example of a process executed by the generation apparatus according to the second embodiment. FIG. 12 is a schematic diagram illustrating an example of a process executed by the generation apparatus according to the second embodiment. FIG. 13 is a flowchart illustrating the procedure of the generation process according to the second embodiment. FIG. 14 is a diagram illustrating an example of a system configuration of a system including the generation apparatus according to the third embodiment. FIG. 15 is a flowchart illustrating the procedure of the generation process according to the third embodiment. FIG. 16 is a diagram illustrating an example of a system configuration of a system including the generation apparatus according to the fourth embodiment. FIG. 17 is a flowchart illustrating the procedure of the generation process according to the fourth embodiment. FIG. 18 is a diagram illustrating a computer that executes a generation program.

  Hereinafter, embodiments of a generation apparatus, a generation program, and a generation method disclosed in the present application will be described in detail with reference to the drawings. The embodiments do not limit the disclosed technology.

[System Configuration of System Having Generating Device 10]
A generation apparatus according to an embodiment will be described. FIG. 1 is a diagram illustrating an example of a system configuration of a system including a generation apparatus according to the first embodiment. The system 1 includes a camera 20 and a generation device 10. The camera 20 is installed in a place where a person can be photographed. For example, the camera 20 is installed in places such as an office, a shopping center, and a shopping street. Images taken by the camera 20 are used for various purposes such as crime prevention, marketing, and services. The camera 20 outputs the captured video image to the generation device 10 for each frame.

  The generation apparatus 10 extracts feature points from an image area of a person in each image of a video photographed by the camera 20. Further, the generation device 10 changes the image area of the person to another image, for example, an image painted in a predetermined color. And the production | generation apparatus 10 superimposes another image and a feature point, and produces | generates an image. Subsequently, the generation device 10 controls display on the display unit 12 described later so that the generated image is displayed in time series.

  Here, when the image generated in this way is displayed in chronological order, the person can grasp what action the person in the image took from the display content. This is because, when a plurality of feature points arranged in two dimensions are moving, a person projects the feature points on each three-dimensional plane from the movement of the feature points, thereby This is because it can be converted into the movement of a three-dimensional person. For example, even if a person's body is expressed by about 10 feature points, the person determines which point is the hand part and which point is the foot part from the movement of each point. Can understand how is moving. Therefore, even if the entire person is an image converted into a predetermined color, for example, a black painted image, the feature point is displayed so that the person can grasp how the person moves. can do.

  FIGS. 2A and 2B are diagrams illustrating an example in which an image in which a person and a background are painted black and a person's feature points are superimposed on the person is displayed in time series. From the images shown in the examples of FIGS. 2A and 2B, it can be understood how the person is moving.

  As illustrated in FIG. 1, the generation device 10 includes an input unit 11, a display unit 12, a communication unit 13, a storage unit 14, and a control unit 15.

  The input unit 11 inputs various information to the control unit 15. For example, the input unit 11 receives various instructions from the user and inputs the received instructions to the control unit 15. As an example of the device of the input unit 11, there is a device that accepts a user operation such as a mouse or a keyboard.

  The display unit 12 outputs various information. For example, the display unit 12 displays an image in which a person region is painted in a predetermined color and an image in which a person's feature points are overlapped in time series under the control of a display control unit 15e described later. An example of the device of the display unit 12 is a liquid crystal display.

  The communication unit 13 is an interface for performing communication. For example, when the communication unit 13 receives a video of a moving image from the camera 20 for each frame, the communication unit 13 outputs each received frame to the generation device 10.

  The storage unit 14 stores various information. For example, the storage unit 14 stores image data 14a.

  The image data 14a is data generated by a generation unit 15d described later. The image data 14a is data of a moving image including each frame in which an image in which a person area is filled with a predetermined color and a person feature point are superimposed.

  FIG. 3 is a diagram illustrating an example of a frame included in the image data. In the example of FIG. 3, the region 22 of the person 21 indicates a frame of an image 24 in which an image painted in a predetermined color, for example, blue, and a plurality of feature points 23 of the person are superimposed. When such an image is displayed in chronological order, the person can grasp how the person 21 is moving.

  The storage unit 14 is, for example, a semiconductor memory device such as a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 14 is not limited to the above type of storage device, and may be a RAM (Random Access Memory) or a ROM (Read Only Memory).

  The control unit 15 has an internal memory for storing programs defining various processing procedures and control data, and executes various processes using these. As shown in FIG. 1, the control unit 15 includes an acquisition unit 15a, an extraction unit 15b, a change unit 15c, a generation unit 15d, and a display control unit 15e.

  The acquisition unit 15a acquires the frame output from the communication unit 13. The acquired frame is used for processing in each unit of the extraction unit 15b and the change unit 15c.

  The extraction unit 15b extracts feature points from an image area of a person in the image. Explaining with a specific example, the extraction unit 15b extracts feature points from the image area of a person in one frame for each frame acquired by the acquisition unit 15a, that is, for each frame of a moving image obtained by photographing a person. To do.

  For example, the extraction unit 15b first detects a person region that is a person region (image region) for each frame acquired by the acquisition unit 15a. Here, an example of a person region detection method will be described. FIG. 4 is a diagram for explaining an example of a person region detection method. As shown in the example of FIG. 4, an image (background image) in the absence of the person 21 is stored in the storage unit 14 in advance, and the extraction unit 15b compares the luminance values of the pixels of the frame and the background image. A set of pixels whose luminance value difference exceeds a threshold value can be detected as the person region 22. In addition, the extraction unit 15b can also detect a person region by a method described in Japanese Patent Application Laid-Open No. 2006-077982. That is, the extraction unit 15b detects the face of a person from the image of the frame, detects the range of the body part of the person in the image from the size and position of the detected face, the position and orientation of the camera, and the like. It is also possible to detect the area of the face part and the body part as a person area. Further, the extraction unit 15b can also detect a person region using another known technique.

  And the extraction part 15b extracts the feature point in a person area. A method for extracting feature points will be described with an example. For example, the extraction unit 15b can extract feature points using edge points with high contrast on the image. This feature point extraction method is described in, for example, ““ Robot Vision ”pp51-76, 4.2 Edge Detection Method, Masahiko Taniuchi”. That is, the extraction unit 15b can extract a feature point by converting an image into an edge image by a Sobel operator or the like, and leaving only those whose edge strength is larger than the surrounding area within a certain range.

  The extraction unit 15b can extract points similar to the template image as feature points. This feature point extraction method is described in, for example, ““ Robot Vision ”pp104-106, 6.1.1 Template Matching Method, Masahiko Taniuchi”. That is, the extraction unit 15b uses a predetermined template image having the same brightness to calculate a similarity with the template image by normalization correlation for all areas of the person area, and the similarity exceeds a predetermined value. Can be detected as feature points.

  The extraction unit 15b can also calculate a SHIFT feature value from the image, and detect a point in the person area where the calculated SHIFT feature value exceeds a threshold value as a feature point. This feature point extraction method is described, for example, in “D. Lowe.“ Distinctive image features from scale invariant keypoints ”, Proc. Of International Journal of Computer Vision (IJCV), 60 (2), pp.91-110, 2004. "It is described in.

  Since feature points are extracted by various methods in this way, the extraction unit 15b compares the case where a person's eyes, nose, mouth, and the like that require processing time to be extracted as feature points are extracted as feature points. Thus, feature points can be easily extracted. Note that the extraction unit 15b may extract a person's eyes, nose, mouth, and the like as feature points.

  FIG. 5 and FIG. 6 are diagrams for explaining an example of processing executed by the generation apparatus according to the first embodiment. The example of FIG. 5 shows a case where the person area 22 of the person 21 is detected by the extraction unit 15b in one frame. As shown in the example of FIG. 5, when the person area 22 is detected, the extraction unit 15 b extracts the feature points 23 in the person area 22.

  The changing unit 15c changes the image area of the person in the image to another image. If it demonstrates with a specific example, the change part 15c will change the image area of the person in 1 frame into another image for every flame | frame acquired by the acquisition part 15a. This makes it difficult to identify a person in the image. Therefore, according to the generation device 10, it is possible to generate image data in which a portrait right or privacy infringement of a photographed person is suppressed.

  For example, the changing unit 15c changes the person area detected by the extracting unit 15b to an image filled with a predetermined color. Here, the changing unit 15c can change the person area to an image painted in a color such as blue, black, or red. Note that any color can be adopted as the color of the filled image. FIG. 7 is a diagram illustrating an example of processing executed by the generation apparatus according to the first embodiment. The example of FIG. 7 shows a case where the extraction unit 15b changes the person region 22 to an image painted in blue. As shown in the example of FIG. 7, the person region 22 is changed to an image painted in blue, and therefore it is difficult to specify a person in the image from the image shown in FIG. 7.

  In this way, the changing unit 15c generates an image with a reduced amount of information for identifying a person in the image for all frames input from the camera 20.

  The generation unit 15d generates an image by superimposing other images and feature points. For example, as shown in the example of FIG. 3, the generation unit 15d generates an image by superimposing the image (the other image) in which the person area 22 is changed and the feature point 23 for each frame. . Examples of how to draw a feature point include a method of drawing a mark such as a colored circle or square, or attaching a small image that does not overlap other feature points. Then, the generation unit 15d stores the generated one-frame image in the storage device 14 as image data 14a. The generation unit 15d performs such processing for all frames input from the camera 20. Therefore, when the image data 14a of the moving image stored in the storage device 14 is displayed on the display unit 12 in time series, for example, in the moving image as shown in FIG. 2A and FIG. A moving image in which a background image of a portion other than 22 is displayed. Therefore, according to the generation device 10, it is possible to generate image data that can easily grasp the behavior of a person. Further, as described above, according to the generation device 10, it is possible to generate image data in which a portrait right of a photographed person and invasion of privacy are suppressed. Therefore, according to the generation device 10, it is possible to suppress infringement of the portrait right and privacy of the photographed person and to easily grasp the action of the photographed person.

  Subsequently, the display control unit 15e controls the display of the display unit 12 so that the moving image indicated by the image data 14a is displayed. That is, the display control unit 15e controls the display of the display unit 12 so that each frame indicated by the image data 14a is displayed in time series. As a result, it is possible to display an image that can suppress infringement of portrait rights and privacy of the photographed person and can easily grasp the action of the photographed person.

  The control unit 15 is an integrated circuit such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA) or an electronic circuit such as a central processing unit (CPU) or a micro processing unit (MPU).

[Process flow]
Next, a processing flow of the generation apparatus 10 according to the present embodiment will be described. FIG. 8 is a flowchart illustrating the procedure of the generation process according to the first embodiment. The generation process is executed, for example, at a timing when the acquisition unit 15a receives a frame from the camera 20 via the communication unit 13.

  As illustrated in FIG. 8, the extraction unit 15b detects a person region from the frame image acquired by the acquisition unit 15a (S101). Subsequently, the extraction unit 15b extracts feature points in the person region (S102). Thereafter, the changing unit 15c changes the person area detected by the extracting unit 15b to an image filled with a predetermined color (S103).

  Then, the generation unit 15d generates one frame image by superimposing other images and feature points (S104). Subsequently, the generation unit 15d stores the generated one-frame image as image data 14a in the storage device 14 (S105). Thereafter, the display control unit 15e controls the display of the display unit 12 so that each frame indicated by the image data 14a is displayed in time series (S106). Then, the display control unit 15e determines whether there is a next frame (S107). The display control unit 15e can determine whether there is a next frame, for example, by determining whether the next frame is acquired by the acquisition unit 15a. When there is a next frame (Yes in S107), the process returns to S101, and the extraction unit 15b detects a person area from the image of the frame newly acquired by the acquisition unit 15a, and repeats the subsequent processing. On the other hand, if there is no next frame (No at S107), the process is terminated.

  As described above, according to the generation device 10 according to the present embodiment, it is possible to generate image data that can easily grasp the behavior of a person. Moreover, according to the production | generation apparatus 10, the image data by which the infringement of the portrait rights and privacy of the image | photographed person were suppressed can be produced | generated. Therefore, according to the generation device 10, it is possible to suppress infringement of the portrait right and privacy of the photographed person and to easily grasp the action of the photographed person.

  In the first embodiment, the case where the changing unit 15c changes the person area to an image painted with a predetermined color has been illustrated. However, the disclosed generation device 10 is not limited to this. For example, the changing unit 15c of the generation device 10 generates an image obtained by blurring a person image in a person region by applying a Gaussian filter or a smoothing filter. And the change part 15c of the production | generation apparatus 10 can change a person area | region into the produced | generated blurred image. Therefore, according to the generation device 10, it is possible to generate image data that can easily grasp the behavior of a person. Moreover, according to the production | generation apparatus 10, the image data by which the infringement of the portrait rights and privacy of the image | photographed person were suppressed can be produced | generated. Therefore, according to the generation device 10, it is possible to suppress infringement of the portrait right and privacy of the photographed person and to easily grasp the action of the photographed person.

[Process flow]
Next, another processing flow of the generation apparatus 10 according to the present embodiment will be described. FIG. 9 is a flowchart illustrating a procedure of another generation process according to the first embodiment. The generation process is executed, for example, at a timing when the acquisition unit 15a receives a frame from the camera 20 via the communication unit 13. Note that the processes of S101, S102, and S104 to S107 are the same as the processes of S101, S102, and S104 to S107 shown in FIG.

  As shown in FIG. 9, after the process in S102, the changing unit 15c generates an image in which the person image in the person area is blurred, and changes the person area to the generated blurred image (S201). The process proceeds to S104.

  As described above, the generation apparatus 10 generates an image obtained by blurring a person image in the person area, and changes the person area to the generated blurred image. Thereby, according to the production | generation apparatus 10, the image data which can grasp | ascertain a person's action easily can be produced | generated. Moreover, according to the production | generation apparatus 10, the image data by which the infringement of the portrait rights and privacy of the image | photographed person were suppressed can be produced | generated. Therefore, according to the generation device 10, it is possible to suppress infringement of the portrait right and privacy of the photographed person and to easily grasp the action of the photographed person.

  In the second embodiment, a case will be described in which the changing unit 35c changes a person area to a background image corresponding to the person area.

[System Configuration of System Having Generating Device 30]
FIG. 10 is a diagram illustrating an example of a system configuration of a system including the generation apparatus according to the second embodiment. As illustrated in FIG. 10, the generation device 30 includes a control unit 35. The control unit 35 is different from the control unit 15 according to the first embodiment illustrated in FIG. 1 in that it includes a changing unit 35c. In the following description, the same reference numerals as those in FIG. 1 are given to the respective units and devices that perform the same functions as those in the first embodiment, and the description thereof will be omitted.

  For each frame acquired by the acquisition unit 15a, the change unit 35c changes the person area in one frame to a background image corresponding to the person area. FIG. 11 and FIG. 12 are diagrams for explaining an example of processing executed by the generation apparatus according to the second embodiment. The example of FIG. 11 shows an example of an image (background image) in the absence of the person 21. For example, a background image is stored in the storage unit 14 in advance, and the changing unit 35c performs the following process on the frame in which the person area is detected by the extracting unit 15b. That is, the changing unit 35c first generates a background image. To explain with a specific example, the changing unit 35 c acquires the background image stored in the storage unit 14. Even if the background image is not stored in the storage unit 14 in advance, the changing unit 35c acquires the frame when the person area is not detected by the extracting unit 15b as the background image, thereby obtaining the background. An image can be generated. Then, the changing unit 35c changes the person area to a background image corresponding to the person area. Thereafter, as illustrated in the example of FIG. 12, the generation unit 15d generates an image in which the background image of the person area and the feature point 23 are superimposed. Therefore, according to the generating device 30, it is possible to generate image data that can easily grasp the behavior of a person. Moreover, according to the production | generation apparatus 30, the image data by which the infringement of the portrait rights and privacy of the image | photographed person were suppressed can be produced | generated. Therefore, according to the generation device 30, it is possible to suppress infringement of the portrait right and privacy of the photographed person and to easily grasp the action of the photographed person.

[Process flow]
Next, another processing flow of the generation apparatus 30 according to the present embodiment will be described. FIG. 13 is a flowchart illustrating the procedure of the generation process according to the second embodiment. The generation process is executed, for example, at a timing when the acquisition unit 15a receives a frame from the camera 20 via the communication unit 13. Note that the processes of S101, S102, and S104 to S107 are the same as the processes of S101, S102, and S104 to S107 shown in FIG.

  As shown in FIG. 13, after the processing in S102, the changing unit 35c generates a background image (S301). Then, the changing unit 35c changes the person area to a background image corresponding to the person area (S302), and proceeds to S104.

  As described above, the generation device 30 generates a background image corresponding to the person area, and changes the person area to the generated background image. Thereby, according to the production | generation apparatus 30, the image data which can grasp | ascertain a person's action easily can be produced | generated. Moreover, according to the production | generation apparatus 30, the image data by which the infringement of the portrait rights and privacy of the image | photographed person were suppressed can be produced | generated. Therefore, according to the generation device 30, it is possible to suppress infringement of the portrait right and privacy of the photographed person and to easily grasp the action of the photographed person.

  In the third embodiment, a case where the extraction unit 45b follows the extracted feature point and extracts the feature point from the person region in the frame next to the frame from which the feature point is extracted will be described.

[System Configuration of System Having Generating Device 40]
FIG. 14 is a diagram illustrating an example of a system configuration of a system including the generation apparatus according to the third embodiment. As illustrated in FIG. 14, the generation device 40 includes a control unit 45. The control unit 45 is different from the control unit 15 according to the first embodiment and the control unit 35 according to the second embodiment in that it includes an extraction unit 45b. In the following description, the same reference numerals as those in FIGS. 1 and 10 are assigned to the respective parts and devices that perform the same functions as those in the first and second embodiments, and the description thereof is omitted.

  When extracting a feature point from a person area for a certain frame, the extraction unit 45b searches for a point similar to the feature point extracted in the previous frame of the frame, and other than the feature point similar to the previous frame. Do not use feature points. Specifically, first, when extracting feature points, the feature amount size (edge strength, etc.) and detected coordinates of each feature point in each frame are stored. For each feature point detected in the current frame, the feature point is similar to the feature point extracted in the previous frame, and is used as a feature point only when the detected coordinates are close. Therefore, according to the generation device 40 of the present embodiment, the same feature point as that of the previous frame is displayed. Therefore, the amount of the feature point does not change depending on the frame, and the person's It is possible to generate image data 14a of a moving image that can easily recognize the operation. The extraction unit 45b according to the third embodiment can be applied to each of the first and second embodiments.

[Process flow]
Next, another processing flow of the generation apparatus 40 according to the present embodiment will be described. FIG. 15 is a flowchart illustrating the procedure of the generation process according to the third embodiment. The generation process is executed, for example, at a timing when the acquisition unit 15a receives a frame from the camera 20 via the communication unit 13. Note that the processes of S101 to S107 are the same as the processes of S101 to S107 shown in FIG.

  As shown in FIG. 15, after the process in S102, the extraction unit 45b determines whether or not the feature point of the person detected in the process of the current frame has been extracted in the previous frame (S401). When feature points are extracted for the first time (Yes at S401), the process proceeds to S103 as before. If the feature points have already been extracted before the previous frame (No in S401), the similarity is investigated between the feature points extracted in the current frame and the feature points extracted in the previous frame. The feature points considered to be associated with each other (S402). Then, the feature point of the current frame that does not correspond to the feature point of the previous frame is deleted (S403). Thereafter, the process proceeds to S103.

  Further, after the processing in S106, information on the feature points detected in the current frame (the size of the feature amount, the detected coordinate position, etc.) is stored. This is used for similarity comparison with the feature point of the next frame.

  As described above, the generation device 40 generates a background image corresponding to the person area, and changes the person area to the generated background image. Thereby, according to the production | generation apparatus 40, the image data which can grasp | ascertain a person's action easily can be produced | generated. Moreover, according to the production | generation apparatus 40, the image data by which the infringement of the portrait rights and privacy of the image | photographed person were suppressed can be produced | generated. Therefore, according to the generation device 40, it is possible to suppress infringement of the portrait right and privacy of the photographed person and to easily grasp the action of the photographed person.

  Further, according to the generation device 40 of the present embodiment, since the feature points are tracked, it is possible to generate the image data 14a of the moving image that makes the movement of the person smoother.

  In the fourth embodiment, a case where the extraction unit 45b changes the extraction method for extracting feature points from a person region when the number of extracted feature points is small will be described.

[System Configuration of System Having Generation Device 50]
FIG. 16 is a diagram illustrating an example of a system configuration of a system including the generation apparatus according to the fourth embodiment. As illustrated in FIG. 16, the generation device 50 includes a control unit 55. The control unit 55 is different from the control unit 15 according to the first embodiment, the control unit 35 according to the second embodiment, and the control unit 45 according to the third embodiment in that it includes an extraction unit 55b. In the following description, the same reference numerals as those in FIGS. 1, 10, and 14 are assigned to the respective parts and devices that perform the same functions as those in the first, second, and third embodiments, and the description thereof is omitted. I will do it.

  The extraction unit 55b extracts feature points using the first feature point extraction method, and if the number of extracted feature points is equal to or less than α, the feature points are extracted using the second feature point extraction method. To extract. For example, similarly to the method described in the first embodiment, the extraction unit 55b extracts feature points from a person region using a skin color template image. Then, the extraction unit 55b determines whether or not the number of extracted feature points is greater than α. When the number of feature points is less than or equal to α, the extraction unit 55b extracts feature points from the person region using the edge points. Therefore, according to the generating apparatus 50 of the present embodiment, it is possible to extract a larger number of feature points, and thus it is possible to generate image data that makes it easier to grasp a person's action. The extraction unit 55b according to the fourth embodiment can be applied to each of the first to third embodiments.

[Process flow]
Next, another processing flow of the generation device 50 according to the present embodiment will be described. FIG. 17 is a flowchart illustrating the procedure of the generation process according to the fourth embodiment. The generation process is executed, for example, at a timing when the acquisition unit 15a receives a frame from the camera 20 via the communication unit 13. In addition, since each process of S101 and S103-S107 is the same as each process of S101 shown in FIG. 8 mentioned above, and S103-S107, description is abbreviate | omitted.

  As shown in FIG. 17, following the processing in S101, the extraction unit 55b extracts feature points from the person region using the skin color template image, as in the method described in the first embodiment (S501). . Then, the extraction unit 55b determines whether or not the number of extracted feature points is larger than α (S502). If the number of feature points is less than or equal to α (No in S502), the extraction unit 55b uses the edge points to extract feature points from the person area (S503), and proceeds to S103. Also, when the number of feature points is larger than α (Yes in S502), the process proceeds to S103.

In the above description, only the case where the number of feature points is small has been described. However, the same applies to the case where the feature amount is too large, or the distribution of the feature points is biased to a part of the person area and the distribution situation is bad.
Also, regarding the change of the feature point extraction method, not only the feature points extracted by the first extraction method and the second extraction method are used, but also only the second extraction method is used, or the first extraction method is used. There is also a method of changing the threshold value of the feature amount of the feature point extracted by the method.

  As described above, the generation device 50 generates a background image corresponding to the person area, and changes the person area to the generated background image. Thereby, according to the production | generation apparatus 50, the image data which can grasp | ascertain a person's action easily can be produced | generated. Moreover, according to the production | generation apparatus 50, the image data by which the infringement of the portrait rights and privacy of the image | photographed person were suppressed can be produced | generated. Therefore, according to the generation device 50, it is possible to suppress infringement of the portrait right and privacy of the photographed person and easily grasp the action of the photographed person.

  Further, according to the generation device 50 of the present embodiment, since the feature points are tracked, it is possible to generate the image data 14a of the moving image that makes the movement of the person smoother.

  Further, according to the generation device 50, it is possible to generate image data that makes it easier to grasp the motion of a person.

  Although the embodiments related to the disclosed apparatus have been described above, the present invention may be implemented in various different forms other than the above-described embodiments. Therefore, another embodiment included in the present invention will be described below.

  For example, all or some of the processes described as being automatically performed among the processes described in the embodiments may be performed manually. In addition, among the processes described in the embodiments, all or a part of the processes described as being performed manually can be automatically performed by a known method.

  In addition, the processing at each step of each processing described in each embodiment can be arbitrarily finely divided or combined according to various loads and usage conditions. Also, the steps can be omitted.

  Further, the order of processing at each step of each processing described in each embodiment can be changed according to various loads and usage conditions.

  Further, each component of each illustrated apparatus is functionally conceptual, and does not necessarily need to be physically configured as illustrated. In other words, the specific state of distribution / integration of each device is not limited to the one shown in the figure, and all or a part thereof may be functionally or physically distributed or arbitrarily distributed in arbitrary units according to various loads or usage conditions. Can be integrated and configured.

[Generator]
The various types of processing of the generation apparatus described in the above embodiments can also be realized by executing a program prepared in advance on a computer system such as a personal computer or a workstation. Therefore, in the following, an example of a computer that executes a generation program having the same function as that of the generation apparatus described in each embodiment will be described with reference to FIG. FIG. 18 is a diagram illustrating a computer that executes a generation program.

  As illustrated in FIG. 18, the computer 300 includes a CPU, a ROM 320, an HDD 330, and a RAM 340.

  The ROM 320 stores basic programs such as an OS. In addition, the HDD 330 stores in advance a generation program 330a that exhibits the same functions as the acquisition unit, extraction unit, change unit, generation unit, and display control unit described in the above embodiment. Note that the generation program 330a may be separated as appropriate. The HDD 330 is provided with image data. The image data corresponds to the image data 14a described above.

  Then, the CPU 310 reads the generation program 330a from the HDD 330 and executes it.

  Then, the CPU 310 reads out the image data and stores it in the RAM 340. Further, the CPU 310 executes the generation program 330a using the image data stored in the RAM 340. Note that the data stored in the RAM 340 may not always be stored in the RAM 330. Data used for processing may be stored in the RAM 340.

  Note that the above-described generation program 330a is not necessarily stored in the HDD 330 from the beginning.

  For example, the program is stored in a “portable physical medium” such as a flexible disk (FD), a CD-ROM, a DVD disk, a magneto-optical disk, or an IC card inserted into the computer 300. Then, the computer 300 may read and execute the program from these.

  Furthermore, the program is stored in “another computer (or server)” connected to the computer 300 via a public line, the Internet, a LAN, a WAN, or the like. Then, the computer 300 may read and execute the program from these.

DESCRIPTION OF SYMBOLS 10 Generation apparatus 11 Input part 12 Display part 13 Communication part 14 Storage part 14a Image data 15 Control part 15a Acquisition part 15b Extraction part 15c Change part 15d Generation part 15e Display control part 20 Camera

Claims (9)

An extraction unit for extracting feature points from an image area of a person in the image;
A changing unit for changing the image area of the person to another image;
A generation unit that generates an image by superimposing the other image and the feature point;
A generation apparatus comprising: The extraction unit extracts a feature point from an image area of a person in one frame for each frame of a moving image obtained by photographing the person,
The change unit changes the image area of the person in one frame to another image for each frame of the moving image,
The generation unit generates a frame by superimposing the other image and the feature point for each frame of the moving image,
The generation apparatus according to claim 1, further comprising a display control unit that controls to display an image indicated by the frame generated by the generation unit. The extraction unit follows the extracted feature point and extracts the feature point from the image area of the person in the frame next to the frame from which the feature point was extracted. The generation according to claim 1, wherein the extraction unit changes an extraction method of extracting feature points from the image area of the person according to the number of extracted feature points. apparatus. The generating device according to claim 1, wherein the changing unit changes the image area of the person to an image filled with a predetermined color. The generation device according to claim 1, wherein the change unit changes the person's image area to a background image corresponding to the person's image area. The generating device according to claim 1, wherein the changing unit changes the image area of the person to an image obtained by blurring the image of the person. On the computer,
Extract feature points from the image area of the person in the image,
Change the image area of the person to another image,
A generation program for executing each process of generating an image by superimposing the other image and the feature point. A generation method executed by a computer,
Extract feature points from the image area of the person in the image,
Change the image area of the person to another image,
A generation method characterized by executing each process of generating an image by superimposing the other image and the feature point.

Images