JP2015023382A - Information processing device, method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To easily synthesize another image to a photographed image while displaying the photographed image in a region not desired of the image synthesis by a user.SOLUTION: An information processing device 100 includes a display 16 for displaying an image and a storage part for storing content data to be synthesized with the image. The information processing device 100 calculates a face recognition region 71, in which content data is not synthesized within a subject person, in a photographed image data by performing face detection processing for photographed image data outputted by a camera. The information processing device 100 determines a display position of the content data and image size according to a face recognition result of a subject in the photographed image data, and displays the content data in a synthesis region 81 with not synthesizing the content data in the face recognition region 71.

Description

  The present disclosure relates to a technique for combining and displaying another image on captured image data obtained by capturing with a camera.

  Information processing apparatuses equipped with cameras such as smartphones and tablet terminals are widely used. Users use these camera functions to take images of users, for example, and post them to an SNS (social networking service) site, a group chat application, etc., and share the taken image between users.

  Numerous applications are provided for information processing apparatuses such as smartphones. The information processing apparatus downloads the application from the distribution server through, for example, an application store. These applications include, for example, an application that makes it easy to use the camera function of the information processing apparatus, an application that processes captured images, and an application that has a function of detecting the face of a subject by image recognition processing. A technique for detecting a face image from a captured image is disclosed in, for example, Japanese Patent Application Laid-Open No. 2009-223581 (Patent Document 1).

  The information processing apparatus activates a camera application having a face detection function, detects the eye position of the subject included in the photographed image, and represents an object such as glasses or a mask based on the detected eye position Composite the images. As a result, the user can obtain an image in which the subject wears an object (such as glasses), and can easily confirm an impression after the user wears the glasses without actually wearing the glasses, for example. . In this way, by detecting the face of the subject included in the captured image data and combining another image such as glasses or beard according to the detection result, the user can easily process the captured image, You can enjoy sharing with other users.

JP 2009-223581 A

  However, some users may wish to display the original captured image data without combining the images of some of the captured images while combining the animated images with the captured image data. is there. For example, there is a case where a person who is a subject wants to synthesize another image around the subject while showing a face, a costume, or the like. For example, the subject may be a character of a work such as a manga, a game, an animation, or a movie, and the hairstyle is adjusted, makeup is applied, and a costume is worn. Such makeup and costumes are considered to have a high specific gravity as an object of appreciation. On the other hand, there is also a demand for synthesizing an animation image for reproducing the effect on the work, such as light rays around the character, with the photographed image.

  Therefore, there is a need for a technique for easily synthesizing an animated image or the like with a captured image while displaying a captured image for an area where the user does not want to combine the images.

  According to one embodiment, a program for controlling the operation of the information processing apparatus is provided. The information processing apparatus includes a display for displaying an image, a memory for storing content data combined with the image, and a processor for controlling the operation of the information processing apparatus. The program performs an image recognition process on the input image data to the processor, thereby detecting a region indicating a person included in the input image data, and indicating the person detected in the detecting step in the input image data. Determining an area for suppressing display of the content data based on the area, and generating output image data by combining the content data with the input image data so that the content data is not displayed in the determined area; Displaying the output image data generated in the generating step on a display.

  According to another embodiment, a method for controlling the operation of an information processing device is provided. The information processing apparatus includes a display for displaying an image, a memory for storing content data combined with the image, and a processor for controlling the operation of the information processing apparatus. In the method, the information processing device performs image recognition processing on the input image data to detect a region indicating a person included in the input image data, and the person detected in the detecting step in the input image data Determining an area for suppressing display of the content data based on the area indicating the content, and generating the output image data by combining the content data with the input image data so that the content data is not displayed in the determined area And displaying the output image data generated in the generating step on a display.

  An information processing apparatus according to another embodiment includes a display for displaying an image, a memory for storing content data combined with the image, and a processor for controlling the operation of the information processing apparatus. Is a detection unit that detects a region indicating a person included in the input image data by performing image recognition processing on the input image data, and a region indicating the person detected by the detection unit in the input image data. An output image generation unit that determines an area for suppressing display of the content data and combines the content data with the input image data so that the content data is not displayed in the determined area; A display control unit for displaying the output image data generated by the generation unit on a display.

  According to the above-described embodiment, it is possible to easily combine another image with a captured image while displaying a captured image for a region where the user does not want to combine the images.

  The above and other objects, features, aspects and advantages of the present invention will become apparent from the following detailed description of the present invention taken in conjunction with the accompanying drawings.

It is a figure which shows an example of the condition where the information processing apparatus 100 concerning embodiment is used. 1 is a diagram illustrating a configuration of a network including an information processing apparatus 100. FIG. 2 is a block diagram illustrating a configuration of an information processing apparatus 100. FIG. It is a figure which shows the content management table 51. FIG. It is a figure which shows the example of the content data synthesize | combined with a picked-up image. 3 is a flowchart illustrating processing for combining content data and captured image data and displaying them by the information processing apparatus 100 according to the first embodiment. It is a figure which shows the process which synthesize | combines picked-up image data and content data. It is a figure which shows an example of the user interface for synthesize | combining picked-up image data and content data. It is a figure which shows the example of the interface for receiving input operation which purchases the content data used as the object combined with picked-up image data. It is a figure which shows the example which the information processing apparatus 100 receives ON or OFF of the synthesis | combination of content data by voice input. 6 is a diagram illustrating a display example of the information processing apparatus 100 according to Embodiment 2. FIG. It is a figure which shows the user allocation information 53 used in the process of the information processing apparatus 100 of Embodiment 2. FIG. 10 is a flowchart illustrating processing for combining content data with captured image data and displaying the information by the information processing apparatus 100 according to the second embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description, the same parts are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed description thereof will not be repeated.

<Embodiment 1>
FIG. 1 is a diagram illustrating an example of a situation in which the information processing apparatus 100 according to the embodiment is used. The information processing apparatus 100 is, for example, a smartphone including a touch panel, but is not limited to a smartphone. The information processing apparatus 100 includes a camera and displays a captured image captured by the camera on the display 16. The information processing apparatus 100 combines the captured image with another image (content data) and displays it on the display 16. For example, as illustrated in FIG. 1, the information processing apparatus 100 synthesizes content data for displaying a flickering flame-shaped image around the face of the subject and displays the synthesized content data on the display 16. At this time, the information processing apparatus 100 extracts a part of the captured image based on the image recognition process, and controls the display content of the display 16 so as not to display the content data in this area. For example, when a user wants to display a person's face in a captured image and does not want to display content data on the person's face, the person's face is detected and detected by image recognition processing. It is possible to synthesize content data and photographed image data while avoiding the face portion.

  In the example of FIG. 1, it is assumed that the subject is a character of a certain work such as a manga, a game, an animation, or a movie. The subject is wearing a character's costume, holding an accessory, and maintaining a certain posture. The information processing apparatus 100 detects the face of the subject by performing image recognition processing on the captured image, and sets a face recognition area 71 that is an area where content data is not combined based on the detection result. Further, the information processing apparatus 100 determines a coordinate value serving as a reference for combining the content data based on the result of the process of detecting the face of the subject in the captured image. The information processing apparatus 100 displays a part of the content data in the synthesis area 81 so that the content data to be synthesized with the captured image data is not displayed in the face recognition area 71. Thereby, for example, when the subject is a character of a certain work, the information processing apparatus 100 can synthesize an effect in the work with the photographed image. Further, for example, when the user does not want to synthesize an image on the face portion of the subject, the information processing apparatus 100 sets an area in which the image is not synthesized based on the result of the image recognition process, The images can be combined and displayed on the display 16 so as not to combine the content data. In this way, the information processing apparatus 100 can provide a sense of being a character to the subject.

<Configuration>
FIG. 2 is a diagram illustrating a network configuration including the information processing apparatus 100. In FIG. 2, information processing apparatuses 100 </ b> A, 100 </ b> B, and 100 </ b> C are illustrated as the information processing apparatus 100. The information processing apparatuses 100A and 100B are smartphones, and the information processing apparatus 100C is a tablet terminal. Referring to FIG. 2, information processing apparatus 100 </ b> A is connected to radio base station 700 by radio communication, and communicates with distribution server 900 via network 800. The distribution server 900 stores a plurality of content data for the information processing apparatus 100 to synthesize with the captured image data and display on the display 16. The information processing apparatus 100 </ b> A acquires content data from the distribution server 900 by communicating with the distribution server 900. For example, the information processing apparatus 100 acquires content data for which a usable period is determined, content data that can be purchased through billing processing, and the like from the distribution server 900. The information processing apparatus 100B is connected to the router 600 by wireless communication, and communicates with the distribution server 900 via the network 800. The information processing apparatus 100C is connected to the router 600 by wireless communication, and communicates with the distribution server 900 via the network 800.

  FIG. 3 is a block diagram illustrating a configuration of the information processing apparatus 100. As shown in FIG. 3, the information processing apparatus 100 includes antennas 11 and 13, a long-distance communication module 12, a short-distance communication module 14, a touch panel 15, a display 16, a positioning sensor 17, and a front camera. 18, rear camera 19, power supply control unit 20, battery 21, microphone 22, speaker 23, sound processing unit 24, storage unit 25, and control unit 26.

  The antenna 11 and the antenna 13 radiate a signal emitted from the information processing apparatus 100 as a radio wave. The antenna 11 receives radio waves from the space and gives a received signal to the long-distance communication module 12. The antenna 13 receives radio waves from the space and gives a received signal to the short-range communication module 14.

  The long-distance communication module 12 and the short-distance communication module 14 perform modulation / demodulation processing for transmitting and receiving signals via the antenna 11 and the like so that the information processing apparatus 100 communicates with other wireless devices. The long-distance communication module 12 and the short-distance communication module 14 are communication modules including a tuner, a high-frequency circuit, and the like, perform modulation / demodulation and frequency conversion of radio signals transmitted and received by the information processing apparatus 100, and control received signals. 26.

  The long-distance communication module 12 is a communication module compatible with mobile communication systems such as LTE (Long Term Evolution) and UMTS (Universal Mobile Telecommunications System), and is compatible with relatively long-distance communication.

  The short-range communication module 14 corresponds to, for example, a short-range wireless communication standard such as Bluetooth (registered trademark) or a wireless LAN (Local Area Network) standard such as IEEE (Institute of Electrical and Electronic Engineers) 802.11. It is a communication module and supports relatively short distance communication. When the information processing apparatus 100 supports a plurality of communication methods, an antenna and a communication module may be provided corresponding to these communication methods.

  The touch panel 15 receives a user input operation. The touch panel 15 accepts a user's contact operation by using, for example, a capacitive type. The touch panel 15 outputs an input operation by the user to the control unit 26.

  The display 16 displays icons and background images on the menu screen, and displays still images, moving images, and the like according to application operations. The display 16 is realized by, for example, an LCD (Liquid Crystal Display) or an organic EL (electroluminescence) display.

  The positioning sensor 17 is a sensor for measuring the position of the information processing apparatus 100, and is a GPS (Global Positioning System) module, for example.

  The front camera 18 and the rear camera 19 are camera modules that receive incident light and convert them into electrical signals, and include a CMOS (Complementary Metal Oxide Semiconductor) sensor, an ADC (Analog to Digital Converter), and the like, and control image signals. 26. In the case of the information processing apparatus 100, when the surface on which the display 16 is disposed is the front surface, the front camera 18 is disposed on the front surface of the information processing apparatus 100. The rear camera 19 is disposed on the back surface of the information processing apparatus 100.

  The power supply control unit 20 controls power supply to each circuit of the information processing apparatus 100. The power control unit 20 is, for example, a power control IC (Integrated Circuit).

  The battery 21 is a supply source that supplies power for operating each circuit of the information processing apparatus 100. The electric power from the battery 21 is supplied to each circuit according to the control of the power supply control unit 20.

  The microphone 22 receives a voice input by the user and gives a voice signal corresponding to the voice input to the voice processing unit 24.

  The speaker 23 converts the sound signal given from the sound processing unit 24 into sound and outputs the sound to the outside of the information processing apparatus 100.

  The audio processing unit 24 performs modulation / demodulation of the audio signal. The audio processing unit 24 modulates the signal given from the microphone 22 and gives the modulated signal to the control unit 26. In addition, the audio processing unit 24 gives an audio signal to the speaker 23. The sound processing unit 24 is realized by, for example, a sound processing processor.

  The storage unit 25 is configured by a flash memory, for example, and stores data and programs used by the information processing apparatus 100. In one aspect, the storage unit 25 stores a content management table 51 and content data 52.

  As will be described in detail later, the content management table 51 includes identification information for identifying each content data to be combined with a captured image, a file name of the content data, a name of the content data, and a period during which the content data can be used. Is a table in which

  The content data 52 is still image or moving image data corresponding to identification information of each content data.

  The control unit 26 controls the overall operation of the information processing apparatus 100 by reading and executing a control program stored in the storage unit 25. The control unit 26 is, for example, an application processor. The control unit 26 performs the functions of the detection processing unit 61, the image generation unit 62, and the display control unit 63 by operating according to a program. The detection processing unit 61 causes the information processing apparatus 100 to perform the function of image recognition processing, and performs image recognition processing on the captured image data output from the camera 7, thereby detecting a human face included in the captured image data. To detect. For example, the detection processing unit 61 performs image recognition processing on the captured image data, and outputs the positions of the eyes of the person, the face size, and the like included in the captured image data. The image generation unit 62 performs processing such as determining the position where the content data is combined based on the detection result of the detection processing unit 61 with respect to the captured image data, and generates combined image data. The display control unit 63 causes the information processing apparatus 100 to function to generate a video signal and control the display content of the display 16.

<Data structure>
With reference to FIG. 4, data used in the processing of the information processing apparatus 100 will be described. FIG. 4 is a diagram showing the content management table 51.

  As shown in FIG. 4, each record of the content management table 51 includes an identification number 511, a file name 512, an animation name 513, and an available period 514.

  The identification number 511 is identification information for identifying each content data. The file name 512 is information indicating a file corresponding to the identification information of the content data. The animation name 513 indicates the title of the content data indicated by the identification information. The available period 514 indicates a period during which the content data can be used. As shown in FIG. 4, for example, the data “2013/10/31” in the available period 514 indicates that the period during which the user can use the content data is October 31, 2013. “None” indicates that an available period of content data is not set.

<Content data to be combined with the shot image>
The content data in the first embodiment will be described with reference to FIG. FIG. 5 is a diagram illustrating an example of content data to be combined with a captured image.

  In FIG. 5, content data 52A, 52B, and 52C are shown as examples of content data stored in the storage unit 25. The content data 52A is an animation image in which a wavy line is emitted from the center. The content data 52B is an animation image in which a line is emitted from a ring-shaped object. The content data 52C is an animation image in the shape of a character. These are examples of content data, and the content data is not limited to these examples.

<Control structure>
The control structure of the information processing apparatus 100 will be described with reference to FIG. FIG. 6 is a flowchart illustrating processing for combining content data and captured image data and displaying them by the information processing apparatus 100 according to the first embodiment. In the first embodiment, the content data is animation data having a two-dimensional shape. Hereinafter, a process in which the information processing apparatus 100 receives an input operation from the user via the touch panel 15 and combines the content data with the captured image data will be described.

  In step S <b> 601, the control unit 26 activates the rear camera 19 and performs initialization processing related to the camera, such as setting the image size of an image captured by the rear camera 19.

  In step S <b> 603, the control unit 26 receives input of image signals generated by the rear camera 19, sequentially acquires captured image data, and displays the acquired captured image data on the display 16.

  In step S605, the control unit 26 performs image recognition processing on the captured image data to detect a human face included in the captured image.

  In step S607, the control unit 26 acquires the result of the face detection process for the captured image data. As a result of the face detection process, the control unit 26 acquires the face size of the person, the coordinates of the eyes of the person, the coordinates of the mouth of the person, and the like in the captured image data.

  In step S609, the control unit 26 calculates a coordinate value indicating the face range of the person of the subject included in the captured image data based on the result of the face detection process acquired in step S607. For example, the control unit 26 estimates a line segment passing through the center of the face based on the average value of the coordinates of the left eye and right eye of the person and a line segment connecting the left eye and the right eye, and based on the estimated line segment To estimate the coordinates of the center of the face. The control unit 26 calculates a rectangular, circular, or other shape area within a certain range from the coordinates of the center of the face as the face range of the subject included in the captured image data. For example, when the control unit 26 calculates the face range of the subject as a rectangular region, the control unit 26 calculates the coordinate values of four points that form the rectangle. When the control unit 26 calculates the face range of the subject as a circular region, the control unit 26 calculates the coordinates of the center of the face and a value indicating the size of the radius.

  In step S611, the control unit 26 determines the image size of the content data to be combined with the photographed image data and the display position of the content data based on the result of the face detection process. For example, the control unit 26 determines the display position of the content data on the basis of the coordinates of the left eye and the right eye of the subject person included in the captured image data. For example, the control unit 26 estimates the size of the subject's face according to the distance between the left eye and the right eye of the subject, and determines the image size of the content data according to the size of the face.

  In step S613, the control unit 26 prevents the content data from being displayed in the face range of the subject based on the face range calculated in step S609 and the image size and display position of the content data determined in step S611. Processing such as making part of the content data transparent.

  In step S615, the control unit 26 combines the processed content data and the captured image data, and displays an image generated by the combination on the display 16. Thereafter, the control unit 26 performs the process of step S603.

<Example of composition of content data>
With reference to FIG. 7, a process in which the control unit 26 combines the captured image data and the content data and displays the combined image on the display 16 will be described. FIG. 7 is a diagram illustrating a process of combining the captured image data and the content data.

  As shown on the display 16A, the control unit 26 performs an image recognition process on the captured image data to detect a human face included in the captured image (step S605), and determines the range of the human face of the subject. Calculate (step S607, step S609). In the display 16 </ b> A, the face range of the subject person calculated by the control unit 26 is shown as a face recognition area 71.

  As shown on the display 16B, the control unit 26 determines the image size and display position of the content data (step S611). On the display 16B, the image size and display position of the content data are shown as content data 52B.

  As shown on the display 16C, the control unit 26 processes the content data so that the content data is not displayed in the range of the face of the subject person (step S613, step S615).

<Example of user interface>
An example of a screen displayed on the display 16 by the information processing apparatus 100 will be described with reference to FIG. FIG. 8 is a diagram illustrating an example of a user interface for combining photographed image data and content data.

  As shown in FIG. 8, on the display 16, the control unit 26 displays an image display area 91, a display switching unit 92, a file name display unit 93, an animation data purchase unit 94, a shooting switching unit 95, and a shooting start. The unit 96 and the voice input receiving unit 97 are displayed. Further, the control unit 26 displays on the display 16 icon images indicating candidate content data to be combined with photographed image data, such as the background animation candidate 98, the selected animation 99, and the character animation candidate 90. In the example of FIG. 8, the content data of the selected animation 99 is selected by the user, and the control unit 26 displays the icon image of the selected animation 99 separately from other icon images.

  The image display area 91 is an area for displaying an image obtained by combining captured image data output from the camera of the information processing apparatus 100 and content data designated by the user. The control unit 26 detects the face of the subject through image recognition processing on the captured image data. In the image display area 91, the face recognition area 71 indicates the face range of the subject. The control unit 26 synthesizes the content data with the captured image data and displays it on the display 16 so that the content data is not displayed in the range indicated by the face recognition area 71. In the example of FIG. 8, the synthesis area 81 indicates an area where content data is displayed.

  The display switching unit 92 is an area indicating whether or not to display on the display 16 an image obtained by combining the content data designated by the user and the captured image data. In the example of FIG. 8, the display switching unit 92 indicates the setting “ON”. In this case, the control unit 26 displays an image generated by combining the content data and the captured image data on the display 16. When the display content of the display switching unit 92 is set to “OFF”, the control unit 26 displays the captured image data acquired by the camera on the display 16. The control unit 26 receives a user input operation through the touch panel 15, receives an input operation on the display switching unit 92, and turns on the setting as to whether or not to display an image obtained by combining the captured image data and the content data. Switch between and off.

  The file name display section 93 indicates the title of the content data to be combined with the captured image data. The control unit 26 reads the animation name 513 corresponding to the content data selected by the user with reference to the content management table 51 and displays it on the file name display unit 93.

  The animation data purchasing unit 94 is a button for accepting an input operation for purchasing content data from the user. The information processing apparatus 100 can download content data from an external apparatus such as the distribution server 900, for example. The information processing apparatus 100 is connected to an external apparatus such as the distribution server 900 and receives an input operation for purchasing content data from the user.

  The shooting switching unit 95 is an area for receiving a setting for saving an image displayed in the image display area 91 as a still image or a moving image.

  The imaging start unit 96 is an area for accepting an input operation for saving the image displayed in the image display area 91 from the user. In the display switching unit 92, when the setting is “ON”, that is, the setting is to display an image in which the content data and the captured image data are combined, the information processing apparatus 100 responds to an input operation to the shooting start unit 96. Save the combined image of the content data as a still image or a moving image.

  The voice input receiving unit 97 is an area for receiving a user input operation by voice. The information processing apparatus 100 starts receiving voice input by receiving an input operation to the voice input receiving unit 97, and performs voice recognition processing based on the input voice. For example, the information processing apparatus 100 causes the input voice to be subjected to voice recognition processing by an application for voice recognition processing or is transmitted to an external device such as the distribution server 900 and performs the voice recognition processing on the external device. Then, the result of the voice recognition process is received. The information processing apparatus 100 operates according to the result of the speech recognition process for the input speech. For example, the information processing apparatus 100 receives the result of the voice recognition process, operates as if the input operation to the shooting start unit 96 has been performed, and stores the combined image displayed in the image display area 91. Perform the process. Further, for example, the information processing apparatus 100 receives the result of the voice recognition processing, operates as if the input operation to the display switching unit 92 has been performed, and displays an image in which the captured image data and the content data are combined. To switch the setting of whether or not.

<Example of user interface for accepting purchase of content data>
With reference to FIG. 9, an example of a user interface for the information processing apparatus 100 to accept purchase of content data from a user will be described. FIG. 9 is a diagram illustrating an example of an interface for accepting an input operation for purchasing content data to be combined with captured image data.

  As shown in FIG. 9, the control unit 26 displays a purchase button 941 and a content data list 942 on the display 16. The control unit 26 receives an input operation for the user to select content data via the touch panel 15. The purchase button 941 is a button for accepting an input operation for purchasing content data selected by the user.

  The content data list 942 displays a list of content data that can be purchased by the user. In the example of FIG. 9, for the content for which the content data availability period is set in the availability period 514 of the content management table 51, the information processing apparatus 100 sets the content data availability period as the content data. Displayed in a list 942.

<Switching of whether to synthesize content data with captured image data>
With reference to FIG. 10, an example will be described in which setting whether or not to synthesize content data with captured image data is received by voice input. FIG. 10 is a diagram illustrating an example in which the information processing apparatus 100 accepts content data synthesis on / off by voice input.

  The information processing apparatus 100 receives voice input by receiving an input operation on the voice input receiving unit 97. In FIG. 10, when the display switching unit 92 indicates the setting “OFF”, the control unit 26 displays the captured image on the display 16 without combining the content data designated by the user and the captured image data. The information processing apparatus 100 switches, for example, whether to display an image in which content data and captured image data are combined by receiving voice input of a preset keyword. For example, as illustrated in FIG. 10, the information processing apparatus 100 receives a voice input of a keyword such as a voice “transformation!” Through the microphone 22. As a result, the information processing apparatus 100 synthesizes the captured image data and the content data, starts a process of displaying an image generated by the synthesis, and displays the setting “ON” on the display switching unit 92.

  In this way, the information processing apparatus 100 can switch whether to display an image obtained by combining content data with captured image data by voice input. Thereby, for example, when the subject is a character of a certain work, it is possible to switch whether or not to display an image in which content data and captured image data are combined by inputting a specific sound, A sense of being a character can be provided to the subject.

  In addition, the information processing apparatus 100 accepts an input operation for saving, for example, a moving image, a process of switching whether to display an image in which content data and captured image data are combined by inputting specific sound, May be able to share videos and the like with other users. For example, the information processing apparatus 100 may be able to post a still image or a moving image generated by synthesizing content data to captured image data, for example, on an image posting website or group chat. Thereby, the variation of how to enjoy a work between users can be expanded.

<Embodiment 2>
An information processing apparatus 100 according to another embodiment will be described with reference to FIGS. The information processing apparatus 100 according to the second embodiment recognizes the faces of a plurality of persons included in the captured image data, and displays each person's face and content data in association with each other on the display 16. The information processing apparatus 100 detects a subject person included in the captured image data by face recognition processing, and has an image database of subject subjects in advance, and who is detected by the image recognition processing Is identified.

  FIG. 11 is a diagram illustrating a display example of the information processing apparatus 100 according to the second embodiment. As illustrated in FIG. 11, the information processing apparatus 100 identifies the first user and the second user by performing image recognition processing on the captured image data. The information processing apparatus 100 estimates the range of the first user's face included in the captured image data as the face recognition area 71A. Further, the information processing apparatus 100 stores the first user and the content data 52A in association with each other. Similarly, the information processing apparatus 100 estimates the range of the face of the second user included in the captured image data as the face recognition area 71B. Further, the information processing apparatus 100 stores the second user and the content data 52B in association with each other.

  The information processing apparatus 100 combines the captured image data and the content data so that the content data is not displayed in the range of the first user's face on the display 16, and displays the content data in the combining area 81A. Further, the information processing apparatus 100 synthesizes the content data with the captured image data so that the content data is not displayed in the range of the face of the second user on the display 16 and displays the content data in the synthesis area 81B.

<Data structure>
FIG. 12 is a diagram illustrating user assignment information 53 used in the processing of the information processing apparatus 100 according to the second embodiment. The information processing apparatus 100 stores user allocation information 53 in the storage unit 25, for example. The user assignment information 53 is information indicating which content data is assigned to each person.

  Each record of the user allocation information 53 includes user identification information 531, a file name 532, and an animation name 533.

  The user identification information 531 is identification information for identifying each user. The file name 532 is information for specifying the content data file associated with the user indicated in the identification information of the user identification information 531. The animation name 533 indicates the name of the content data indicated in the file name 532.

<Control structure>
With reference to FIG. 13, the operation of the information processing apparatus 100 in the second embodiment will be described. The same parts as those in the first embodiment are denoted by the same reference numerals, and description thereof will not be repeated. FIG. 13 is a flowchart illustrating processing for combining content data with captured image data and displaying the information by the information processing apparatus 100 according to the second embodiment. Note that the information processing apparatus 100 receives user registration, and stores in the storage unit 25 a reference image for identifying the user who has received registration through face recognition processing.

  In step S <b> 1301, the control unit 26 activates the rear camera 19 and performs initialization processing related to the camera, such as setting the image size of an image captured by the rear camera 19.

  In step S <b> 1303, the control unit 26 receives input of an image signal generated by the rear camera 19 to acquire captured image data, and displays a preview of the acquired captured image data on the display 16.

  In step S1305, the control unit 26 sets an image for reference in order to identify a person from the captured image data through face recognition processing through image recognition processing. The control unit 26 performs image recognition processing on the captured image data to detect a person's face included in the captured image, and compares the detected person with a reference image to detect the person's face. An identifier for specifying who the person is is output to a memory or the like.

  In step S1307, the control unit 26 acquires various data output by the face recognition process. For example, the control unit 26 reads out and acquires data such as a person identifier, person face size, person eye coordinates, person mouth coordinates, and the like in the captured image data.

  In step S1309, the control unit 26 calculates a coordinate value indicating the range of the face of the subject person included in the captured image data based on the result of the face recognition processing acquired in step S1307.

  In step S1311, the control unit 26 determines the image size of the content data to be combined with the captured image data and the display position of the content data for each content data corresponding to the user identifier based on the result of the face recognition process. To do.

  In step S1313, based on the face range calculated in step S1309 and the image size and display position of the content data determined in step S1311, the control unit 26 prevents content data from being displayed in the face range of the subject. Processing such as making part of the content data transparent.

  In step S1315, the control unit 26 combines the processed content data and the captured image data, and displays an image generated by the combining on the display 16. Thereafter, the control unit 26 performs the process of step S1303.

  As described above, the information processing apparatus 100 according to the second embodiment performs image recognition processing on the captured image data, performs face recognition of the subject person, associates the subject person with the content data, and creates a content. Data and captured image data can be combined. For this reason, for example, when a plurality of persons who are subjects are each a specific character, the region where the image is not desired to be synthesized is displayed as it is, while the captured image data is displayed as it is, and content data suitable for each character is displayed on the subject. Can be combined with other people. For example, it becomes easy to synthesize content data to reproduce a composition in which a plurality of characters appear in a work.

<Modification>
As described above, the information processing apparatus 100 according to each embodiment has been described, but can be modified as follows.

<1. Human detection by image recognition processing>
In the description of each of the embodiments described above, the image recognition process is performed on the captured image data to detect the face of the subject person in the captured image data, and the face of the subject person is detected for reference. A configuration for collating images and identifying who the subject is was explained. The method for detecting a person as a subject is not limited to this, and for example, if the information processing apparatus 100 can use algorithms for pedestrian detection processing, these algorithms may be applied to captured image data. The information processing apparatus 100 may detect the subject as a pedestrian to detect the whole body of the subject in the captured image data, and may determine a region where the content data is combined based on the detection result.

<2. Display position of content data>
In the description of the above embodiment, the information processing apparatus 100 makes part of the content data 52 transparent so that the content data 52 is not displayed in the face recognition area 71 as described with reference to FIGS. 6 and 7. In the above description, it is described that the processed content data and the captured image data are combined to obtain combined image data. In addition to this, the information processing apparatus 100 calculates the face range (face recognition area 71) of the subject included in the captured image data so that the content data 52 is not displayed in the face recognition area 71 (step S609). The display position and the image size at which the content data 52 is not displayed in the face recognition area 71 may be determined so that no overlap occurs between the recognition area 71 and the display area of the content data 52 (step S611).

<3. Content data distribution method>
In the description of the above embodiment, the information processing apparatus 100 has been described as being able to acquire the content data 52 from, for example, the distribution server 900. However, in addition to this, for example, the content is limited to a specific area. The data 52 may be acquired. For example, the distribution server 900 that distributes content data uses the position of the information processing apparatus 100 as a result of (a) GPS measurement by the positioning sensor 17 and (b) a wireless base station 700 in which the information processing apparatus 100 is performing wireless communication. And (c) information such as the identification information of the wireless LAN router with which the information processing apparatus 100 is performing wireless communication. When the distribution server 900 receives a request from the information processing apparatus 100 to acquire the content data 52 whose distribution area is limited, the distribution server 900 distributes the content data 52 based on the position information of the information processing apparatus 100. Whether or not it is an area where the distribution of the content data 52 can be received may be controlled in the information processing apparatus 100.

  In this way, by limiting the areas where the information processing apparatus 100 can acquire the content data 52, for example, the user is encouraged to visit an event or a sightseeing spot held in an area related to the content data 52. be able to.

  In addition, the information processing apparatus 100 has been described as performing the image recognition processing on the captured image data to detect the face area of the subject, thereby determining the face recognition area 71 so that the content data 52 is not displayed. However, the range of the face recognition area 71 may be editable by the user. Further, the information processing apparatus 100 may accept input of a handwritten image by a user and store the received image as content data 52 to be combined with captured image data.

  The embodiment disclosed this time must be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

  11, 13 Antenna, 12 Long-distance communication module, 14 Short-distance communication module, 15 Touch panel, 16 Display, 17 Position sensor, 18 Front camera, 19 Rear camera, 20 Power supply control unit, 21 Battery, 22 Microphone, 23 Speaker 24 audio processing unit 25 storage unit 26 control unit 51 content management table 52 content data 53 user allocation information 61 detection processing unit 62 image generation unit 63 display control unit 71 face recognition area 81 synthesis Area, 91 image display area, 92 display switching section, 93 file name display section, 94 animation data purchasing section, 95 shooting switching section, 96 shooting start section, 97 voice input reception section, 98 background animation candidate, 99 selected animation, 90 character animation 100 information processing device, 511 identification number, 512 file name, 513 animation name, 514 usable period, 531 user identification information, 532 file name, 533 animation name, 600 router, 700 wireless base station, 800 network, 900 Distribution server, 941 Purchase button, 942 Animation list.

Claims (8)

A program for controlling the operation of the information processing apparatus,
The information processing apparatus includes a display for displaying an image, a memory for storing content data combined with the image, and a processor for controlling the operation of the information processing apparatus,
The program is stored in the processor.
Detecting an area indicating a person included in the input image data by performing image recognition processing on the input image data;
In the input image data, an area for suppressing display of the content data is determined based on the area indicating the person detected in the detecting step, and the content data is not displayed in the determined area. Generating output image data by combining the content data with the input image data; and
And displaying the output image data generated in the generating step on the display. The program is stored in the processor.
In the detecting step, as a region indicating the person, a region of a human face included in the input image data is detected,
The program according to claim 1, wherein in the generating step, the detected face area is determined as an area for suppressing display of the content data. The program is stored in the processor.
In the generating step, a position for displaying the content data is determined based on an area indicating the detected person, and an area for suppressing the display of the content data and a position for displaying the content data The program according to claim 1, wherein the output image data is generated by combining the content data with the input image data so that the content data is not displayed in an overlapping portion. The program is stored in the processor.
In the generating step, the region of the content data is determined by determining a region for displaying the content data so that an overlapping portion does not occur between the region for suppressing the display of the content data and the region for displaying the content data. The program according to claim 1, wherein composition is performed on input image data. The information processing apparatus further includes:
A camera that generates an image signal by shooting and outputs the generated image signal to the processor;
The program is stored in the processor.
In the detecting step, an image signal output from the camera to the processor is received as the input image data, and an area indicating the person is detected based on the input image data,
The program according to claim 1, wherein in the generating step, output image data is generated based on the input image data received by the processor from the camera. The information processing apparatus further includes:
An operation accepting unit for accepting an input operation from a user;
A communication module for causing the information processing apparatus to exert a communication function;
The information processing apparatus is configured to be connected to an external communication apparatus through the communication module and receive the content data from the external communication apparatus.
The program is stored in the processor.
The program according to claim 1, wherein the operation receiving unit performs a process of receiving an input operation for purchasing the content data from the external communication device. A method for controlling the operation of an information processing apparatus,
The information processing apparatus includes a display for displaying an image, a memory for storing content data combined with the image, and a processor for controlling the operation of the information processing apparatus,
In the method, the information processing apparatus
Detecting an area indicating a person included in the input image data by performing image recognition processing on the input image data;
In the input image data, an area for suppressing display of the content data is determined based on the area indicating the person detected in the detecting step, and the content data is not displayed in the determined area. Generating output image data by combining the content data with the input image data; and
Displaying the output image data generated in the generating step on the display. An information processing apparatus,
A display for displaying images,
A memory for storing content data to be combined with the image;
A processor for controlling the operation of the information processing apparatus,
The processor is
A detection unit that detects an area indicating a person included in the input image data by performing image recognition processing on the input image data;
In the input image data, an area for suppressing display of the content data is determined based on an area indicating the person detected by the detection unit, and the content data is not displayed in the determined area. An output image generation unit that generates output image data by combining data with the input image data;
An information processing apparatus comprising: a display control unit configured to display the output image data generated by the output image generation unit on the display.

Images