JP2015122707A - Information processing device, method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technique that reduces trouble of a user when combining a photographic image with another image and improves accuracy in detecting a position where images should be combined.SOLUTION: An information processing device 100S and an information processing device 100M establish communication connection and exchange data with each other. If the photo-taking side information processing device 100S is not successful in trace of an AR marker 65 by image recognition processing, it instructs the information processing device 100M on a color as a display color of the AR marker 65 by preferentially selecting the color used less among colors included in a photographic image, and the information processing device 100M displays the AR marker 65 according to the instruction. The information processing device 100S detects a region indicating the AR marker 65 included in the photographic image, and combines content data according to a detection result. In addition, the information processing device 100M is able to designate content data to be displayed to the information processing device 100S by communication connection.

Description

  The present disclosure relates to a technique for combining and displaying another image on captured image data obtained by capturing with a camera.

  Information processing apparatuses equipped with cameras such as smartphones and tablet terminals are widely used. Users use these camera functions, for example, to photograph users and post them to an SNS (Social Networking Service) site, a group chat application, etc., and share the photographed image between users.

  Numerous applications are provided for information processing apparatuses such as smartphones. The information processing apparatus downloads the application from the distribution server through, for example, an application store. These applications, for example, based on the image signal generated by the camera, while displaying a preview of the captured image on the display of the information processing apparatus, accepting an operation for synthesizing another image such as an icon with the captured image from the user, For generating a composite image. A technique for synthesizing another image with a captured image is disclosed in, for example, Japanese Patent Application Laid-Open No. 2008-103873 (Patent Document 1).

  For example, the information processing apparatus starts an application, receives an input operation for designating an image to be combined with the captured image, for example, and combines the image with the captured image at a position specified by the user. For example, it is assumed that the user of the information processing apparatus has photographed another person as a subject and designated an animation image such as a flame or a sword as an image to be combined with the photographed image. Also, assume that the user designates the hand portion of the subject of the captured image as the position where the animation image is to be synthesized. The information processing apparatus displays a captured image and synthesizes an animation image at a position designated by the user (a portion of the subject's hand). Thereby, the user can synthesize the subject and the animation image, and enjoy sharing the image with other users (for example, the user who becomes the subject).

JP 2008-103873 A

  However, when a user synthesizes an animation image with a captured image, it is necessary to perform an operation for designating a position where the animation image is to be synthesized, which may make the input operation complicated for the user. For example, when it is determined which part of the subject should synthesize the animation image (for example, when the animation image is held by a hand such as a sword), the user designates the position of the subject's hand each time. Thus, it may be troublesome for the user to synthesize the animation image.

  However, when it is determined to some extent in which part of the subject the animation image should be synthesized, the position to be synthesized can also be detected by image recognition processing or the like. For example, it is assumed that an image indicating an article to be worn on the face such as glasses is synthesized with a part of the face of the subject. In this case, the information processing apparatus can detect the position of the eyes and the like by face detection processing, and can specify to some extent the position where the animation image is to be synthesized based on the detected position. This can also be expected to reduce the user's input operation. However, depending on the situation where the subject is photographed, such as the amount of light in the photographing environment, it may be difficult to specify the position where the animation image should be synthesized from the photographed image by image recognition processing or the like.

  Therefore, when combining another image with the captured image, the user's effort to adjust the position for combining the other image is reduced, and the image should be combined regardless of the shooting environment of the subject. There is a need for a technique that improves the accuracy of position detection.

  According to one embodiment, a program for controlling the operation of the information processing apparatus is provided. An information processing apparatus stores a content module combined with a camera, a communication module for communicating with another information processing apparatus, a first display for displaying an image captured by the camera, and a captured image. A memory and a processor for controlling the operation of the information processing apparatus are provided. The other information processing apparatus is configured to display a specific image detected by the image recognition process on a second display included in the other information processing apparatus. This program establishes a communication connection with another information processing apparatus in the processor, and a display setting for displaying a specific image on the second display for the other information processing apparatus that has established the communication connection. And a step of detecting a specific image included in the photographed image by performing image recognition processing on the photographed image obtained by photographing the other information processing apparatus with the camera based on the display setting. Determining a position where the content data is to be combined based on the detection result of the specific image with respect to the photographed image, generating composite image data by combining the content data at the determined position, and the generated composite image data Are displayed on the first display.

  According to another embodiment, a program for controlling the operation of the information processing apparatus is provided. The information processing apparatus includes a communication module for communicating with another information processing apparatus, a display for displaying an image, and a processor for controlling the operation of the information processing apparatus. The program includes a step of establishing a communication connection with another information processing apparatus on the processor, a step of displaying a specific image detected by the image recognition process on the display, and another information processing apparatus establishing the communication connection. Receiving a display setting for displaying the specific image on the display, and displaying the specific image in accordance with the display setting received from another information processing apparatus. Including the step of displaying.

  According to another embodiment, a method for generating a composite image in a system including a first information processing apparatus and a second information processing apparatus is provided. The first information processing device is combined with the camera, the first communication module for communicating with the second information processing device, the first display for displaying a photographed image by the camera, and the photographed image. A memory for storing content data and a processor for controlling the operation of the first information processing apparatus are provided. The second information processing apparatus includes a second communication module for communicating with the first information processing apparatus, and a second display configured to display a specific image detected by the image recognition process. . In this method, the first information processing apparatus and the second information processing apparatus establish a communication connection with each other, and the first information processing apparatus establishes a communication connection with the second information processing apparatus. A step of transmitting a display setting for displaying a specific image on the second display; and a captured image obtained by photographing the second information processing apparatus with the camera by the first information processing apparatus, The step of detecting a specific image included in the photographed image by performing image recognition processing based on the display setting, and the first information processing apparatus in the photographed image based on the detected specific image Determining a position where the data is to be combined, generating composite image data by combining the content data at the determined position, and the first information processing apparatus using the generated composite image data as the first And displaying on the display.

  According to another embodiment, an information processing apparatus having a photographing function is provided. An information processing apparatus stores a content module combined with a camera, a communication module for communicating with another information processing apparatus, a first display for displaying an image captured by the camera, and a captured image. A memory and a processor for controlling the operation of the information processing apparatus are provided, and the other information processing apparatus displays a specific image detected by the image recognition process on a second display included in the other information processing apparatus. The processor establishes a communication connection with another information processing apparatus, and sets display settings for displaying a specific image on the second display with respect to the other information processing apparatus that has established the communication connection. By performing image recognition processing based on the display settings for captured images obtained by photographing the communication processing unit to be transmitted and other information processing devices with the camera A detection unit that detects a specific image included in the captured image, and a position where the content data is combined based on a detection result of the specific image with respect to the captured image, and a combined image in which the content data is combined at the determined position The image processing unit is configured to generate data and display the generated composite image data on the first display.

  An information processing apparatus according to another embodiment includes a communication module for communicating with another information processing apparatus, a display for displaying an image, and a processor for controlling the operation of the information processing apparatus. A communication processing unit that establishes a communication connection with another information processing apparatus and receives a display setting for displaying a specific image on the display from the other information processing apparatus that has established the communication connection; According to the display setting received from the apparatus, each function of the video processing unit that displays a specific image on the display is configured.

  According to the above-described embodiment, it is possible to reduce the user's trouble of adjusting the position where another image is combined in the captured image, and it is possible to easily combine another image with the captured image. Further, by performing communication between the information processing apparatuses corresponding to the shooting environment of the subject, it is possible to improve the accuracy of detecting a position where another image is combined with the shot image.

  The above and other objects, features, aspects and advantages of the present invention will become apparent from the following detailed description of the present invention taken in conjunction with the accompanying drawings.

It is a figure which shows an example of the condition where the information processing apparatus 100 concerning embodiment is used. 1 is a diagram illustrating a configuration of a network including an information processing apparatus 100. FIG. 2 is a block diagram illustrating a configuration of an information processing apparatus 100. FIG. It is a figure which shows the content management table. It is a figure which shows an example of the user interface for synthesize | combining a picked-up image and content data in the information processing apparatus 100 by the imaging | photography side which image | photographs a to-be-photographed object. It is a figure which shows an example of the user interface for displaying AR (Augmented Reality) marker in information processing apparatus 100M, and designating the content data combined with the picked-up image of information processing apparatus 100S. 6 is a diagram illustrating an outline of processing for combining and displaying content data and a captured image by the information processing apparatus 100 according to Embodiment 1. FIG. 10 is a flowchart showing processing for displaying a marker on a display 16 by an information processing apparatus 100M as a subject. It is a flowchart which shows the process which information processing apparatus 100M used as a subject receives the communication from information processing apparatus 100S of the imaging | photography side, and changes the display of a marker. It is a flowchart which shows the process at the time of the transmission timer event for the information processing apparatus 100M used as a subject to transmit the measurement data by the information processing apparatus 100M to the information processing apparatus 100S. 10 is a flowchart showing processing for the information processing apparatus 100S on the photographing side to establish a communication connection with the information processing apparatus 100M serving as a subject and recognize the AR marker 65 displayed on the display 16 of the information processing apparatus 100M. . 12 is a flowchart illustrating processing for the information processing apparatus 100S on the photographing side to receive data through communication with the information processing apparatus 100M serving as a subject and update the setting for displaying AR data on the display 16. 7 is a flowchart illustrating processing for setting content data to be combined with a captured image on the display 16 by the information processing apparatus 100S on the imaging side. It is a flowchart which shows the process which information processing apparatus 100S used as the imaging | photography side performs an image recognition process with respect to a picked-up image, and synthesize | combines content data according to the detection result of a marker. It is a figure which shows the example which information processing apparatus 100S receives ON or OFF of the synthesis | combination of content data by the instruction | indication of information processing apparatus 100M. 4 is a diagram illustrating an example of an interface for receiving an input operation for purchasing content data to be combined with a captured image in the information processing apparatus 100. FIG.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description, the same parts are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed description thereof will not be repeated.

<Embodiment 1>
FIG. 1 is a diagram illustrating an example of a situation in which the information processing apparatus 100 according to the embodiment is used. In FIG. 1, an information processing device 100M and an information processing device 100S are shown as the information processing device 100 (hereinafter, the information processing device 100M and the information processing device 100S may be collectively referred to as the information processing device 100). . The information processing apparatus 100 is, for example, a smartphone including a touch panel, but is not limited to a smartphone. The information processing apparatus 100 includes a camera and displays a captured image captured by the camera on the display 16. In the example of FIG. 1, the information processing apparatus 100 </ b> S images the subject that holds the information processing apparatus 100 </ b> M. The information processing apparatus 100M displays an AR (Augmented Reality) marker 65 on the display 16M. The information processing apparatus 100S captures a subject with a camera and displays a captured image on the display 16S. The information processing apparatus 100S detects the AR marker 65 displayed on the display 16 of the information processing apparatus 100M held by the subject from the captured image by image recognition processing. The information processing apparatus 100S synthesizes AR model data (content data) to be synthesized with the captured image at the detected position of the AR marker 65.

  The information processing device 100S and the information processing device 100M can communicate and communicate with each other according to a short-range wireless communication protocol such as a wireless local area network (LAN) standard or Bluetooth (registered trademark). The information processing apparatus 100S communicates with the information processing apparatus 100M and emits light from the AR marker 65 in order to facilitate detection of the AR marker 65 displayed on the display 16M of the information processing apparatus 100M in the captured image by image recognition processing. Specify the color. For example, when the AR marker 65 emits light with the same color as the environment around the information processing apparatus 100M, it may be difficult to detect the AR marker 65 by the information processing apparatus 100S. Therefore, the information processing apparatus 100S instructs the information processing apparatus 100M to cause the AR marker 65 to emit light with less colors in the environment around the information processing apparatus 100M. Thereby, the accuracy of detection of the AR marker 65 by the information processing apparatus 100S is increased regardless of the environment around the subject.

  In addition, the user of the information processing apparatus 100M serving as a subject should synthesize the AR marker 65 at the position of the AR marker 65 when the AR marker 65 is detected from the captured image with respect to the information processing apparatus 100S to which the information processing apparatus 100M is connected. Content data can be specified. As a result, the user of the information processing apparatus 100M can specify to the information processing apparatus 100S which content data is to be combined in the captured image that is the subject.

  As shown in FIG. 1, the information processing apparatus 100S includes a speaker 23S and a front camera 18S. The user of the information processing apparatus 100S can easily photograph the user himself / herself by activating the front camera 18S. For example, when the user holds a marker pattern for synthesizing content data, the user of the information processing apparatus 100S can easily obtain a synthesized image of the user displayed on the display 16S and the content data. .

  In the example of FIG. 1, it is assumed that the subject user (the user of the information processing apparatus 100S) has become a character of a certain work such as a cartoon, a game, an animation, and a movie. The subject wears character costumes and maintains a certain posture. The information processing apparatus 100S detects the AR marker 65 displayed on the display 16M of the information processing apparatus 100M held by the subject by performing image recognition processing on the captured image, and synthesizes the content data based on the detection result. Set the position. Thus, for example, when the subject is a character of a certain work, the information processing apparatus 100 can synthesize effects in the work, items in the work, and the like with the photographed image. In this way, the information processing apparatus 100 can provide a sense of being a character to the subject.

<Configuration>
FIG. 2 is a diagram illustrating a network configuration including the information processing apparatus 100. In FIG. 2, information processing apparatuses 100 </ b> A, 100 </ b> B, and 100 </ b> C are illustrated as the information processing apparatus 100. The information processing apparatuses 100A and 100C are smartphones, and the information processing apparatus 100B is a tablet terminal. Referring to FIG. 2, information processing apparatus 100 </ b> A is connected to radio base station 700 by radio communication, and communicates with distribution server 900 via network 800. The distribution server 900 stores a plurality of pieces of content data for the information processing apparatus 100 to synthesize with the captured image and display on the display 16. The information processing apparatus 100 </ b> A acquires content data from the distribution server 900 by communicating with the distribution server 900. For example, the information processing apparatus 100 acquires content data for which a usable period is determined, content data that can be purchased through billing processing, and the like from the distribution server 900. The information processing apparatus 100B is connected to the router 600 by wireless communication, and communicates with the distribution server 900 via the network 800. The information processing apparatus 100C is connected to the router 600 by wireless communication, and communicates with the distribution server 900 via the network 800.

  The information processing apparatus 100B and the information processing apparatus 100C can communicate and communicate with each other according to a wireless communication standard and can transmit and receive data to and from each other. When the information processing apparatus 100B and the information processing apparatus 100C detect a specific image (for example, a marker) from the captured image by image recognition processing, the content data is combined with the captured image and displayed on the display 16. The information processing apparatus 100B and the information processing apparatus 100C communicate with each other to change the emission color of the marker displayed on the display 16 to facilitate the detection of the marker from the captured image, and the content data to be combined with the captured image The process etc. which transmit / receive are performed.

  FIG. 3 is a block diagram illustrating a configuration of the information processing apparatus 100. As shown in FIG. 3, the information processing apparatus 100 includes antennas 11 and 13, a long-distance communication module 12, a short-distance communication module 14, a touch panel 15, a display 16, a positioning sensor 17, and a front camera. 18, rear camera 19, power supply control unit 20, battery 21, microphone 22, speaker 23, sound processing unit 24, storage unit 25, acceleration sensor 26, and control unit 27.

  The antenna 11 and the antenna 13 radiate a signal emitted from the information processing apparatus 100 as a radio wave. The antenna 11 receives radio waves from the space and gives a received signal to the long-distance communication module 12. The antenna 13 receives radio waves from the space and gives a received signal to the short-range communication module 14.

  The long-distance communication module 12 and the short-distance communication module 14 perform modulation / demodulation processing for transmitting and receiving signals via the antenna 11 and the like so that the information processing apparatus 100 communicates with other wireless devices. The long-distance communication module 12 and the short-distance communication module 14 are communication modules including a tuner, a high-frequency circuit, and the like, perform modulation / demodulation and frequency conversion of radio signals transmitted and received by the information processing apparatus 100, and control received signals. 27.

  The long-distance communication module 12 is a communication module compatible with mobile communication systems such as LTE (Long Term Evolution) and UMTS (Universal Mobile Telecommunications System), and is compatible with relatively long-distance communication.

  The short-range communication module 14 corresponds to, for example, a short-range wireless communication standard such as Bluetooth (registered trademark) or a wireless LAN (Local Area Network) standard such as IEEE (Institute of Electrical and Electronic Engineers) 802.11. It is a communication module and supports relatively short distance communication. When the information processing apparatus 100 supports a plurality of communication methods, an antenna and a communication module may be provided corresponding to these communication methods.

  The touch panel 15 receives a user input operation. The touch panel 15 accepts a user's contact operation by using, for example, a capacitive type. The touch panel 15 outputs an input operation by the user to the control unit 27.

  The display 16 displays icons and background images on the menu screen, and displays still images, moving images, and the like according to application operations. The display 16 is realized by, for example, an LCD (Liquid Crystal Display) or an organic EL (electroluminescence) display.

  The positioning sensor 17 is a sensor for measuring the position of the information processing apparatus 100, and is a GPS (Global Positioning System) module, for example.

  The front camera 18 and the rear camera 19 are camera modules that receive incident light and convert them into electrical signals, and include a CMOS (Complementary Metal Oxide Semiconductor) sensor, an ADC (Analog to Digital Converter), and the like, and control image signals. To 27. In the case of the information processing apparatus 100, when the surface on which the display 16 is disposed is the front surface, the front camera 18 is disposed on the front surface of the information processing apparatus 100. The rear camera 19 is disposed on the back surface of the information processing apparatus 100.

  The power supply control unit 20 controls power supply to each circuit of the information processing apparatus 100. The power control unit 20 is, for example, a power control IC (Integrated Circuit).

  The battery 21 is a supply source that supplies power for operating each circuit of the information processing apparatus 100. The electric power from the battery 21 is supplied to each circuit according to the control of the power supply control unit 20.

  The microphone 22 receives a voice input by the user and gives a voice signal corresponding to the voice input to the voice processing unit 24.

  The speaker 23 converts the sound signal given from the sound processing unit 24 into sound and outputs the sound to the outside of the information processing apparatus 100.

  The audio processing unit 24 performs modulation / demodulation of the audio signal. The audio processing unit 24 modulates the signal given from the microphone 22 and gives the modulated signal to the control unit 27. In addition, the audio processing unit 24 gives an audio signal to the speaker 23. The sound processing unit 24 is realized by, for example, a sound processing processor.

  The storage unit 25 is configured by a flash memory, for example, and stores data and programs used by the information processing apparatus 100. In one aspect, the storage unit 25 stores AR content data 31, image data 32, and a content management table 33.

  The AR content data 31 is still image data or moving image data corresponding to identification information of each content data to be combined with a captured image.

  The image data 32 is still image data or moving image data generated by a process of storing an image obtained by combining a captured image and content data as data.

  As will be described in detail later, the content management table 33 includes identification information for identifying each content data to be combined with a captured image, a file name of the content data, a name of the content data, and a period during which the content data can be used. Is a table in which

  The control unit 27 controls the operation of the information processing apparatus 100 by reading and executing a control program stored in the storage unit 25. The control unit 27 is, for example, an application processor. The control unit 27 operates according to the program, thereby exhibiting the functions of the detection processing unit 41, the video processing unit 42, the communication processing unit 43, and the voice recognition unit 44.

  The detection processing unit 41 causes the information processing apparatus 100 to perform the function of image recognition processing, and performs image recognition processing on the captured image output from the rear camera 19 or the front camera 18 to thereby include a marker included in the captured image. Detect specific subjects such as. For example, the detection processing unit 41 performs image recognition processing on the captured image, and outputs the position of the marker included in the captured image, the inclination of the marker, and the like.

  The video processing unit 42 performs processing such as determining the position where the content data is to be combined based on the detection result of the detection processing unit 41 on the captured image, and generates a combined image. The video processing unit 42 generates a video signal based on the captured image, the generated image, and the like, and controls the display content on the display 16.

  The communication processing unit 43 controls communication processing between the information processing apparatus 100 and another information processing apparatus according to a communication protocol.

  The voice recognition unit 44 performs voice recognition processing on the voice signal output from the microphone 22, the voice signal received by the information processing apparatus 100 from another information processing apparatus, and other voice signals, and the person indicated in the voice signal Character data is generated based on the voice. The information processing apparatus 100 may operate a speech recognition processing program in its own apparatus to generate character data from the speech signal. Further, the information processing apparatus 100 transmits audio data indicating an audio signal to an external server (such as the distribution server 900), causes the external server to perform voice recognition processing, and receives the processing result from the external server. It is good as well.

<Data structure>
With reference to FIG. 4, data used in the processing of the information processing apparatus 100 will be described. FIG. 4 is a diagram showing the content management table 33.

  As shown in FIG. 4, each record of the content management table 33 includes an identification number 331, a file name 332, an AR data name 333, and an available period 334.

  The identification number 331 is identification information for identifying each content data. The file name 332 indicates the name of the file corresponding to the identification information of the content data. The AR data name 333 indicates the title of the content data indicated in the identification information. The available period 334 indicates a period during which the content data can be used. As shown in FIG. 4, for example, the data “2013/10/31” in the available period 334 indicates that the period during which the user can use the content data is October 31, 2013. “None” indicates that an available period of content data is not set.

<Example of user interface (photographing device)>
An example of a screen displayed on the display 16 by the information processing apparatus 100 will be described with reference to FIG. FIG. 5 is a diagram illustrating an example of a user interface for combining a captured image and content data in the information processing apparatus 100 on the photographing side that photographs a subject.

  As shown in FIG. 5, the control unit 27 of the information processing apparatus 100S on the photographing side includes a display switching unit 51, an image display area 52, a photographing switching unit 54, a photographing start unit 55, and a photographed image on the display 16S. A posting unit 56, a selected file display unit 57, and an instruction transmission unit 58 are displayed. Further, the control unit 27 displays content data to be combined with the captured image on the animation combining unit 53. In the example of FIG. 5, the content data shown in the selected file display unit 57 is selected by the user. The control unit 27 combines the content data shown in the selected file display unit 57 and the photographed image and displays them in the image display area 52.

  The display switching unit 51 is an area that indicates a setting as to whether or not to display on the display 16 an image in which content data designated by the user and a captured image are combined. In the example of FIG. 8, the display switching unit 51 indicates the setting “ON”. In this case, the control unit 27 displays an image generated by combining the content data and the captured image on the display 16. When the display content of the display switching unit 51 is set to “OFF”, the control unit 27 displays the captured image acquired by the camera on the display 16. The control unit 27 receives a user input operation through the touch panel 15, receives an input operation on the display switching unit 51, and sets ON whether to display an image obtained by combining a captured image and content data. Switch off.

  The image display area 52 is an area for displaying an image obtained by combining the captured image output from the camera of the information processing apparatus 100 and the content data specified by the user. In the image display area 52, an area where the content data is synthesized is shown to the animation composition unit 53. The control unit 27 detects a specific subject such as a marker through image recognition processing on the captured image, determines the animation synthesis unit 53 based on the detected position, and synthesizes content data to the animation synthesis unit 53.

  The shooting switching unit 54 is an area for receiving a setting for saving an image displayed in the image display area 52 as a still image or a moving image.

  The imaging start unit 55 is an area for accepting an input operation for saving the image displayed in the image display area 52 from the user. In the display switching unit 51, when the setting is “ON”, that is, the setting is to display an image in which the content data and the captured image are combined, the information processing apparatus 100 displays the content according to the input operation to the shooting start unit 55. Save the combined image as a still image or video.

  The photographed image posting unit 56 receives an input operation for sharing an image displayed in the image display area 52 or an image stored in response to an input operation to the photographing start unit 55 with another information processing apparatus. It is an area for receiving. The information processing apparatus 100 receives an input operation on the photographed image posting unit 56, thereby processing to transmit an image file to another information processing apparatus, accessing an SNS site for image posting, and posting an image Perform processing.

  The selected file display unit 57 is an area indicating content data selected by the user. The selected file display unit 57 is an area for receiving an operation for selecting content data from the user. For example, the information processing apparatus 100 displays a screen for receiving selection of content data from the user by receiving an input operation on the selected file display unit 57.

  The instruction transmission unit 58 is an area that communicates with the information processing apparatus 100M of the user who is the subject and receives an input operation for giving an instruction to the information processing apparatus 100M such as changing the marker color.

<User interface example (marker side device)>
With reference to FIG. 6, an example of a screen displayed on the display 16M by the information processing apparatus 100M held by the user as the subject will be described. FIG. 6 is a diagram illustrating an example of a user interface for displaying an AR marker and designating content data to be combined with a captured image of the information processing apparatus 100S in the information processing apparatus 100M.

  As illustrated in FIG. 6, the control unit 27 of the information processing apparatus 100M displays a marker display area 61, a content specifying unit 62, a data transmission unit 63, and a data purchase unit 64 on the display 16M.

  The marker display area 61 is an area for displaying the AR marker 65 detected by the image processing side information processing apparatus 100M through image recognition processing on the captured image.

  The content designation unit 62 is an area that receives an input operation for designating content data to be combined with a captured image in the information processing device 100S on the photographing side in the information processing device 100M. The control unit 27 displays an image indicating the content data specified by the user of the information processing apparatus 100M in the content specifying unit 62. For example, the control unit 27 receives an input operation on the content specifying unit 62 and displays a screen for selecting content data on the display 16M.

  The data transmission unit 63 is an area that accepts an input operation for transmitting information specifying content data specified by the user of the information processing apparatus 100M to the information processing apparatus 100S on the photographing side. The control unit 27 of the information processing device 100M receives an input operation on the data transmission unit 63, and transmits content data itself or information specifying content data such as an identifier of content data to the information processing device 100S.

  The data purchasing unit 64 is an area for accepting an input operation for purchasing content data from the user. The control unit 27 receives an input operation to the data purchase unit 64, displays a list of content data that can be purchased, and performs communication connection with an external server that distributes the content data.

<Control structure>
A control structure of the information processing apparatus 100 will be described with reference to FIG. FIG. 7 is a diagram illustrating an outline of processing for combining and displaying content data and a captured image by the information processing apparatus 100 according to the first embodiment. Note that FIG. 7 does not show a user holding the information processing apparatus 100M.

  As illustrated in FIG. 7A, the information processing apparatus 100S activates the camera and photographs the information processing apparatus 100M. The information processing apparatus 100M displays the AR marker 65 on the display 16M. However, the information processing apparatus 100S fails to detect the AR marker 65 from the photographed image due to the influence of the photographing environment (such as surrounding color and light amount). The information processing apparatus 100S communicates with the information processing apparatus 100M, and in order to make it easier to detect the AR marker 65 from the captured image, the AR marker such as the emission color of the AR marker 65 displayed in the marker display area 61 of the information processing apparatus 100M. Specify 65 display settings.

  As illustrated in FIG. 7B, the information processing apparatus 100M changes the setting of the AR marker 65 displayed on the display 16M (setting of the emission color or the like) in response to an instruction from the information processing apparatus 100S. For example, the information processing apparatus 100S refers to the distribution of each color included in the captured image, detects the smallest color among the colors, and sends an instruction to the information processing apparatus 100M to display the AR marker 65 with this color. The information processing apparatus 100S applies a filter using the color instructed to the information processing apparatus 100M, and detects the AR marker 65 from the captured image. The information processing apparatus 100S determines the position where the content data is displayed based on the position where the AR marker 65 is detected in the captured image.

  As shown in FIG. 7C, the subject-side information processing apparatus 100M can also specify content data to be combined with the photographed image in the photographing-side information processing apparatus 100S. The information processing apparatus 100M communicates with the information processing apparatus 100S, and transmits information specifying content data such as content data itself, an identifier of content data, or information indicating a method for acquiring content data to the information processing apparatus 100S. For example, the information processing apparatus 100M may communicate with the information processing apparatus 100S and acquire a list of content data shown in the content management table 33 of the information processing apparatus 100S. The information processing apparatus 100M may select information for identifying content data from the list of content data managed by the information processing apparatus 100S, and may transmit the selected identification information to the information processing apparatus 100S.

  As illustrated in FIG. 7D, the information processing apparatus 100S determines a position where the content data is to be combined based on the position where the AR marker 65 is detected from the captured image, and designates the determined position from the information processing apparatus 100M. Displayed content data.

  In the first embodiment, the content data is, for example, animation data having a three-dimensional shape. Hereinafter, a process in which the information processing apparatus 100 receives an input operation from the user via the touch panel 15 and combines the content data with the photographed image will be described.

<Control structure of information processing apparatus 100M on subject side>
FIG. 8 is a flowchart illustrating a process in which the information processing apparatus 100M as a subject displays a marker on the display 16.

  In step S801, the control unit 27 of the information processing apparatus 100M establishes a communication connection with the information processing apparatus 100S on the photographing side according to a short-range wireless communication standard such as a wireless LAN or Bluetooth (registered trademark).

  In step S803, the control unit 27 determines whether to specify content data for the information processing apparatus 100S on the photographing side. For example, the control unit 27 receives an input operation from the user as to whether to specify content data. When information processing apparatus 100M designates content data for information processing apparatus 100S (YES in step S803), control unit 27 performs the process of step S805, otherwise (NO in step S803), control unit 27 Then, the process of step S807 is performed.

  In step S805, the control unit 27 compares the data size of the content data transmitted from the information processing apparatus 100M to the information processing apparatus 100S with a threshold value, and if the data size exceeds the threshold value, Information (for example, URL (Uniform Resource Locator)) indicating a method for acquiring content data is transmitted to the information processing apparatus 100S. In addition, when the data size of the content data does not exceed the threshold value, the control unit 27 of the information processing device 100M transmits the content data itself (content data main body) to the information processing device 100S.

  In step S807, the control unit 27 displays the AR marker 65 in the marker display area 61 based on the settings for displaying the AR marker 65 (the shape of the marker to be displayed, the color to be displayed, the size to be displayed, etc.). . When the setting for displaying the AR marker 65 is designated by the information processing apparatus 100S, the control unit 27 displays the AR marker 65 in the marker display area 61 according to the designation of the information processing apparatus 100S.

  In step S809, the control unit 27 sets a timer and shifts to a state of waiting for an event to occur. In the timer set by the control unit 27 in step S809, the information processing apparatus 100M acquires information indicating the usage state of the information processing apparatus 100M by the positioning sensor 17 and the acceleration sensor 26, and transmits the acquired information to the information processing apparatus 100S. Indicates the timing to perform.

  FIG. 9 is a flowchart illustrating a process in which the information processing apparatus 100M serving as a subject receives communication from the information processing apparatus 100S on the photographing side and changes the marker display.

  In step S901, the control unit 27 of the information processing device 100M receives data indicating the setting for displaying the AR marker 65 on the display 16 from the information processing device 100S.

  In step S903, the control unit 27 updates the setting of the display method (marker color, pattern, shape, display size, etc.) of the AR marker 65 based on the data received in step S901, and displays the display according to the display method setting. 16 displays an AR marker 65.

  FIG. 10 is a flowchart showing processing when a transmission timer event occurs for the information processing apparatus 100M to be a subject to transmit measurement data from the information processing apparatus 100M to the information processing apparatus 100S. As shown in FIG. 10, the information processing apparatus 100M includes an acceleration sensor 26, a positioning sensor 17, a gyro sensor, and the like, and holds measurement data obtained by these sensors. The information processing apparatus 100M transmits measurement data to the information processing apparatus 100S according to the cycle indicated by the timer.

  In step S1001, the control unit 27 of the information processing device 100M reads measurement data of various sensors of the information processing device 100M from the memory.

  In step S1003, the control unit 27 transmits the measurement data read from the memory to the information processing apparatus 100S on the photographing side.

  In step S1005, the control unit 27 sets a transmission timer for transmitting measurement data of various sensors of the information processing apparatus 100M.

  The control unit 27 performs the process shown in FIG. 10 every time an event is generated by the transmission timer set in step S1005.

  FIG. 11 illustrates a process for the information processing apparatus 100S on the photographing side to establish a communication connection with the information processing apparatus 100M serving as a subject and recognize the AR marker 65 displayed on the display 16 of the information processing apparatus 100M. It is a flowchart to show.

  In step S1101, the control unit 27 of the information processing apparatus 100S establishes a communication connection with the information processing apparatus 100M serving as a subject according to the short-range wireless communication standard.

  In step S1103, the control unit 27 performs processing for setting content data (AR model data) to be combined with the captured image when the AR marker 65 is detected from the captured image. Details of the processing shown in step S1103 will be described later.

  In step S <b> 1105, the control unit 27 performs initialization processing for exhibiting the photographing function of the front camera 18, for example, sets a capture cycle for acquiring a photographed image, and performs photographing by the front camera 18. Start.

  FIG. 12 is a flowchart illustrating processing for the information processing apparatus 100S on the photographing side to receive data through communication with the information processing apparatus 100M serving as the subject and update the setting for displaying the AR data on the display 16. .

  In step S1201, the control unit 27 of the information processing device 100S receives various types of data such as measurement data of various sensors of the information processing device 100M and content data specified by the information processing device 100M from the information processing device 100M on the photographing side. Receive.

  In step S1203, the control unit 27 performs processing for setting content data (AR model data) to be combined with the captured image when the AR marker 65 is detected from the captured image. Details of the processing shown in step S1203 will be described later.

  In step S1205, the control unit 27 updates parameters such as a direction for displaying content data, which is three-dimensional AR data, on the display 16 based on the data received in step S1201.

  FIG. 13 is a flowchart showing a process for setting content data to be combined with a photographed image on the display 16 by the information processing apparatus 100S on the photographing side. The process shown in FIG. 13 corresponds to each process of step S1103 and step S1203.

  In step S1301, when the control unit 27 of the information processing apparatus 100S receives the designation of content data from the information processing apparatus 100M, the control unit 27 refers to the memory, sets the content data to be combined with the captured image, and the information processing apparatus 100M. Compare the content data specified by. As a result of the comparison, if the setting of the content data stored in the memory matches the content data specified from the information processing apparatus 100M, the control unit 27 determines that it is not necessary to change the setting of the content data ( In step S1301, “no change”). If the setting of the content data stored in the memory and the specification of the content data by the information processing apparatus 100M do not match, the control unit 27 determines that the setting of the content data needs to be changed (in step S1301). , “Changed”), the process of step S1303 is performed.

  In step S1303, the control unit 27 determines whether the content data itself is received from the information processing apparatus 100M, or whether information on a URL from which content data can be acquired is received. When the content data itself is received from information processing apparatus 100M (YES in step S1303), control unit 27 performs the process of step S1307. If the control unit 27 has received URL information from which content data can be acquired (NO in step S1303), the control unit 27 performs the process of step S1305.

  In step S1305, the control unit 27 receives content data based on the URL received from the information processing apparatus 100M, and sets the received content data as content data to be combined with the captured image.

  In step S1307, the control unit 27 sets the content data received from the information processing apparatus 100M as content data to be combined with the captured image.

  FIG. 14 is a flowchart illustrating processing in which the information processing apparatus 100S on the photographing side performs image recognition processing on a photographed image and synthesizes content data according to the marker detection result. Each time the information processing apparatus 100S acquires a captured image according to the capture cycle and displays a preview of the acquired captured image on the display 16, the information processing apparatus 100S performs the process illustrated in FIG.

  In step S1401, the control unit 27 of the information processing device 100S has finished the process of tracing the AR marker (AR marker 65 displayed on the display 16 of the information processing device 100M) from the captured image (for detecting the AR marker). Success or failure), or whether the process of tracing the AR marker is being executed. If the process of tracing the AR marker has ended (YES in step S1401), control unit 27 performs the process of step S1407. When the process of tracing the AR marker is being executed (NO in step S1401), the control unit 27 performs the process of step S1403.

  In step S1403, the control unit 27 determines whether or not the AR marker has been successfully traced by the image recognition processing on the captured image. The control unit 27 performs the process of step S1405 if the AR marker has been successfully traced (YES in step S1403), and performs the process of step S1425 otherwise (NO in step S1403).

  In step S1405, the control unit 27 determines a position where the content data is displayed based on the coordinate value of the AR marker detected by the image recognition process on the captured image, and performs various processes of the information processing apparatus 100M received from the information processing apparatus 100M. Based on the sensor measurement data, the direction of the three-dimensional content data is determined. The control unit 27 synthesizes the content data with the photographed image and displays it on the display 16 according to the position and orientation in which the content data is displayed.

  In step S1407, the control unit 27 of the information processing device 100S determines whether or not it is a mode instructing the information processing device 100M to change the setting for displaying the AR marker 65 (color change waiting mode). To do. If the control unit 27 determines that the color change waiting mode is set (YES in step S1407), the control unit 27 performs the process of step S1409. If control unit 27 determines that it is not the color change waiting mode (NO in step S1407), it performs the process of step S1417.

Therefore, a case where the information processing apparatus 100S is in the color change waiting mode will be described.
In step S1409, the control unit 27 applies filter processing to the captured image using the color designated for displaying the AR marker 65 to the information processing apparatus 100M.

  In step S1411, the control unit 27 detects the AR marker 65 based on the image obtained by applying the filter process to the captured image, and acquires the coordinate value of the AR marker 65 in the captured image.

  In step S1413, the control unit 27 refers to the coordinate value acquired by the process in step S1411 and determines whether or not the coordinate value of the AR marker 65 is uniquely determined. If the coordinate value of the AR marker 65 is uniquely determined (YES in step S1413), the coordinate value is set as the coordinate of the AR marker 65 in the photographed image, and the process shifts from the color change waiting mode to the mode in which the AR marker is traced, and in step S1415. Perform the process. The control unit 27 resets the value of the variable “n” indicating the number of times the marker display color has been changed. When the coordinate value of the AR marker 65 is not uniquely determined (the coordinate value is not detected or a plurality of coordinate values are detected) (NO in step S1413), the control unit 27 performs the processing after step S1419, The display color of the marker displayed on the information processing apparatus 100M is changed.

  In step S1415, the control unit 27 starts a process of tracing the AR marker based on the coordinate value of the AR marker 65 detected from the captured image.

  Next, a case where the information processing apparatus 100S is not in the color change waiting mode in step S1407 will be described.

  In step S1417, the control unit 27 refers to the memory and determines whether or not the process for changing the setting for displaying the AR marker 65 on the information processing apparatus 100M is the first time. The control unit 27 manages a variable “n” indicating the number of times the marker display color has been changed, and determines whether or not the value of the variable “n” is 1, for example. If the process for changing the setting for displaying AR marker 65 is the first time (YES in step S1417), control unit 27 performs the process of step S1425; otherwise (NO in step S1417), the control unit 27 performs the process of step S1419.

  In step S1425, the control unit 27 refers to the color distribution included in the photographed image and selects a color having the smallest distribution from the colors constituting the photographed image. For example, the control unit 27 classifies the colors included in the captured image into several groups such as red, blue, and green, and selects the group with the fewest colors.

  In step S1427, the control unit 27 sends an instruction to the information processing apparatus 100M to change the emission color of the AR marker 65 based on the color selected in step S1425. The information processing apparatus 100S shifts to the color change waiting mode.

  In step S1429, the control unit 27 updates the variable “n” indicating the number of times the marker display color has been changed. Here, since this is the first process of changing the display color of the AR marker 65 displayed on the information processing apparatus 100M, for example, the value of the variable “n” is set to “1”.

  Further, in step S1419 (when the process of changing the display method of the AR marker 65 is not the first time), the control unit 27 refers to the value of the variable “n” and determines the value among the color distributions included in the captured image. Select the “n” th least color.

  In step S1421, the control unit 27 transmits an instruction to the information processing apparatus 100M to cause the AR marker 65 to emit light with the color selected in step S1419, and shifts the information processing apparatus 100S to the “color change waiting mode”.

  In step S <b> 1423, the control unit 27 updates the value of the variable “n” by counting up.

  By performing the processing described above, the information processing apparatus 100S on the photographing side and the information processing apparatus 100M serving as the subject synthesize content data with the photographed image while communicating with each other. (1) The information processing apparatus 100S instructs the information processing apparatus 100M to set the AR marker 65 so that the AR marker 65 can be easily detected from the captured image. (2) The information processing apparatus 100M sends content data from the information processing apparatus 100M to the information processing apparatus 100S in order to make the content data displayed on the information processing apparatus 100S desired by the user of the information processing apparatus 100M. Information for designating (content data itself, URL at which content data can be acquired, etc.) is transmitted.

<Switching settings for whether or not to synthesize content data with captured images>
With reference to FIG. 15, an example will be described in which the setting as to whether or not to synthesize content data with the captured image of the information processing apparatus 100S is switched from the information processing apparatus 100M that is the subject. FIG. 15 is a diagram illustrating an example in which the information processing apparatus 100S receives content data composition ON or OFF according to an instruction from the information processing apparatus 100M.

<1. ON-OFF control by reception of designation from information processing apparatus 100M>
For example, the information processing apparatus 100S may switch ON / OFF of the composition of content data by receiving specification of content data from the information processing apparatus 100M. In FIG. 15, the user of the information processing apparatus 100M serving as a subject performs an input operation on the data transmission unit 63 displayed on the display 16M. Thereby, information for designating content data is transmitted from the information processing apparatus 100M to the information processing apparatus 100S. When the information processing apparatus 100S receives information for specifying content data from the information processing apparatus 100M, the information processing apparatus 100S turns on the composition of the content data and superimposes the content data on the animation composition unit 53 of the image display area 52.

<2. ON-OFF control based on voice recognition processing in information processing apparatus 100M>
In addition, the information processing apparatus 100M receives voice input from the user of the information processing apparatus 100M via the microphone 22, and switches ON / OFF of content data synthesis in the information processing apparatus 100S based on the received voice. It is good. For example, the information processing apparatus 100M performs voice recognition processing on the voice signal output from the microphone 22 and acquires text data indicated by the voice signal. The information processing apparatus 100M may transmit content data to the information processing apparatus 100S as an input operation performed on the data transmission unit 63 according to a command indicated in the text data.

  Further, the information processing apparatus 100M may instruct the information processing apparatus 100S to turn on and off the composition of content data in accordance with a command indicated in the text data acquired by the speech recognition process. As a result, the user of the information processing apparatus 100M can control the ON / OFF timing of content data synthesis in the information processing apparatus 100S at a timing desired by the user.

<3. On-off control based on vibration detection in information processing apparatus 100M>
Further, the information processing apparatus 100M detects a motion (shake) that the user of the information processing apparatus 100M shakes while holding the information processing apparatus 100M based on the measurement result of the acceleration sensor 26 or the like. An instruction to switch ON / OFF of the composition of content data may be transmitted to 100S.

  As a result, when the user of the information processing apparatus 100M has become a character of a certain work, for example, the user himself can easily determine the timing at which the content data indicating the characteristics of the character is synthesized in the information processing apparatus 100S. Can be controlled. Therefore, the information processing apparatus 100S and the information processing apparatus 100M can provide the user with a sense of being a character. In the example of FIG. 15, the user of the information processing apparatus 100M controls the timing at which content data indicating an energy wave is displayed in the image display area 52 of the information processing apparatus 100S.

  In addition, the information processing apparatus 100S includes, for example, a moving image that includes the image before the content data and the captured image are combined and the image after the combination including the process of switching ON and OFF of the content data combining. It can also be saved by, for example. The information processing apparatus 100 </ b> S may be able to share a stored video or the like with other users. For example, the information processing apparatus 100S may be able to post a still image or a moving image generated by synthesizing content data to a captured image, for example, on an image posting website or group chat. Thereby, the variation of how to enjoy a work between users can be expanded.

<Example of user interface for accepting purchase of content data>
An example of a user interface for the information processing apparatus 100 to accept purchase of content data from a user will be described with reference to FIG. FIG. 16 is a diagram illustrating an example of an interface for receiving an input operation for purchasing content data to be combined with a captured image in the information processing apparatus 100.

  As shown in FIG. 16, the control unit 27 displays a purchase button 72 and a content data list 71 on the display 16. The content data list 71 displays a list of content data that can be purchased by the user.

  The control unit 27 receives an input operation for the user to select content data via the touch panel 15. The purchase button 72 is a button for accepting an input operation for purchasing content data selected by the user.

<Embodiment 2>
Next, the information processing apparatus 100 according to the second embodiment will be described. In the second embodiment, a plurality of information processing apparatuses 100 establish a communication connection (for example, another apparatus establishes a communication connection with respect to one information processing apparatus 100). In the plurality of information processing apparatuses 100, each of the information processing apparatuses 100 serving as subjects displays a marker on the display 16.

  The information processing apparatus 100 on the photographing side recognizes each of a plurality of markers included in the photographed image, and displays the recognized markers on the display 16 in association with the content data. For example, the information processing apparatus 100 registers a plurality of markers to be detected when performing image recognition processing on a captured image. The information processing apparatus 100 associates each of these markers with the content data and holds them in the memory. As a result, each information processing device that is a subject displays a marker on the display 16, and the information processing device 100 on the photographing side is positioned at the position of each marker displayed on the information processing device 100 that is the subject in the captured image. Content data can be synthesized. For example, two subjects display a marker on each information processing apparatus 100 and associate each subject with content data. As a result, the information processing apparatus 100 can display on the display 16 a scene in which a plurality of subjects are holding saber-like weapons having different emission colors, for example.

<Modification Example Content Data Distribution Method>
In the above description of the embodiment, the information processing apparatus 100 has been described as being able to acquire the AR content data 31 from, for example, the distribution server 900. However, for example, the information processing apparatus 100 is limited to a specific area, for example. The AR content data 31 may be acquired. For example, the distribution server 900 that distributes content data uses the position of the information processing apparatus 100 as a result of (a) positioning by the GPS of the positioning sensor 17, and (b) a wireless base station with which the information processing apparatus 100 is performing wireless communication. Information such as identification information 700, (c) identification information of the wireless LAN router with which the information processing apparatus 100 is performing wireless communication is acquired. When the distribution server 900 receives a request from the information processing apparatus 100 to acquire the AR content data 31 whose distribution area is limited, the distribution server 900 stores the AR content data 31 based on the position information of the information processing apparatus 100. Whether or not distribution is possible is controlled (or it may be determined whether or not the information processing apparatus 100 is in a region where the AR content data 31 can be distributed).

  In this way, by limiting the areas in which the information processing apparatus 100 can acquire the AR content data 31, for example, the user visits an event or a sightseeing spot held in an area related to the AR content data 31. Can be encouraged. For example, the distribution server 900 distributes content data indicating local products, articles related to sightseeing spots, items appearing in the production of animation works related to the area, etc. to the information processing apparatus 100 in a limited area. It is good as well.

  The embodiment disclosed this time must be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

  11, 13 Antenna, 12 Long-distance communication module, 14 Short-distance communication module, 15 Touch panel, 16 Display, 17 Position sensor, 18 Front camera, 19 Rear camera, 20 Power supply control unit, 21 Battery, 22 Microphone, 23 Speaker , 24 voice processing unit, 25 storage unit, 26 acceleration sensor, 27 control unit, 31 AR content data, 32 image data, 33 content management table, 41 detection processing unit, 42 video processing unit, 43 communication processing unit, 44 voice recognition Part, 100 information processing device, 600 router, 700 wireless base station, 800 network, 900 distribution server.

Claims (10)

A program for controlling the operation of the information processing apparatus,
The information processing apparatus stores a camera, a communication module for communicating with another information processing apparatus, a first display for displaying an image captured by the camera, and content data combined with the captured image. And a processor for controlling the operation of the information processing apparatus,
The other information processing apparatus is configured to display a specific image detected by an image recognition process on a second display included in the other information processing apparatus,
The program is stored in the processor.
Establishing a communication connection with the other information processing apparatus;
Transmitting display settings for displaying the specific image on the second display to the other information processing apparatus that has established the communication connection;
Detecting the specific image included in the photographed image by performing image recognition processing on the photographed image obtained by photographing the other information processing apparatus with the camera based on the display setting; ,
Determining a position to synthesize the content data based on a detection result of the specific image with respect to the captured image, and generating synthesized image data by synthesizing the content data at the determined position;
Displaying the generated composite image data on the first display.   The transmitting step obtains a distribution of colors included in the photographed image, determines a display color of the specific image by giving priority to colors having a small number of colors included in the photographed image in the distribution, and determines the determined The program according to claim 1, further comprising a step of including a display color in the display setting and transmitting the display color to the other information processing apparatus.   The detecting step acquires and acquires an image after applying the filter by applying a filter of a display color of the specific image included in the display setting transmitted to the other information processing apparatus to the captured image. The program according to claim 2, further comprising: detecting the specific image included in the captured image by performing the image recognition process on the image. The program is stored in the processor.
Receiving a designation of content data to be combined with the first display from the other information processing apparatus;
The program according to claim 1, wherein the displaying includes synthesizing the content data designated by the other information processing apparatus with the photographed image. A program for controlling the operation of the information processing apparatus,
The information processing apparatus includes a communication module for communicating with another information processing apparatus, a display for displaying an image, and a processor for controlling the operation of the information processing apparatus,
The program is stored in the processor.
Establishing a communication connection with the other information processing apparatus;
Displaying a specific image detected by an image recognition process on the display;
Receiving a display setting for displaying the specific image on the display from the other information processing apparatus that has established the communication connection; and
The step of displaying the specific image includes a step of displaying the specific image on the display according to the display setting received from the other information processing apparatus. The program is stored in the processor.
The program according to claim 5, wherein a step of transmitting designation of the content data to be combined with the captured image of the other information processing apparatus is executed to the other information processing apparatus that has established the communication connection. The information processing apparatus includes a memory for storing content data to be combined with a captured image in the other information processing apparatus,
The transmitting step includes a step of transmitting at least one of the content data main body, the identification information of the content data, or information indicating an acquisition method of the content data to the other information processing apparatus. 6. The program according to 6. A method for generating a composite image in a system including a first information processing apparatus and a second information processing apparatus,
The first information processing apparatus includes a camera, a first communication module for communicating with the second information processing apparatus, a first display for displaying an image captured by the camera, and the captured image. A memory for storing the content data to be combined; and a processor for controlling the operation of the first information processing apparatus;
The second information processing apparatus includes a second communication module for communicating with the first information processing apparatus, and a second display configured to display a specific image detected by an image recognition process. With
The method
The first information processing apparatus and the second information processing apparatus establishing a communication connection with each other;
The first information processing apparatus transmitting a display setting for displaying the specific image on the second display to the second information processing apparatus that has established the communication connection;
Included in the captured image by performing image recognition processing based on the display setting for a captured image obtained by capturing the second information processing apparatus with the camera by the first information processing apparatus. Detecting the specific image to be detected;
The first information processing apparatus determines a position for combining the content data in the captured image based on the detected specific image, and combines the content data at the determined position. Generating image data; and
The first information processing apparatus includes a step of displaying the generated composite image data on the first display. An information processing apparatus having a photographing function,
The information processing apparatus stores a camera, a communication module for communicating with another information processing apparatus, a first display for displaying an image captured by the camera, and content data combined with the captured image. And a processor for controlling the operation of the information processing apparatus,
The other information processing apparatus is configured to display a specific image detected by an image recognition process on a second display included in the other information processing apparatus,
The processor is
Communication that establishes a communication connection with the other information processing apparatus and transmits a display setting for displaying the specific image on the second display to the other information processing apparatus that has established the communication connection. A processing unit;
A detection unit that detects the specific image included in the photographed image by performing image recognition processing on the photographed image obtained by photographing the other information processing apparatus with the camera based on the display setting. When,
Based on a detection result of the specific image with respect to the photographed image, a position to synthesize the content data is determined, combined image data is generated by combining the content data at the determined position, and the generated An information processing apparatus configured to perform each function of a video processing unit that displays composite image data on the first display. An information processing apparatus,
The information processing apparatus includes a communication module for communicating with another information processing apparatus, a display for displaying an image, and a processor for controlling the operation of the information processing apparatus,
The processor is
A communication processing unit that establishes a communication connection with the other information processing apparatus and receives a display setting for displaying the specific image on the display from the other information processing apparatus that has established the communication connection;
An information processing apparatus configured to exhibit each function of a video processing unit that displays the specific image on the display in accordance with the display setting received from the other information processing apparatus.

Images