JP2020017176A - Information processing device - Google Patents

Abstract

To display an image having a movement in accordance with the distance between a plurality of virtual objects.SOLUTION: Between AR marker groups having AR marker identification information related to one combination, a moving image corresponding to this set is superimposed. For example, a moving image is displayed on a virtual plane a formed of an AR marker M1 and an AR marker M2. This moving image is a moving image in which the state when an actual player corresponding to a character C1 superimposed on the AR marker M1 and an actual player corresponding to a character C2 superimposed on the AR marker M2 are playing in an actual space is recorded. The display size of this moving image is the size corresponding to the distance between the AR markers M1 and M2.SELECTED DRAWING: Figure 9

Description

  The present invention relates to a technique for superimposing and displaying an image on a captured image.

  The so-called AR (augmented reality) extends the real world as seen by humans by adding, deleting, emphasizing, and attenuating information in the real space. For example, Patent Literature 1 describes that when two game cards are brought into contact, a virtual object is displayed between the characters. Furthermore, it is described that when the game cards are separated from each other, the virtual object is expressed so as to extend, and when the game cards are close to each other, the virtual object is expressed so as to be contracted.

JP 2006-72668 A

  It is considered that this AR is used for music reproduction. For example, when an AR marker as a real object is imaged, a character performing a music performance is displayed as a virtual object superimposed on the captured image. When displaying such a moving virtual object, for example, if a moving image corresponding to a plurality of virtual objects can be superimposed and displayed, there is a possibility that an interesting service for a user may be provided.

  Therefore, an object of the present invention is to provide a mechanism capable of displaying a moving image according to the distance between a plurality of virtual objects.

  In order to solve the above-described problem, the present invention provides an acquisition unit that acquires a captured image, a detection unit that detects a real object on which identification information is written in the captured image acquired by the acquisition unit, A calculating unit that calculates a distance between the plurality of real objects when a plurality of the real objects are detected, and a virtual object corresponding to the identification information of the real object detected by the detecting unit, A display control unit configured to superimpose and display a moving image in a mode corresponding to the distance calculated by the calculation unit on the captured image when the detection unit detects the plurality of real objects. An information processing apparatus comprising: a display control unit.

  The display control unit may superimpose the virtual object performing the operation and superimpose the moving image in conjunction with the operation of the virtual object.

  The display control unit may superimpose and display the virtual object performing the performance operation, and superimpose and display the moving image including a player performing the performance operation corresponding to the performance operation of the virtual object.

  The display control unit may superimpose and display the moving image having a size corresponding to the distance calculated by the calculation unit.

  The display control unit may superimpose and display the moving image having contents corresponding to the distance calculated by the calculation unit.

  The display control unit may superimpose and display the moving image on a virtual plane in a direction corresponding to the direction of the line segment on a line segment connecting the plurality of real objects.

  The display control unit may superimpose and display the moving image on a virtual plane in a direction corresponding to a viewing direction of a user who views the captured image.

  When a predetermined operation is performed by a user who browses the captured image, one of the plurality of real objects is moved, and the display control unit moves between the plurality of real objects. Even if the distance of the moving image changes, the display mode of the moving image may not be changed.

  The display control unit may superimpose and display the moving image in a mode corresponding to a combination of the identification information of the plurality of real objects detected by the detection unit.

  The display control unit may superimpose and display the moving image in a mode corresponding to a position of a virtual object corresponding to one of the plurality of real objects detected by the detection unit.

  The present invention also provides an acquisition step of acquiring a captured image, a detection step of detecting a real object with identification information written in the captured image acquired in the acquisition step, and a plurality of the real objects in the detection step. Calculating a distance between the plurality of real objects, and displaying a virtual object corresponding to the identification information of the real object detected in the detecting step on the captured image in a superimposed manner. A control step, wherein when the plurality of real objects are detected in the detection step, a display control step of superimposing and displaying a moving image in a mode corresponding to the distance calculated in the calculation step on the captured image. An information processing method characterized by comprising:

  The present invention also provides a computer with an acquisition step of acquiring a captured image, a detection step of detecting a real object with identification information in the captured image acquired in the acquisition step, A step of calculating a distance between the plurality of real objects when the real object is detected; and superimposing a virtual object corresponding to the identification information of the real object detected in the detection step on the captured image. A display control step of displaying the moving image in a manner corresponding to the distance calculated in the calculating step when the plurality of real objects are detected in the detecting step; And a program for executing the steps.

  According to the present invention, it is possible to display a moving image in accordance with the distance between a plurality of virtual objects.

1 is a diagram illustrating a configuration of an information processing system according to an embodiment of the present invention. FIG. 3 is a block diagram illustrating an example of a hardware configuration of the imaging display device. FIG. 2 is a block diagram illustrating an example of a hardware configuration of an information processing device. FIG. 4 is a diagram illustrating an example of an image database stored in the information processing device. FIG. 9 is a diagram illustrating an example of a display size table stored in the information processing device. FIG. 2 is a block diagram illustrating an example of a functional configuration of the information processing system. 9 is a flowchart illustrating an example of an operation of the information processing device. It is a figure showing the example of a display in an image pick-up display. It is a figure showing the example of a display in an image pick-up display.

[Constitution]
FIG. 1 is a diagram illustrating an example of an information processing system 1 according to the present embodiment. The information processing system 1 includes an information processing apparatus 10 that performs processing for realizing a so-called AR (augmented reality), an imaging display apparatus 20 that captures and displays an image, and an information processing apparatus 10 and an imaging display apparatus 20. And a network 90 communicably connected. The network 90 is, for example, a LAN (Local Area Network) or a WAN (Wide Area Network), or a combination thereof, and includes a wired section or a wireless section. Although FIG. 1 shows one information processing device 10 and one imaging display device 20, each of them may be plural.

  The information processing device 10 controls the imaging display device 20 so as to acquire a captured image of the real space captured by the imaging display device 20, and to display the additional image superimposed on the real space. Specifically, the information processing device 10 detects a real object called an AR marker in a captured image of the real space captured by the imaging display device 20, and superimposes a virtual object corresponding to the real object on the real space. Is displayed on the imaging display device 20. Further, the information processing apparatus 10 displays a moving image in a mode corresponding to the distance between a plurality of real objects on the image display device 20 in a state of being superimposed on the real space in the image of the real space imaged by the image display device 20. Let it.

  Here, the AR marker is a sign on which an image to be superimposed and displayed in the real space in the AR and identification information for designating the position are written. The identification information is described on the AR marker by a graphic, a character, or the like, but frequently used is a monochrome graphic, a QR code (registered trademark), or the like. In the present embodiment, an AR marker in which a black-and-white figure is written on a circular medium is used, but the shape, size, and expression format of the identification information may be any. Since the virtual object is displayed in a state of being superimposed on the AR marker, the user feels as if the virtual object exists in the real space. Virtual objects are also called AR objects.

  In the present embodiment, a more complicated example in which the imaging display device 20 displays an image under the control of the information processing device 10 will be described. However, the present invention is not limited to such an example. The operation related to image display may be performed alone without receiving the control of the processing device 10. That is, the imaging display device 20 detects a plurality of AR markers in a captured image of the real space captured by the imaging unit 206 of the own device, and superimposes a moving image in a mode corresponding to the distance between the real objects on the real space. May be displayed on the display unit 207.

  FIG. 2 is a diagram illustrating a hardware configuration of the imaging display device 20. The imaging display device 20 is, for example, a glass-type, head-mount display-type, or contact lens-type wearable computer, or an information processing device such as a smartphone or a tablet. As a device for realizing AR, a type in which a virtual object is superimposed on a captured image in a display surface for displaying a captured image in a real space and a light transmission type display surface arranged in front of a user's eyes, There is a type in which a virtual object is displayed at a position on a display surface that overlaps a predetermined position in a real space that is visible through a display surface to a user. The imaging display device 20 according to the present embodiment may be of any type, and functions as a device that superimposes and displays a virtual object on a real space.

  The imaging display device 20 includes a CPU 201 (Central Processing Unit), a ROM (Read Only Memory) 202, a RAM (Random Access Memory) 203, an auxiliary storage device 204, a communication IF 205, an imaging unit 206, a display unit 207, and an operation unit 208. .

  The CPU 201 is a processor that performs various operations. The ROM 202 is a non-volatile memory that stores, for example, a program and data used to start the imaging display device 20. The RAM 203 is a volatile memory that functions as a work area when the CPU 201 executes a program. The auxiliary storage device 204 is a non-volatile storage device such as a hard disk drive (HDD) or a solid state drive (SSD), and stores programs and data used in the imaging display device 20. When the CPU 201 executes the program, the imaging display device 20 functions as a computer device. The communication IF 205 is an interface for performing communication via the network 90 according to a predetermined communication standard. This communication standard may be a wireless communication standard or a wired communication standard. In the case of wireless communication, the communication IF 205 includes an antenna and a communication circuit that operate according to a communication standard such as LTE (Long Term Evolution) or Wi-Fi (registered trademark). The imaging unit 206 includes a lens and an optical element, and generates captured image data indicating a captured image. The display unit 207 includes, for example, a liquid crystal panel and a liquid crystal driving circuit, and displays an image according to the image data. The operation unit 208 includes, for example, an operation element such as a key and a touch sensor, receives an operation by the user, and supplies a signal corresponding to the operation to the CPU 201. Note that the imaging display device 20 may include other components such as a GPS (Global Positioning System) unit and an audio input / output unit in addition to the configuration illustrated in FIG.

  FIG. 3 is a diagram illustrating a hardware configuration of the information processing apparatus 10. The information processing device 10 is a computer device having a CPU 101, a ROM 102, a RAM 103, an auxiliary storage device 104, and a communication IF 105. The CPU 101 is a processor that performs various operations. The ROM 102 is a non-volatile memory that stores, for example, a program and data used for activating the information processing device 10. The RAM 103 is a volatile memory that functions as a work area when the CPU 101 executes a program. The auxiliary storage device 104 is a nonvolatile storage device such as an HDD or an SSD, and stores programs and data used in the information processing device 10. When the CPU 101 executes this program, a function shown in FIG. 6 described later is realized. The communication IF 105 is an interface for performing communication via the network 90 according to a predetermined communication standard. The information processing apparatus 10 may include other components such as a display unit and an operation unit, in addition to the configuration illustrated in FIG.

  The auxiliary storage device 104 stores an image database (hereinafter, the database is abbreviated as DB) and a display size table. FIG. 4 is a diagram illustrating an example of the image DB. The image DB includes identification information (AR marker identification information) written on the AR marker, data (character reproduction data) for reproducing a character superimposed and displayed on the AR marker in the imaging display device 20, and a plurality of (character reproduction data). In the embodiment, data (moving image reproduction data) for reproducing the moving image determined for the combination of the two AR markers on the imaging and display device 20 is associated with the combination. For example, a character indicated by character reproduction data having a data name of “C001” is superimposed and displayed on an AR marker on which AR marker identification information “ID001” is described. For example, a character indicated by character reproduction data having a data name of “C002” is superimposed on an AR marker on which AR marker identification information “ID002” is described. The character referred to here simulates, for example, the appearance of each member constituting a group (band) playing music. The character reproduction data includes image data and audio data. When the character reproduction data is reproduced on the imaging display device 20, it is displayed as if the character is performing a music operation, and the sound of the music resulting from the performance is output.

  The AR marker identification information “ID001” and the AR marker identification information “ID002” constitute one set, and the AR marker identification information “ID003” and the AR marker identification information “ID004” are one other than the above. Make up a set. Each of the AR marker groups in which AR marker identification information related to one combination is described corresponds to each of the members constituting a group (band) playing a certain music. For example, a character playing a guitar for a certain piece of music is reproduced by character reproduction data “C001” corresponding to a combination of AR marker identification information “ID001” and “ID002”, and character reproduction data “C002” For example, a character performing a vocal performance for the music is reproduced.

  Also, the music played by each of the AR marker groups related to one combination is synchronized. For example, the movement and voice of a character performing a guitar performance based on the character reproduction data “C001” and the motion and voice of a character performing a vocal performance based on the character reproduction data “C002” are reproduced in synchronization with the progress of the music performance. Is done. In other words, these characters are reproduced as if they were playing one piece of music in each part at the same time. For the synchronous reproduction, the character reproduction data “C001” and the character reproduction data “C002” include a common time code, and these data are reproduced in a synchronized state according to the time code.

  Further, between AR markers of one combination, a moving image is superimposed and displayed by moving image reproduction data corresponding to this combination. For example, the moving image indicated by the moving image reproduction data having the data name “A001” is superimposed and displayed between the AR marker with the AR marker identification information “ID001” and the AR marker with the AR marker identification information “ID002”. Is done. This moving image includes a real player (a member of the band) corresponding to the character indicated by the character reproduction data “C001” and a real player (a band member) corresponding to the character indicated by the character reproduction data “C002”. Members) are performing at live concerts and concerts in the real space. That is, the moving image reproduction data includes image data and audio data. Further, the music reproduced at each of the AR markers according to one combination and the moving image superimposed and displayed between these AR markers are synchronized (that is, linked). For example, the movement and voice of the character performing the guitar playing and the character reproduction data of "C002" are reproduced by the character reproduction data of "C001" corresponding to the combination of the AR marker identification information "ID001" and "ID002". The motion and voice of the character performing the vocal performance are reproduced in synchronization with the motion and voice of the real player playing in the real space, which is reproduced by the moving image reproduction data “A001”. You. In other words, in the imaging display device 20, each character is played as if it were playing one piece of music at the same time in each part, and the recorded moving image being played is played back in synchronization with the performance operation of the character. . For this synchronous reproduction, the character reproduction data “C001”, the character reproduction data “C002”, and the moving image reproduction data “A001” include a common time code, and these data are synchronized according to this time code. Will be played back.

  FIG. 6 is a block diagram illustrating an example of a functional configuration of the information processing system 1. As illustrated in FIG. 6, in the information processing apparatus 10, functions of an acquisition unit 11, a detection unit 12, a calculation unit 13, and a display control unit 14 are realized. A captured image is obtained from the imaging unit 206 via the communication IF 205. The detection unit 12 detects an AR marker in the captured image acquired by the acquisition unit 11. The calculating unit 13 calculates a distance between the plurality of AR markers when the detecting unit 12 detects the plurality of AR markers. The display control unit 14 performs display control on the display unit 207 of the imaging and display device 20 via the communication IF 205 to convert a character corresponding to the AR marker identification information of the AR marker detected by the detection unit 12 into a captured image. Display is superimposed. Further, when the detection unit 12 detects a plurality of AR markers, the display control unit 14 superimposes and displays a moving image in a mode corresponding to the distance calculated by the calculation unit 13 on the captured image.

  As described above, in the present embodiment, an example in which the imaging display device 20 displays an image under the control of the information processing device 10 will be described. Therefore, the functional configuration is as shown in FIG. However, as described above, in the present invention, the imaging display device 20 may perform an operation related to image display alone without being controlled by the information processing device 10. In this case, the imaging and display device 20 includes a plurality of acquisition units 11 that acquire a captured image from the imaging unit 206, a detection unit 12 that detects an AR marker in the captured image acquired by the acquisition unit 11, and a plurality of detection units 12. When an AR marker is detected, the calculation unit 13 that calculates the distance between the plurality of AR markers and the character corresponding to the AR marker identification information of the AR marker detected by the detection unit 12 are controlled by the display unit 207. A display control unit 14 that superimposes and displays the captured image on the captured image. When the detection unit 12 detects a plurality of AR markers, a moving image in a mode corresponding to the distance calculated by the calculation unit 13 is added to the captured image. The function of the display control unit 14 for superimposed display is realized.

[motion]
An operation example of the information processing apparatus 10 will be described with reference to FIGS. First, the user places the AR marker on a desk or a table, and activates the imaging unit 206 of the imaging display device 20. Thus, imaging of the space including the AR marker is started. The captured image data captured by the imaging unit 206 is transmitted from the communication IF 205 to the information processing device 10 via the network 90.

  When the acquisition unit 11 of the information processing apparatus 10 acquires captured image data (FIG. 7, step S11), the detection unit 12 performs image recognition processing such as pattern matching on the captured image data to try to detect an AR marker, It is determined whether an AR marker is included in the captured image (step S12).

  If the captured image includes the AR marker (Step S12; YES), the display control unit 14 reads the character reproduction data corresponding to the AR marker identification information of the recognized AR marker from the auxiliary storage device 104, and reads the read character. The reproduction data is transmitted to the imaging display device 20 by specifying its display position (for example, a predetermined position on the AR marker), and the character is displayed (step S13).

  Then, when the user performs a predetermined reproduction operation on the imaging display device 20 (step S14; YES), the fact is notified to the information processing device 10. In response to this notification, the display control unit 14 of the information processing device 10 instructs the imaging display device 20 to reproduce the character reproduction data. As a result, in the imaging display device 20, reproduction control of the reproduction of the music and the operation of the character is started according to the character reproduction data (step S15).

  According to the embodiment described above, a moving image having a size corresponding to the distance between a plurality of AR markers can be displayed. For each AR marker, each character corresponding to the band member is superimposed and displayed as if a single piece of music is being played in each part at the same time, and a video recording the performance in real space is played by the character. Is superimposed and displayed in synchronization with. This makes it possible to provide a service that makes the user feel interesting.

[Modification]
The invention is not limited to the embodiments described above. The embodiment described above may be modified as follows. Further, two or more of the following modifications may be combined.
[Modification 1]
In the above-described embodiment, the display control unit 14 superimposes and displays a character performing a performance operation in the real space, and also displays a moving image containing a real player performing the performance operation corresponding to the performance operation of the character in the real space. It was superimposed. The display target is not limited to this example. For example, the display control unit 14 may superimpose a virtual object that performs some operation in the real space and superimpose a moving image that is linked to the operation of the virtual object in the real space. The virtual object does not need to perform any operation. In short, when a plurality of real objects are detected, the display control unit 14 may superimpose a moving image in a mode corresponding to the distance between the real objects on the captured image.

[Modification 2]
The display control unit 14 may superimpose and display a moving image in a mode according to the distance between the real objects. This mode is the display size of the moving image in the embodiment, but may be, for example, the content of the moving image according to the distance. For example, when the distance between the AR markers is equal to or less than the threshold, a moving image in which the performance of a specific member of the band is mainly recorded as a subject is displayed, and when the distance between the AR markers exceeds the threshold, the band is displayed. A moving image recorded from a shooting position where the performance scenery of all the members is overlooked may be displayed. That is, the display control unit 14 may superimpose and display a moving image having contents corresponding to the distance calculated by the calculation unit 13. In this case, in the information processing apparatus 10, the distance between the AR markers and the moving image reproduction data to be displayed are stored in association with each other, and the display control unit 14 corresponds to the distance calculated by the calculation unit 13. The moving image reproduction data is read out and superimposed and displayed.

[Modification 3]
In the embodiment, a virtual plane a having a direction corresponding to the direction of the line segment is provided on a line connecting the AR markers, and a moving image is superimposed on the virtual plane a. Instead, the display control unit 14 may superimpose and display a moving image on a virtual plane in a direction corresponding to the line of sight of the user viewing the captured image. The direction of the user's line of sight may be specified by the image capturing and displaying device 20 capturing the user's eyes by image analysis. Thereby, the direction of the virtual plane for displaying the moving image can be changed depending on the viewing direction of the user. Therefore, for example, the virtual plane directly facing the user (that is, the normal of the virtual plane and the viewing direction of the user are always It becomes possible to superimpose and display the moving image on (parallel).

[Modification 4]
When one of the two AR markers is moved and the distance between the AR markers changes, the display mode of the moving image may change accordingly, or the distance between the AR markers may change. Even if is changed, the moving image may be continuously displayed in the state before the movement. For example, when a predetermined operation is performed by a user while a moving image is displayed in a certain mode, the moving image is displayed even if one of the plurality of AR markers is moved after the operation. Leave as is. That is, when a predetermined operation is performed by the user, the display control unit 14 moves one of the plurality of real objects and changes the distance between the plurality of real objects. However, the display mode of the moving image may not be changed. This is convenient when the user wants to maintain the display mode of the moving image. Further, for example, when any of the AR markers is moved while a moving image is displayed in a certain mode, the display of the moving image may be deleted.

[Modification 5]
For example, if there are three or more members constituting a band playing music and three or more AR markers corresponding to these members, moving images having different contents may be displayed according to the combination of the AR markers. Good. For example, if AR markers M1, M2, and M3 corresponding to members A, B, and C of a band composed of three members, members A and B are mainly recorded as subjects in the case of a combination of AR markers M1 and M2. In the case of a combination of the AR markers M2 and M3, a moving image in which members B and C are mainly recorded as subjects is displayed, and in the case of a combination of the AR markers M1 and M3, the members A and C are mainly displayed. For example, a moving image recorded as a subject is displayed. That is, the display control unit 14 may superimpose and display a moving image in a mode according to the combination of the identification information of the plurality of real objects detected by the detection unit 12.

  In the display size table, different threshold values may be set according to the combination of AR markers. For example, in the case of the combination of the AR markers M1 and M2, the thresholds L1 and L2 are used, and in the case of the combination of the AR markers M2 and M3, the thresholds L3 and L4 different from the thresholds L1 and L2 are used. In the case of a combination, thresholds L5 and L6 different from thresholds L1 to L4 are used.

  The number of AR markers per combination may be three or more. In this case, the distance between any two AR markers among these AR markers may be used, or the distance between each AR marker may be calculated and the average value thereof may be used. In this case, the display position of the moving image may be a virtual plane on a line segment connecting any two AR markers among these AR markers, or the center of gravity of a figure connecting all the AR markers. It is also possible to provide a virtual plane at a position where, and use it as a display position.

[Modification 6]
The content of the moving image is not limited to the example of the embodiment. For example, an appearance when each character is viewed from a certain character position may be displayed as a moving image. Specifically, the display control unit 14 of the information processing device 10 specifies the positional relationship of each AR marker based on the position of the AR marker in the captured image, and, for example, determines the position relationship of one AR marker specified by the user. The position of another AR marker when viewed in a certain visual field range is specified. Then, the display control unit 14 generates, as a moving image, a state in which a character playing operation corresponding to another AR marker is performed in the visual field range viewed from the position of a certain AR marker, and causes the imaging display device 20 to display the moving image. . That is, the display control unit 14 may superimpose and display a moving image in a mode corresponding to the position of the virtual object corresponding to any one of the plurality of real objects detected by the detection unit 12.

[Modification 7]
The information processing apparatus 10 includes a user identification unit that identifies a user who is viewing the captured image by, for example, a method of recognizing a face image of the user or a method of inputting a user ID or biometric information. Alternatively, a moving image corresponding to the user identified by the user identification unit may be superimposed and displayed. For example, if the identification information of the member who likes the user among the members constituting the band is registered in the information processing device 10, the user who is identified by the user identification unit among the plurality of moving images and likes the user It is possible to select a moving image mainly including members and superimpose the selected moving image. That is, a moving image corresponding to the user's hobby and taste may be registered, and the moving image corresponding to the identified user may be superimposed.

[Modification 8]
The method for calculating the distance between the AR markers is not limited to the example of the embodiment. In addition, there may be no upper limit or lower limit of the display size of the moving image.

[Other modifications]
The block diagram used in the description of the above-described embodiment shows blocks in functional units. These functional blocks (components) are realized by an arbitrary combination of hardware and / or software. The means for implementing each functional block is not particularly limited. That is, each functional block may be realized by one device physically and / or logically coupled, or two or more devices physically and / or logically separated from each other directly and / or indirectly. (For example, wired and / or wireless), and may be realized by the plurality of devices. For example, at least a part of the functions of the information processing device 10 may be mounted on the imaging display device 20. When all of the functions of the information processing device 10 are mounted on the imaging display device 20, the imaging display device 20 functions as the information processing device according to the present invention.

  Each aspect / embodiment described in this specification includes LTE (Long Term Evolution), LTE-A (LTE-Advanced), SUPER 3G, IMT-Advanced, 4G, 5G, FRA (Future Radio Access), W-CDMA. (Registered trademark), GSM (registered trademark), CDMA2000, Ultra Mobile Broadband (UMB), IEEE 802.11 (Wi-Fi), IEEE 802.16 (WiMAX), IEEE 802.20, UWB (Ultra-WideBand), The present invention may be applied to a system using Bluetooth (registered trademark), another appropriate system, and / or a next-generation system extended based on the system.

The processing procedures, sequences, flowcharts, and the like of each aspect / embodiment described in this specification may be interchanged as long as there is no inconsistency. For example, the methods described herein present elements of various steps in a sample order, and are not limited to the specific order presented.
Each aspect / embodiment described in this specification may be used alone, may be used in combination, or may be switched as the execution goes on. In addition, the notification of the predetermined information (for example, the notification of “X”) is not limited to explicitly performed, and is performed implicitly (for example, not performing the notification of the predetermined information). Is also good.

  The terms "system" and "network" as used herein are used interchangeably.

  The information, parameters, and the like described in this specification may be represented by an absolute value, a relative value from a predetermined value, or another corresponding information. For example, the radio resource may be indicated by an index.

  The names used for the parameters described above are not limiting in any way. Further, the formulas and the like that use these parameters may differ from those explicitly disclosed herein. Since the various channels (eg, PUCCH, PDCCH, etc.) and information elements (eg, TPC, etc.) can be identified by any suitable name, the various names assigned to these various channels and information elements may differ in any way. However, it is not limited.

  The terms "determining" and "determining" as used herein may encompass a wide variety of operations. The “judgment” and “decision” are, for example, judgment (judging), calculation (computing), processing (processing), deriving (investigating), investing (looking up) (for example, a table). , Searching in a database or another data structure), ascertaining what is considered to be “determined”, “determined”, and the like. Also, “determining” and “determining” refer to receiving (eg, receiving information), transmitting (eg, transmitting information), input (input), output (output), access (Accessing) (for example, accessing data in the memory) may be regarded as “determined” or “determined”. In addition, “determination” and “decision” mean that resolving, selecting, selecting, establishing, establishing, comparing, and the like are regarded as “determination” and “determined”. May be included. In other words, “determination” and “determination” may include deeming that any operation is “determination” and “determined”.

  The present invention may be provided as an information processing method including steps of processing performed in the imaging display device 20 or the information processing device 10. Further, the present invention may be provided as a program executed in the imaging display device 20 or the information processing device 10. Such a program may be provided in a form recorded on a recording medium such as an optical disk, or may be provided in a form such as being downloaded to a computer via a network such as the Internet and installed and made available. It is possible.

  Software, instructions, etc. may be sent and received via transmission media. For example, if the software uses a wired technology such as coaxial cable, fiber optic cable, twisted pair and digital subscriber line (DSL) and / or a wireless technology such as infrared, wireless and microwave, the website, server, or other When transmitted from a remote source, these wired and / or wireless technologies are included within the definition of transmission medium.

  The information, signals, etc. described herein may be represented using any of a variety of different technologies. For example, data, instructions, commands, information, signals, bits, symbols, chips, etc., that can be referred to throughout the above description are not limited to voltages, currents, electromagnetic waves, magnetic or magnetic particles, optical or photons, or any of these. May be represented by a combination of

  The terms described in the present specification and / or the terms necessary for understanding the present description may be replaced with terms having the same or similar meaning. For example, channels and / or symbols may be signals. Also, the signal may be a message. Further, the component carrier (CC) may be called a carrier frequency, a cell, or the like.

  Any reference to elements using designations such as "first," "second," etc., as used herein, does not generally limit the quantity or order of those elements. These designations may be used herein as a convenient way to distinguish between two or more elements. Thus, reference to a first and second element does not mean that only two elements may be employed therein, or that the first element must somehow precede the second element.

  “Means” in the configuration of each device described above may be replaced with “unit”, “circuit”, “device”, or the like.

  As long as “including”, “comprising”, and variations thereof, are used herein or in the claims, these terms are inclusive as well as the term “comprising” It is intended to be relevant. Further, it is intended that the term "or", as used herein or in the claims, not be the exclusive OR.

  Throughout this disclosure, when articles are added by translation, e.g., a, an, and the in English, unless the context clearly indicates otherwise, It shall include a plurality.

  Although the present invention has been described in detail, it will be apparent to those skilled in the art that the present invention is not limited to the embodiments described in this specification. The present invention can be embodied as modified and changed aspects without departing from the spirit and scope of the present invention defined by the description of the claims. Therefore, the description in the present specification is for the purpose of illustrative explanation, and does not have any restrictive meaning to the present invention.

DESCRIPTION OF SYMBOLS 1 ... Information processing system, 10 ... Information processing apparatus, 11 ... Acquisition part, 12 ... Detection part, 13 ... Calculation part, 14 ... Display control part, 20 ... Image pickup display device, 90 ... Network, 101 ... CPU, 102 ... RAM 103, ROM, 104, auxiliary storage device, 105, communication IF, 201, CPU, 202, RAM, 203, ROM, 204, auxiliary storage device, 205, communication IF, 206, imaging unit, 207, display unit, 208 ... operation unit.

Claims (10)

An acquisition unit that acquires a captured image;
In the captured image obtained by the obtaining unit, a detecting unit that detects a real object on which identification information is written,
A calculating unit that calculates a distance between the plurality of real objects when the plurality of real objects is detected by the detection unit;
A display control unit that superimposes a virtual object corresponding to the identification information of the real object detected by the detection unit on the captured image, and when the detection unit detects a plurality of the real objects, A display control unit configured to superimpose and display a moving image in a mode corresponding to the distance calculated by the calculation unit on the captured image. The information processing apparatus according to claim 1, wherein the display control unit superimposes the virtual object performing the operation and superimposes the moving image in conjunction with the operation of the virtual object. The display control unit superimposes and displays the virtual object performing the performance operation, and superimposes and displays the moving image recording a player performing a performance operation corresponding to the performance operation of the virtual object. 2. The information processing apparatus according to item 2. The information processing device according to claim 1, wherein the display control unit superimposes and displays the moving image having a size corresponding to the distance calculated by the calculation unit. The information processing apparatus according to any one of claims 1 to 4, wherein the display control unit superimposes and displays the moving image having content corresponding to the distance calculated by the calculation unit. The said display control part superimposes and displays the said moving image on the virtual plane of the direction according to the direction of the said line segment on the line segment which connects several said real objects. 2. The information processing apparatus according to claim 1. The information processing according to any one of claims 1 to 5, wherein the display control unit superimposes and displays the moving image on a virtual plane oriented in a direction of a line of sight of a user who views the captured image. apparatus. When a predetermined operation is performed by a user who browses the captured image, one of the plurality of real objects is moved, and the display control unit moves between the plurality of real objects. The information processing apparatus according to any one of claims 1 to 7, wherein the display mode of the moving image is not changed even when the distance of the moving image changes. The said display control part superimposes and displays the said moving image of the aspect according to the combination of the identification information of the some said real object detected by the said detection part. The Claim 1 characterized by the above-mentioned. An information processing apparatus according to claim 1. The display control unit superimposes and displays the moving image in a mode according to a position of a virtual object corresponding to identification information of one of the plurality of real objects detected by the detection unit. Item 10. The information processing apparatus according to any one of Items 1 to 9.

Images