JP5786411B2 - Information processing system, search device, and program - Google Patents

Description

  The present invention relates to an information processing system, a search device, and a program. More specifically, a user can easily view a desired tag display image from a plurality of tag display images generated based on a plurality of tag information. The present invention relates to an information processing system, a search device, and a program.

  In recent years, in the field of information processing, a technology related to so-called augmented reality (AR), in which video in a real space is computer-processed and various information is added to the real space video or an image is superimposed. Research and development is actively conducted. Those using this augmented reality, especially recently, in mobile phones with camera functions, etc., those created in virtual reality space to images of subjects such as facilities and people in the real space captured by the camera. A function of superimposing an object display image such as a character or a graphic representing information on a facility or the like and displaying it on a monitor is beginning to be adopted in various modes such as a facility guide or a travel guide.

  For example, Patent Document 1 measures the position and imaging orientation of a camera, and acquires predetermined information from information recorded (registered) in a recording medium in advance based on the measured camera location and imaging orientation. A technology related to augmented reality is disclosed in which an object display image generated based on the acquired information is displayed superimposed on a captured image.

  By the way, in the technology related to augmented reality, a large amount of information for generating various object display images to be superimposed on the captured image in the real space is acquired, and a large number of object display images are complicatedly superimposed on the captured image. There are many things. As a result, there is a problem that the user cannot easily visually recognize the object display image originally desired by the user.

In order to solve this problem, for example, in Patent Document 2, information to be acquired is limited from the position of the device and the imaging direction, acquisition of information included in the limited range, and generation of an object display image based on the information A technique for performing the above is disclosed.
Patent Document 3 describes a technique for specifying an imaging field of view from an apparatus position, an imaging posture, an imaging direction, and a viewing angle (viewing angle), and acquiring information included in the specified range. ing.

Japanese Patent No. 3700021 JP 2010-231741 A JP 2010-239568 A

However, in the technique described in Patent Document 2, the information acquisition range in the virtual reality space and the real space range that the user is actually capturing do not match, so the object display image is the captured image of the subject in the real space. It is presumed that there arises a problem that it is not possible to superimpose and display in association with.
In the technique described in Patent Document 3, all information included in the viewing angle (view angle) of the camera included in the apparatus is displayed. Here, when the user changes the viewing angle (viewing angle) by changing the zoom magnification of the camera, the display size of the object image is also changed according to the change of the viewing angle (viewing angle). As a result, there is a problem that the object display image is displayed in an inappropriate size or the object display images are overlapped with each other, and the object display image located far from the user is difficult to visually recognize on the monitor. It is inferred.

  Accordingly, the present invention provides an information processing apparatus capable of displaying an object display image generated based on information acquired in a virtual reality space in an appropriately associated manner and superimposed on a subject image in a captured image. With the goal.

  Further, according to the present invention, when an object display image generated based on information acquired in a virtual reality space is displayed in a superimposed manner on a subject image in a captured image, the user can display a plurality of superimposed objects. An object of the present invention is to provide an information processing apparatus that enables a desired object display image to be easily visually recognized from among display images.

In order to achieve the above object, the present invention is an information processing system in which an information processing device and a search device communicate with each other via a network,
The information processing apparatus includes: an imaging unit that captures a subject and acquires a captured image; an imaging magnification changing unit that changes a magnification of imaging performed by the imaging unit; a position detection unit that detects a position of the information processing apparatus; A direction detection unit that detects an imaging direction of the imaging unit, an angle of view calculation unit that calculates an angle of view of imaging by the imaging unit, an imaging magnification changed by the imaging magnification change unit, and a position detected by the position detection unit An information transmission unit that transmits information including the imaging direction detected by the direction detection unit and the angle of view calculated by the angle of view calculation unit to the search device; and an object that generates an object display image using object information A display image generation unit; an object information reception unit that receives object information used by the object display image generation unit from the search device; and an image acquired by the imaging unit. And a display unit for displaying an object display image by the image and the object display image generating unit has generated synthesized and,
The search device includes: an information reception unit that receives the information transmitted by the information transmission unit included in the information processing device; a storage unit that stores a plurality of object information; and the plurality of object information stored in the storage unit Is searched based on the information received by the information receiving unit, the search unit for specifying the object information used by the object display image generation unit of the information processing device from the storage unit, and the search unit specified Object information prohibition range storage for storing a plurality of object acquisition prohibition ranges in which object information transmission units that transmit the object information to an object information reception unit included in the information processing apparatus and object information that prohibits transmission by the object information transmission unit exists Of the information received by the information receiver and the position detector A predetermined object acquisition prohibition range is specified from the output position and the imaging direction detected by the direction detection unit, and the search unit displays object information included in the specified object acquisition prohibition range in the object of the information processing apparatus. There is provided an information processing system comprising an object acquisition prohibition range specifying unit excluded from the object information used by a display image generation unit .

The present invention also provides:
Information processing comprising: an imaging unit that images a subject and acquires a captured image; an imaging magnification changing unit that changes a magnification of imaging by the imaging unit; and a display unit that combines and displays the captured image and an object display image A search device that communicates with a device via a network,
An information receiving unit that receives information about the imaging magnification changed by the imaging magnification changing unit, the position of the information processing device, the imaging direction of the imaging unit, and the angle of view of imaging by the imaging unit from the information processing device; ,
A storage unit storing a plurality of object information for generating the object display image;
A search unit for searching the plurality of object information stored in the storage unit based on the information received by the information receiving unit, and specifying predetermined object information from the storage unit;
Object acquisition prohibition range for storing a plurality of object acquisition prohibition ranges in which object information transmission unit for transmitting predetermined object information specified by the search unit to the information processing apparatus and object information for prohibiting transmission by the object information transmission unit exist A storage unit;
A predetermined object acquisition prohibition range is specified from the position and imaging direction in the information received by the information reception unit, and the search unit transmits object information included in the specified object acquisition prohibition range to the information processing apparatus. Another object of the present invention is to provide a search device including an object acquisition prohibition range specifying unit that is excluded from the object information.

Furthermore, the present invention is a program for executing processing in which an information processing device and a search device communicate with each other via a network,
  The information processing apparatus includes:
  An imaging process for imaging a subject and acquiring a captured image;
  An imaging magnification changing process for changing an imaging magnification by the imaging process;
  Position detection processing for detecting the position of the information processing apparatus;
  Direction detection processing for detecting the imaging direction of the imaging processing;
  An angle-of-view calculation process for calculating an angle of view of imaging by the imaging process;
  The search device includes information including the imaging magnification changed by the imaging magnification changing process, the position detected by the position detection process, the imaging direction detected by the direction detection process, and the angle of view calculated by the angle of view calculation process. Information transmission processing to be transmitted to
  Object display image generation processing for generating an object display image using object information;
  An object information reception process for receiving object information used by the object display image generation process from the search device;
  Performing a display process of combining and displaying the captured image acquired by the imaging process and the object display image generated by the object display image generation process;
  The search device includes:
  An information reception process for receiving the information transmitted by the information transmission process executed by the information processing apparatus;
  A storage process for storing a plurality of object information;
  The plurality of object information stored by the storage process is searched based on the information received by the information reception process, and the object information used by the object display image generation process of the information processing apparatus is identified from the storage process Search process to
  An object information transmission process for transmitting the object information specified by the search process to an object information reception process executed by the information processing apparatus;
  Object acquisition prohibited range storage processing for storing a plurality of object acquisition prohibited ranges in which object information that prohibits transmission by the object information transmission processing exists;
A predetermined object acquisition prohibition range is specified from the position detected by the position detection process in the information received by the information reception process and the imaging direction detected by the direction detection process, and the search process is the acquisition of the specified object An object of the present invention is to provide a program that executes an object acquisition prohibited range specifying process that excludes object information included in a prohibited range from the object information used by the object display image generation process of the information processing apparatus. .

  According to the information processing apparatus according to the present invention, an object display image generated based on information acquired in the virtual reality space can be superimposed and displayed on a subject image in a captured image appropriately.

  Further, according to the information processing apparatus of the present invention, when the object display image generated based on the information acquired in the virtual reality space is displayed in a superimposed manner on the subject image in the captured image, the user can The desired object display image can be easily visually recognized from the plurality of superimposed object display images.

It is a figure explaining the problem of the conventional information processing apparatus. It is a figure for demonstrating another problem of the conventional information processing apparatus. 1 is a block diagram illustrating a configuration of an information processing system 1 according to an embodiment of the present invention. 5 is a flowchart for explaining tag information acquisition and tag display image generation / display processing by the information processing system 1; It is a figure for demonstrating the standard and the reason for calculating the tag information acquisition shortest distance and the tag information acquisition longest distance. It is a figure for demonstrating the calculation method of tag information acquisition shortest distance and tag information acquisition longest distance. 5 is a flowchart for explaining tag information search processing by a search server 200; It is a figure for demonstrating the example of the identification process of the tag information which prohibits the acquisition by the tag acquisition prohibition range specific | specification part 205. FIG. It is a figure for demonstrating the detail of tag information. It is a figure for demonstrating the process which selects the tag image used for the tag display image displayed from the three types of tag images contained in tag information. It is a figure for demonstrating the process which calculates the display position on the display screen of the display part 110 of a tag display image. It is a figure for demonstrating the example at the time of displaying a tag display image on the display screen of the display part 110 by making the information processing apparatus 100 into a horizontal attitude | position. It is a figure for demonstrating another example at the time of displaying a tag display image on the display screen of the display part 110 by making the information processing apparatus 100 into a horizontal attitude | position. It is a block diagram which shows the structure of the information processing apparatus 300 which concerns on embodiment of this invention.

  Hereinafter, preferred embodiments of an information processing apparatus according to the present invention will be described with reference to the drawings. The dimensions, specific numerical values, and the like shown in the embodiments are merely examples for facilitating understanding of the invention, and do not limit the present invention unless otherwise specified. In the present specification and drawings, elements having substantially the same function and configuration are denoted by the same reference numerals, and redundant description is omitted, and elements not directly related to the present invention are not illustrated. To do.

First, with reference to FIG. 1 and FIG. 2, the details of the problems of a conventional information processing apparatus having a function relating to augmented reality will be described.
In FIG. 1, for example, as in the technique described in Patent Document 2, the range for acquiring object information arranged in the virtual reality space is set separately from the imaging range of the camera mounted on the information processing apparatus. It is a figure for demonstrating the problem in a case. Here, the acquisition range of the object information is set based on the position of the information processing apparatus and the imaging direction of the mounted camera.

  In FIG. 1, the information processing apparatus 01 has a monitor (for example, about 7 cm in the horizontal direction and about 5.25 cm in the vertical direction) having different lengths in the horizontal and vertical sides mounted on one surface of the housing. It is assumed that an imaging unit such as a lens is mounted, for example, on a surface opposite to the surface on which the monitor is mounted so as to be substantially parallel to the monitor.

  In the following, for ease of explanation and understanding, when a horizontal side of the monitor is turned down, a captured image displayed on the monitor (that is, a captured image having a horizontal to vertical ratio of 16: 9, for example) is captured. The attitude of the information processing apparatus is “horizontal attitude”, and the angle of view of the camera included in the information processing apparatus 01 in the horizontal attitude is “horizontal angle of view”. In addition, when the vertical side of the monitor is turned down, the attitude of the information processing apparatus that captures a captured image (that is, a captured image having a horizontal to vertical ratio of 9:16, for example) displayed on the monitor is set to “vertical”. The angle of view of the camera included in the information processing apparatus 01 in the vertical posture is defined as “vertical angle of view”. FIGS. 1A and 1B illustrate problems when the information processing apparatus 01 performs image capturing in a horizontal posture, and FIG. 1C illustrates problems when image capturing is performed in a vertical posture.

  The upper diagram in FIG. 1A shows the subject A01 (a), the subject B01 (a), the subject C01 (a), the information processing device 01, and the range VS01 (a ) (Range indicated by hatching in the drawing) and object information TA01 (a) corresponding to the subject A01 (a) and object information TB01 corresponding to the subject B01 (a) arranged in the range VS01 (a) It is the conceptual diagram which looked at the top view of (a), and the lower figure is a figure which shows display image MG01 (a) of the monitor with which the information processing apparatus in that case is provided.

  The information processing apparatus 01 captures a subject included in the imaging range defined by the horizontal angle of view AV01 (a) as a captured image. Since the subject B01 (a) and the subject C01 (a) are included in this imaging range, the image BG01 (a) of the subject B01 (a) and the image CG01 of the subject C01 (a) are displayed on the display screen MG01 (a). (A) is displayed. On the other hand, since the subject A01 (a) is outside this imaging range, the image corresponding to the subject A01 (a) is not displayed on the display screen MG01 (a). On the other hand, regardless of the imaging range, the object VS01 (a) corresponding to the subject A01 (a) and the object corresponding to the subject B01 (a) are included in the range VS01 (a) defined only by the imaging direction. Information TB01 (a) is included. Accordingly, object display images TGA01 (a) and TGB01 (a) corresponding to the object information A01 (a) are displayed on the display screen MG01 (a).

  In this case, there arises a problem that the user cannot grasp through the monitor which object the object display image TGA (a) displayed on the display screen MG01 (a) is associated with.

  On the other hand, in FIG. 1A, the horizontal imaging width of the imaging range of the information processing device V01 matches the horizontal width of the range VS01 (a) for acquiring object information. In the screen MG01 (a), the object display image TGB01 (a) corresponding to the subject B01 (a) is superimposed on the image BG01 (a) corresponding to the subject B01 (a), and the two are displayed in association with each other. The Therefore, the user can obtain information regarding the subject B01 (a) through the display screen MG01 (a).

However, when the horizontal imaging width of the imaging range does not match the horizontal width of the virtual space object information acquisition range, another problem occurs. This problem will be described in detail with reference to FIG.
The upper diagram in FIG. 1B shows the subject B01 (b), the subject C01 (b), the information processing device 01 that captures them, and the subject B01 that exists in the range VS01 (b) for acquiring object information. It is the conceptual diagram which looked at object information TB01 (b) corresponding to (b) from the top, and the lower figure is a figure which shows display image MG01 (b) of the monitor with which the information processing apparatus 01 is provided at that time.

  The information processing device 01 captures a subject included in the imaging range defined by the horizontal angle of view AV01 (b) as a captured image. At this time, since the horizontal imaging width of the imaging range does not match the horizontal width of the range VS01 (b) in which the object information is acquired, the subject B01 (b) is displayed on the display screen MG01 (b). Although the subject image BG01 (b) corresponding to b) is displayed slightly from the center, the object display image TGB01 (b) corresponding to the subject B01 (b) is displayed at the edge of the screen. The display position is shifted.

  In this case, there arises a problem that the user cannot recognize through the display screen MG01 (b) which object in the real space the object display image TGB01 (b) corresponds to.

In addition, when the imaging posture of the information processing apparatus is not taken into consideration, a new problem occurs. This problem will be described in detail with reference to FIG.
The upper diagram in FIG. 1C shows a subject B01 (c), a subject C01 (c), an information processing device 01 equipped with a camera for imaging them, and a range VS01 (c) for acquiring object information. It is the conceptual diagram which looked at the object information TB01 (c) corresponding to the to-be-photographed object B01 (c) from the top, and the lower figure is a figure which shows display image MG01 (c) of the monitor with which the information processing apparatus 01 at that time is equipped. is there. Here, unlike the case of FIG. 1A or 1B, the information processing apparatus 01 is assumed to perform imaging in a vertical posture.

  The information processing apparatus 01 captures a subject included in the imaging range defined by the vertical angle of view AV01 (c) as a captured image. At this time, the subject B01 (c) is out of this area, and therefore the image corresponding to the subject B01 (c) is not displayed on the display screen MG01 (c).

  On the other hand, the information processing apparatus 01 sets the range of the virtual space VS01 regardless of the imaging posture. Therefore, it is not distinguished whether the captured image is acquired with a horizontal angle of view or a vertical angle of view. Therefore, the object information TB01 (c) is acquired, and the object display image TGB01 (c) is displayed on the display screen MG01 (c). In addition, the object display image TGB01 (c) is displayed at a display position corresponding to a predetermined imaging posture without considering the imaging posture of the information processing apparatus. Therefore, for example, when the user rotates the information processing apparatus 90 degrees clockwise from the horizontal posture in FIG. 1B to the vertical posture in FIG. 1C, TGB01 (c) is the subject image CG01 ( There is also a possibility that the information related to the subject B is displayed as if it was a captured image of the subject C, superimposed on c).

  Therefore, in the above-described conventional information processing apparatus 01, even if the imaging posture is changed, the user corresponds to which subject in the real space the object display image TGB01 (c) corresponds to the case of FIG. There is a problem that it cannot be recognized through the display screen MG01 (c).

  As described above, in an information processing apparatus equipped with conventional technology related to virtual reality, the range for acquiring object information arranged in the virtual reality space is different from the imaging range of the camera mounted on the information processing apparatus. Since it is set and the imaging posture of the information processing apparatus is not taken into consideration, the object display image cannot be displayed in association with the captured image of the subject in the real space appropriately.

  Next, FIG. 2 shows an object display image display screen based on the object information acquisition range, which is the horizontal field angle of imaging, for example, as in the technique described in Patent Document 3 above. A problem when the display position is set in the screen will be described.

  In FIG. 2, the information processing apparatus 02 has a monitor (for example, about 7 cm in the horizontal direction and about 5.25 cm in the vertical direction) on which one side of the horizontal direction and the length of the vertical side are different is mounted on one surface of the housing. For example, an imaging unit such as a lens is mounted on a surface opposite to the surface on which the monitor is mounted so as to be substantially parallel to the monitor, as in the conventional information processing apparatus 01 described above.

  The upper diagram in FIG. 2A is a conceptual diagram of a top view of the relationship between the information processing device 02 including a camera and the object information TA02, TB02, and TC02 arranged in the virtual reality space. FIG. 11 is a diagram showing a monitor display screen MG02 (a) included in the information processing apparatus 02 at that time.

  The information processing apparatus 02 acquires object information TA02, TB02, and TC02 that are within the range defined by the horizontal field angle AV02 (a) of imaging from the virtual reality space. Then, object display images TGA02 (a), TGB02 (a), and TGC02 (a) generated based on each object information are displayed in a display screen MG02 (a) displayed on the monitor.

  Here, the conventional information processing apparatus 02 sets the display position of each object display image within the display screen based on the object information acquisition range that matches the imaging range defined by the horizontal angle of view AV02 (a). Yes. Accordingly, as with each subject image included in the imaging range, the display width in the horizontal display screen of each object display image included in the object information acquisition range is small in the vicinity of the information processing apparatus 02, and becomes farther away. growing. Therefore, even if the images are arranged in the virtual space as the same size object display image, the size in the display screen changes according to the distance from the information processing apparatus 02. In FIG. 2A, the object display image TGA02 (a) based on the object information TA02 located closest to the information processing device 02 is the largest and the object display image TCG02 (a) based on the object information TC02 located farthest. Is displayed in the smallest size.

  Therefore, the object display image TAG02 (a) is easily visible to the user, but the object display images TBG02 (a) and TCG02 (a) displayed smaller than the object display image TAG02 (a) are difficult to visually recognize.

  Therefore, a user who desires information related to the object display image TCG02 (a) changes the zoom magnification, that is, changes the horizontal angle of view, and the object display image TAG02 in which the object display image TCG02 (a) is currently displayed is displayed. The case of displaying the same size as (a) will be described with reference to FIG.

  As shown in FIG. 2B, the zoom magnification is changed, that is, the horizontal angle of view is changed from the horizontal angle of view AV02 (a) to a smaller horizontal angle of view AV02 (b), and an image with a large magnification is displayed. As a result of the display on the screen MG02 (b), the object display image TGC02 (b) related to the object information TC02 is displayed large enough to be visually recognized by the user, but at the same time, the object display image TGA02 (b) related to the object information TA02 ) Will be displayed larger than TGA02 (a). The large display of the object display image TGA02 (b) for which information is not desired is disadvantageous to the user because the display screen becomes complicated.

  In addition, when the display screen is generated so that the near object display image is in front and the far object display image is in the back as viewed from the information processing apparatus 02 as in the above example, another object display image is displayed regardless of the zoom magnification. As a result, an object display image that cannot be visually recognized by the user is generated. In FIG. 2, the object display image TGB02 (a) or TGB02 (b) related to the object information TB02 is hidden by the superimposed object display image TGA02 (a) or TGA02 (b), and the user visually recognizes them. Can not do it.

  As described above with reference to FIG. 2, in the information processing apparatus having the conventional technology related to virtual reality, the horizontal field angle of imaging is set as the object information acquisition range, and the object information acquisition range is included in the object information acquisition range. Since the display position of the object display image in the display screen is set based on this, there is a problem that the object display image is displayed so small that it cannot be visually recognized by the user, or conversely, it is displayed excessively large. Also, when an object display image that appears as if it is in the distance overlaps with an object display image that appears as if it is nearby, the object that appears as if it is nearby There is a problem that it is hidden in the display image and cannot be seen completely.

<First embodiment: Information processing system>
Therefore, an information processing system according to the first embodiment of the present invention for solving the above-described problem will be described below.
In the following description, it is as if the virtual object (subject) related to the object information exists in the virtual reality space linked to the real space from the information such as the position, shape and size included in the object information. The assumed virtual object is called an “object”. In the following description of each embodiment, for ease of explanation and understanding, the information processing apparatus does not have a vertical inclination, the object does not have a vertical inclination, and the information processing apparatus includes Description will be made assuming that the center of the object is located at the same height as the center (optical axis) of the lens of the imaging unit.

FIG. 3 is a block diagram showing a configuration of the information processing system 1 according to the embodiment of the present invention. The information processing system 1 according to the present embodiment largely includes an information processing apparatus 100 that acquires object information and displays an object display image, and a search server 200 that searches for object information related to the displayed object display image. The information processing apparatus 100 and the search server 200 are connected by using means such as wireless communication.
In the following description, the object display image is referred to as “tag display image”, the object information is referred to as “tag information”, and the object is referred to as “tag”.

The information processing apparatus 100 can capture a moving image and a still image through a built-in imaging unit.
The present invention can be applied to other electronic devices capable of imaging such as a digital still camera and a digital video camera, a mobile phone, a PHS (Personal Handyphone System), and a PDA (Personal Digital Assistant).

<Configuration of Information Processing Device 100>
First, the configuration of the information processing apparatus 100 will be described in detail below.
The imaging unit 101 functions as an imaging unit that obtains a captured image of a subject by photoelectrically converting imaging light entering a lens system (not shown) by an imaging element (not shown) and converting it to a video signal. It is a member that performs an optical function such as focusing the lens system on the subject.

  The image enlarging unit 102 optically zooms a lens system (not shown) with a lens driving member such as a zoom actuator (not shown) or electronically zooms a captured image held by an image sensor (not shown). In other words, it is a member that functions as a captured image magnification changing unit that enlarges a captured image captured by the imaging unit 101.

  The posture detection unit 103 is a member that detects the posture of the information processing apparatus 100, and includes a method of detecting a direction with respect to gravity using an element (acceleration sensor) that detects gravitational acceleration, and an acceleration sensor (gyroscope). By using a method for obtaining the rotation angle by integrating the information, information on the attitude of the information processing apparatus 100, for example, the inclination of the information processing apparatus 100 in the horizontal direction from the horizontal attitude is detected. Information on the posture detected by the posture detection unit 103 is used for calculation of a horizontal angle of view by an angle-of-view calculation unit 106 described later, and a captured image of a tag display image generated by a tag display image generation unit 108 described later. It is used to calculate the position of superimposing on

  The position detection unit 104 is a member that detects the current position of the information processing apparatus 100 by using a GPS satellite or the like. The direction detection unit 105 is a member that detects in which direction the information processing apparatus 100 is directed by using an electronic compass or the like.

  The angle-of-view calculation unit 106 is a member that calculates the horizontal angle of view of the imaging unit 101 based on the zoom magnification that the image enlargement unit 102 enlarges and information on the posture of the information processing apparatus 100 that the posture detection unit 103 detects. It is. The view angle calculation unit 106 calculates the horizontal view angle and the vertical view angle of the image pickup unit 101 of the information processing apparatus 100 based on the zoom magnification that the image enlargement unit 102 enlarges. For example, when the posture of the information processing apparatus 100 detected by the posture detection unit 103 is less than 45 degrees with respect to the horizontal posture, the horizontal angle of view is set as the horizontal angle of view, and when the angle is 45 degrees or more with respect to the horizontal posture. Let the vertical angle of view be the horizontal angle of view.

  The tag information acquisition distance calculation unit 107 acquires tag information acquisition shortest distance, which is the shortest distance for acquiring tag information based on the zoom magnification at which the image enlargement unit 102 enlarges the captured image, and tag information. It is a member that performs a process of calculating the tag information acquisition longest distance that is the longest distance.

  The tag display image generation unit 108 includes tag information searched by a tag search unit 202 of the search server 200 described later, an attitude of the information processing apparatus 100 detected by the attitude detection unit 103, and an information processing apparatus detected by the position detection unit 104. This is a member that generates a tag display image based on the current position of 100. The tag display image generation unit 108 includes information about the position of the tag in the virtual space, the horizontal and vertical sizes of the tag, the arrangement direction of the tag in the virtual space, and the shape of the tag, which are included in the tag information The display position and display shape of the tag display image on the display unit 110 are calculated from the information regarding the attitude of the information processing apparatus 100 detected by the detection unit 103.

  Further, the tag display image generation unit 108 calculates an angle formed by a straight line connecting the position of the information processing apparatus 100 detected by the position detection unit 104 and the position of the tag in the virtual space and the arrangement direction of the tag. Based on the relationship between the angle and the display direction of the tag, a tag image to be used for display is selected from the three types of tag images including the front surface tag image, the back surface tag image, and the side surface tag image included in the tag information.

  The image synthesis unit 109 synthesizes and outputs the tag display image generated by the tag display image generation unit 108 and the captured image captured by the imaging unit 101, and the display unit 110 receives the output and outputs the image synthesis unit 109. Displays the synthesized image.

  The operation unit 111 is configured by an operation button (not shown), a touch panel provided on the display unit 110, and the like, and is a member that receives input of various operations related to setting of a zoom magnification, imaging, and screen display to the image enlargement unit 102.

  The communication unit 112 includes a horizontal angle of view calculated by the angle of view calculation unit 106, a position of the information processing device 100 detected by the position detection unit, a direction of the information processing device 100 detected by the direction detection unit 105, and a tag. The tag information acquisition shortest distance and the tag information acquisition longest distance calculated by the information acquisition distance calculation unit 107 are transmitted to the communication unit 203 of the search server 200 described later, or the tag search unit 202 of the search server 200 searches. It is a member that receives tag information from the communication unit 203. As described above, the communication unit 112 and the communication unit 203 can transmit and receive a plurality of pieces of information including the above by communicating using wireless means.

  The control unit 113 comprehensively controls various functions such as imaging and display of the information processing apparatus 100. For example, optical zoom magnification and digital zoom magnification corresponding to the zoom magnification setting value read from the operation unit 111 are set in the image enlargement unit 102, and optical control such as focus adjustment and exposure adjustment for the imaging unit 101 is performed. I do. Alternatively, as control related to display on the display unit 110, for example, display / non-display control, control regarding backlight brightness, power supply control of the information processing apparatus 100, and the like are performed. In addition, the above-described various members are controlled in an integrated manner.

  The ROM 114 stores a program for the operation of the control unit 113. The RAM 115 is used as a temporary storage area for various data, or as a work area for operations related to various programs.

<Configuration of Search Server 200>
Next, the configuration of the search server 200 according to the information processing system 1 will be described in detail below.
The tag storage unit 201 includes, for each tag, three types of tag images for the front surface, the back surface, and the side surface, the position of each tag in the virtual space, the arrangement direction of each tag in the virtual space, and the vertical direction of each tag. Tag information including the size, shape, and display direction of the direction is stored.

  The tag search unit 202 obtains tag information from the position of the information processing device 100, the direction of the information processing device 100, the angle of view in the horizontal direction, the shortest tag information acquisition distance, and the longest tag information acquisition distance obtained from the communication unit 203 described later. This is a part for calculating the acquisition range and searching for tag information included in the tag information acquisition range among the tag information stored in the tag storage unit 201. In addition, when the tag acquisition prohibition range specified by the tag acquisition prohibition range specifying unit 205 described later is included in the tag information acquisition range, the tag search unit 202 is included in a range obtained by removing the tag acquisition prohibition range from the tag information acquisition range. Search tag information.

  The communication unit 203 transmits the tag information searched by the tag search unit 202 to the communication unit 112, the horizontal angle of view calculated by the angle of view calculation unit 106, and the information processing apparatus detected by the position detection unit 104. In the part that receives the position of 100, the direction of the information processing apparatus 100 detected by the direction detection unit 105, the tag information acquisition shortest distance and the tag information acquisition longest distance calculated by the tag information acquisition distance calculation unit 107 from the communication unit 112. is there. As described above, the communication unit 203 and the communication unit 112 can transmit and receive a plurality of pieces of information including the above by communicating using wireless means.

  The tag acquisition prohibition range storage unit 204 includes a range to which tag information acquisition prohibition is added (hereinafter referred to as “tag acquisition prohibition setting range”) and a tag information acquisition prohibition range (hereinafter “tag”). This is a member that stores the “acquisition prohibited range” in association with each other.

  The tag acquisition prohibition range specifying unit 205 specifies the tag acquisition prohibition setting range including the position of the information processing apparatus 100 received through the communication unit 203 from the tag acquisition prohibition range storage unit 204, and specifies tag information for which acquisition is prohibited. Part.

  The control unit 206 performs overall transmission / reception of various information via the communication unit 203, control of the above-described various members, control of power supply to the various members, and the like.

  The ROM 207 is a member that stores a program for the operation of the control unit 206, and the RAM 208 is used as a temporary storage area for various data, or as a work area at the time of calculation related to various programs.

<Acquisition of tag information and generation / display processing of tag display image by information processing apparatus 100>
Next, tag information acquisition and tag display image generation / display processing by the information processing system 1 according to the present embodiment will be described with reference to a flowchart shown in FIG.

  First, the position detection unit 104 acquires the current position of the information processing apparatus 100 by using a GPS satellite or the like (step S101). Next, the direction detection unit 105 detects the imaging direction of the imaging unit 101 (step S102).

  Subsequently, the view angle calculation unit 106 calculates the horizontal view angle of the imaging unit 101 (step S103). Similar to other general imaging devices, in the information processing device 100, the horizontal angle of view and the vertical angle of view of the imaging unit 101 change according to the change in zoom magnification by optical zoom or digital zoom. Further, as described above, the horizontal angle of view when imaged in a horizontal posture and the horizontal angle of view when imaged in a vertical posture are different. Accordingly, the angle-of-view calculation unit 106 calculates the angle of view in the horizontal direction from the zoom magnification at which the image enlargement unit 102 enlarges the captured image and the attitude of the information processing apparatus 100 detected by the attitude detection unit 103.

  Then, the posture detection unit 103 detects the posture of the information processing apparatus 100 (step S104).

  The RAM 115 stores information on the current position, imaging direction, horizontal angle of view, and orientation of the information processing apparatus 100 acquired in the above steps (step S105).

  Next, the control unit 113 determines whether or not the tag display image generation / display process is the first one (step S106). If the control unit 113 determines that the process is not the first time (NO in step S106), the control unit 113 stores the current position, imaging direction, and horizontal direction of the information processing apparatus 100 stored in the RAM 115 in step S105. Information on the angle of view and orientation is read (step S107).

  Next, the control unit 113 stores and reads out the current position of the information processing apparatus 100 read in step S107, the imaging direction, the angle of view and orientation in the horizontal direction, and the previous generation and display processing of the tag display image. The position of the processing device 100, the imaging direction, the horizontal angle of view, and the posture are individually compared to determine whether or not these values have changed (step S108). Specifically, each value has a predetermined value. A range is provided, and when the value exceeding the predetermined range matches the value stored in step S107, the control unit 113 determines that a change has occurred.

  When none of the current position, imaging direction, horizontal angle of view, and orientation of the information processing apparatus 100 changes (NO in step S108), the control unit 113 determines whether a predetermined time has elapsed (step S109). ). If the predetermined time has not elapsed (NO in step S109), step S109 is repeated. If the predetermined time has elapsed (YES in step S109), the process returns to step S101 and the above-described processing is repeated.

  When the control unit 113 determines in step S106 that the tag display image generation / display process is the first one (YES in step S106), or in step S108, the control unit 113 captures the current position of the information processing apparatus 100 and the imaging. When it is determined that any of the direction, the horizontal angle of view, and the posture has changed (YES in step S108), the tag information acquisition distance calculation unit 107 calculates the tag information acquisition shortest distance and the tag information acquisition longest distance. (Step S110).

  Since the information processing apparatus 100 and the search server 200 are connected by wireless communication as described above, it is better not to perform wireless communication as much as possible in consideration of power consumption of the information processing apparatus 1. Therefore, by performing the steps S106 to S109 described above, the current position, the imaging direction, the horizontal angle of view, and the posture of the information processing apparatus 100 are set at predetermined time intervals, except for the initial display processing of the tag display image. Only when the values are acquired and the values have changed, the tag information search process and the tag display image update process can be performed.

<Tag information acquisition range>
The tag information acquisition distance calculation unit 107 is suitable for the display unit 110 as a tag display image having a standard size in the horizontal and vertical directions of the tag included in the tag information in accordance with an instruction from the control unit 113. The tag information acquisition shortest distance and the tag information acquisition longest distance are calculated so that they can be superimposed and displayed in size (step S110). In describing this calculation process, the tag information acquisition shortest distance and the standard for calculating the tag information acquisition longest distance and the reason for the calculation will be described below with a part of the tag information acquisition range calculated by the search server 200. Explained. The tag information acquisition range by the search server 200 will be described in detail later.

FIG. 5 is a conceptual diagram of a top view of the relationship between the information processing apparatus 100 and the tags TA05, TB05, and TC05 disposed in the virtual reality space, and the lower part is a display unit included in the information processing apparatus 100 at that time. 10 is a diagram showing a display screen MG05 of 110. FIG.
Here, the tag TB05 has a standard size in the horizontal and vertical directions, and the tag display image TGB05 is appropriate for the display screen MG05 displayed on the display unit 110 based on the tag TB05. Displayed in size. The tag information acquisition distance calculation unit 107 calculates the tag information acquisition shortest distance and the tag information acquisition longest distance on the basis of a tag in a state like the tag TB05. That is, the tag information acquisition distance calculation unit 107 is based on the distance Lb on the optical axis from the current position of the information processing apparatus 100 acquired by the position detection unit 104 to the position in the virtual space of the tag TB05. The distance short by the distance Lc is calculated as the tag information acquisition shortest distance, and the distance long by the distance Lc is calculated as the tag information acquisition longest distance.

  Then, an arc whose radius is the shortest tag information acquisition distance centered on the position of the information processing apparatus 100, an arc whose radius is the longest tag information acquisition distance centered on the position of the information processing apparatus 100, and the angle of view calculation unit 106 A range surrounded by the horizontal angle of view calculated by (a hatched range in FIG. 5) is a tag information acquisition range in the information processing apparatus 100.

  As described above, the tag information acquisition shortest distance and the tag information acquisition longest distance based on the position (hereinafter referred to as “reference position”) of a tag having a standard horizontal and vertical size in the virtual space. By calculating and setting the tag information acquisition range based on these, the tag display image related to each tag information included in the tag information acquisition range is displayed on the display unit 110 in a generally appropriate size. Become. In the example of FIG. 5, since the tags TB05 and TC05 are included in the tag information acquisition range, the tag display images TGB05 and TGC05 are displayed in an appropriate size on the display screen MG05. On the other hand, since the tag TA05 is not included in the tag information acquisition range, the tag display image based on the tag TA05 is not displayed on the display screen MG05.

<Calculation of tag information acquisition shortest distance and tag information acquisition longest distance of tag information acquisition distance calculation unit 107>
Further, the shortest tag information acquisition distance and the longest tag information acquisition distance are calculated based on the zoom magnification of the image enlargement unit 102. This calculation method will be described in detail below with reference to FIG.

  FIG. 6 shows an arc with the tag information acquisition shortest distance as the radius centered on the position of the information processing apparatus 100, an arc with the tag information acquisition longest distance as the radius centered on the position of the information processing apparatus 100, and the imaging unit 101. The range surrounded by the angle of view is the tag information acquisition range (the range indicated by hatching in FIGS. 6A, 6B, and 6C), and a top view of the image picked up by the information processing apparatus 100 Is shown. The horizontal imaging width at the reference position when the information processing apparatus 100 is in the horizontal posture is Wb1, and the horizontal imaging width at the reference position when the information processing apparatus 100 is in the vertical attitude is Wb2. Give an explanation.

  First, a method for calculating the distance Lc from the reference position to the tag information acquisition shortest distance (equal to the distance from the reference position to the tag information acquisition longest distance) Lc will be described with reference to FIG.

  FIG. 5A shows that the image enlargement unit 102 is set to the minimum zoom magnification (1 × in the present embodiment) and the information processing apparatus 100 is imaged in a horizontal posture. In order to display the tag display image on the display screen based on the tag information immediately before the information processing apparatus 100, the tag information acquisition shortest distance Rmin1 when the image enlargement unit 102 is set to the minimum zoom magnification may be zero. . That is, the distance Lc may be a distance from the information processing apparatus 100 to the reference position when the image enlargement unit 102 is set to the minimum zoom magnification. In this case, the tag information acquisition maximum distance Rmax1 is obtained as a distance obtained by doubling the distance Lc, and the tag information acquisition range is a range of 0 to 2Lc within the horizontal angle of view of the information processing apparatus 100. For example, the horizontal angle of view is 60 degrees, and the standard tag is a quadrangle having a size of 200 mm square. The horizontal length of the tag display image is 1/10 of the horizontal direction of the display screen. 2, the horizontal imaging width Wb1 at the reference position is 2 m, the distance Lc is approximately 1.73 m, and the tag information acquisition range is a range of 0 m to 3.46 m within the horizontal angle of view of the information processing apparatus 100. It becomes.

  Next, the distance from the information processing apparatus 100 to the reference position when the image enlargement unit 102 is set to m times will be described with reference to FIG.

  FIG. 6B shows a state where the image enlargement unit 102 is set to m times (> 1 time) and the information processing apparatus 100 is set in a horizontal posture for imaging. When the screen enlargement unit 102 operates so that the length of the subject displayed on the display screen becomes m times when the zoom magnification is increased from 1 time to m times, as shown in FIG. When the zoom magnification of the image enlargement unit 102 is set to m times, the reference position is a position at a distance mLc from the information processing apparatus 100. In this case, the tag information acquisition shortest distance Rmin2 is (mLc−Lc), the tag information acquisition longest distance Rmax2 is (mLc + Lc), and the tag information acquisition range is within the horizontal angle of view of the information processing apparatus 100 (mLc−Lc) ˜ ( mLc + Lc). For example, when the zoom magnification of the image enlargement unit 102 is set to 4 times, the distance from the information processing apparatus 100 to the reference position is 6.92 m, and the tag information acquisition range is 5. The range is 19m to 8.65m.

  Next, the distance from the information processing apparatus 100 to the reference position when imaging is performed in a vertical posture and the image enlargement unit 102 is set to m times will be described with reference to FIG.

  FIG. 6C shows a state where the information processing apparatus 100 of FIG. 6B is rotated 90 degrees around the center of the lens and imaged in a vertical posture. Since the position of the reference position does not change due to the rotation, the tag information acquisition shortest distance Rmin2 and the tag information acquisition longest distance Rmax2 have the same values as in the horizontal posture of FIG. In this case, since the horizontal angle of view is the vertical angle of view of the information processing apparatus 100, the tag information acquisition range is a range of (mLc−Lc) to (mLc + Lc) within the vertical angle of view of the information processing apparatus 100.

As described above, the tag information acquisition distance calculation unit 107 calculates the tag information acquisition shortest distance and the tag information acquisition longest distance based on the zoom magnification of the image enlargement unit 102. Therefore, the tag information acquisition distance calculation unit 107 receives information on the current zoom magnification from the image enlargement unit 102, and calculates the tag information acquisition shortest distance and the tag information acquisition longest distance based on the zoom magnification.
Note that the distance Lc used for calculating the tag information acquisition shortest distance and the tag information acquisition longest distance is the size of the horizontal angle of view at the minimum zoom magnification of the information processing apparatus 100, the standard size of the tag, and the display screen. It is obtained from the display size and is stored in advance in the tag information acquisition distance calculation unit 107 when the information processing apparatus 100 is manufactured.

  Returning to FIG. 4, when the tag information acquisition shortest distance and the tag information acquisition longest distance are calculated in step S <b> 110, the control unit 113 uses the communication unit 112 of the information processing apparatus 100 acquired in steps S <b> 101 to S <b> 103. The current position, imaging direction, horizontal angle of view, and the tag information acquisition shortest distance and tag information acquisition longest distance acquired in step S110 are transmitted to the communication unit 203 of the search server 200 (step S111).

  Next, the search server 200 performs a tag information search process (step S112). Details of the tag information search processing by the search server 200 will be described in detail below with reference to the flowchart shown in FIG.

  First, the control unit 206 transmits the current position of the information processing device 100 transmitted from the communication unit 112 of the information processing device 100, the imaging direction and the horizontal angle of view, and the tag information acquisition shortest distance acquired in step S110 and The communication unit 203 waits for the communication unit 203 to receive the tag information acquisition longest distance, and after the reception is completed, the communication unit 203 passes various information received under the control of the control unit 206 to the tag search unit 202 (step S201).

  Next, the tag search unit 202 calculates the above-described tag information acquisition range from the various types of information received in step S201 (step S202).

  When the tag information acquisition range is calculated in step S202, the tag acquisition prohibition range specifying unit 205 next specifies tag information for which acquisition is prohibited (step S203). For example, a pub near a facility where children such as school facilities gather If there is a restaurant such as a restaurant, it is desirable not to display a tag display image for displaying an advertisement of the pub in the information processing apparatus used by the child at school facilities. Thus, when the display device is in a predetermined range, it may be desired not to display a specific tag display image.

  Therefore, the tag acquisition prohibition range specifying unit 205 specifies tag information whose acquisition is prohibited in the following manner according to an instruction from the control unit 206. First, the tag acquisition prohibition range specifying unit 205 determines whether or not the current position of the information processing apparatus 100 acquired in step S201 is within the tag acquisition prohibition setting range. The tag acquisition prohibition range storage unit 204 stores a tag acquisition prohibition setting range that is a range to which a setting for prohibiting the acquisition of tag information is added and a tag acquisition prohibition range that is a range that prohibits the acquisition of tag information in association with each other. If the current position of the information processing apparatus 100 is determined to be within the tag acquisition prohibition setting range, the tag information included in the related tag acquisition prohibition range is identified as tag information prohibited from acquisition. To do.

  With reference to FIG. 8, an example of a tag information specifying process for prohibiting acquisition by the tag acquisition prohibition range specifying unit 205 will be described. In FIG. 8, the information processing apparatus 100 located in the school area SA08 is in a horizontal posture and the vicinity is imaged as viewed from above. As shown in FIG. 8, the tag information acquisition range TA08 includes a dining area PA08. Therefore, a tag display image indicating information on a dining facility such as a pub in the dining area PA08 is displayed on the monitor 110 of the information processing apparatus 100.

  Therefore, the tag acquisition prohibition range storage unit 204 stores a school area SA08 as a tag acquisition prohibition setting range in association with a restaurant area PA08 as a tag acquisition prohibition range, and the tag acquisition prohibition range specifying unit 205 When it is determined that the current position of the information processing apparatus 100 is within the school area SA08, the tag information related to the tags TA08 and TB08 included in the dining area PA08 is specified as tag information for acquiring the prohibition, and the acquisition is prohibited. .

  As described above, the tag information included in the tag acquisition prohibited range is specified as the tag information for which acquisition is prohibited and the acquisition is prohibited, so that the information processing apparatus 100 of the child in the school area SA08 is based on the tag TC08. Although the tag image is displayed, the tag image based on the tag TB08 can be prevented from being displayed.

  Returning to FIG. 7, when tag information prohibited from being acquired is specified in step S203, the tag search unit 202 reads arbitrary tag information from all tag information stored in the tag storage unit 201 (step S204).

<Tag information>
Details of the tag information will be described first with reference to FIG. FIG. 9 is a top view of an image taken from the position of the point P with the information processing apparatus 100 in a horizontal posture. In FIG. 9A, in step S204, the tag search unit 202 receives tag information TA09, TB09, TC09, TD09 located in the vicinity of the point P where the information processing apparatus 100 is located as arbitrary tag information from the tag storage unit 201. , And TE09 are conceptually shown. Here, it is assumed that the position of the point P where the information processing apparatus 100 is located is in the tag acquisition prohibition setting range SA09, and the tag acquisition prohibition range PA09 associated with this tag acquisition prohibition setting range includes tag information TD09. The following will be described.

  As described above, each tag information includes three types of tag images for the front surface, the back surface, and the side surface, the position of each tag in the virtual space, the size of each tag in the horizontal and vertical directions, and the virtual value of each tag. Information about the arrangement direction in the space, the shape of each tag, and the display direction of each tag is included. In FIG. 9A, the tag information TA09 has a position in the virtual space (Xs1, Ys1), a horizontal size of Ht1, a vertical size of Vt1, and an arrangement direction in the virtual space of θt1. Includes information indicating that the display has a rectangular shape and the display direction is the reverse side. The tag information TB09 has a position in the virtual space of (Xs2, Ys2), a horizontal size of Ht2, a vertical size of Vt2, a placement direction in the virtual space of θt2, a rectangular shape, and a display orientation Contains information that is facing the other side. The tag information TC09 has a position in the virtual space (Xs3, Ys3), a horizontal size of Ht3, a vertical size of Vt3, an arrangement direction in the virtual space θt3, a shape of a rectangular shape, and Information indicating that the display direction is the front side is included.

FIG. 9B illustrates the concept of assuming a tag that is a virtual object (subject) on the virtual reality space based on the tag information in order to explain the tag information shown in FIG. 9A in more detail. The conceptual diagram for doing is shown. The position in the virtual space included in the tag information indicates one end of the tag, and from this one end, the tag in the virtual reality space follows the size and shape in the horizontal and vertical directions also included in the tag information. Composed. Hereinafter, the position in the virtual reality space as one end is also referred to as a “start position” of the tag. The arrangement direction indicates the direction from the X-axis direction of the tag in the virtual reality space when it is assumed that the tag exists in the real space, and the vector extended from the start position by the horizontal size and the X-axis vector The angle formed by is shown. Hereinafter, the other end of the vector extended by the horizontal size from the start position is referred to as “end position”. The display direction means the display direction of the tag in the virtual reality space when it is assumed that the tag is present in the real space. When the tag is arranged so that the start position is down and the end position is up The front and back of the right side are shown.
With the above-described concept, the tag can be handled as if it is a virtual object (subject) that exists in the virtual reality space linked to the real space and is imaged by the imaging unit 101.

Returning to FIG. 7, when arbitrary tag information is read in step S <b> 204, search for tag information acquired in the following procedure is performed.
First, the tag search unit 202 sets the position of the arbitrary tag information read in step S204 in the virtual space, that is, the tag outside the circle whose center is the current position of the information processing apparatus 100 and whose radius is the minimum tag information acquisition distance. It is determined whether or not the start position is located (step S205).

  When the start position of the tag is located outside the circle (YES in step S205), the tag search unit 202 has the current position of the information processing apparatus 100 as the center and the radius is the tag information acquisition maximum distance, It is determined whether the start position of the tag related to the arbitrary tag information read in step S204 is located (step S206).

  If the start position of the tag is located within the circle (YES in step S206), the tag search unit 202 sets the step within the angle of view determined from the current position of the information processing apparatus 100, the imaging direction, and the horizontal angle of view. It is determined whether the start position of the tag related to the arbitrary tag information read in is located (step S207).

  If it is located within the angle of view (YES in step S207), it is determined whether the arbitrary tag information read in step S204 is the tag information forbidden to be acquired specified in step S203 (step S208).

  If it is determined that the tag information is not prohibited from being acquired (NO in step S208), the read tag information is stored in the temporary storage area of the tag search unit 202 (step S209).

Next, the tag search unit 202 determines whether or not the search in steps S205 to S208 has been performed on all tag information stored in the tag storage unit 201 (step S210).
Note that the tag start position related to the arbitrary tag information read in step S204 is not located outside the circle centered on the current position of the information processing apparatus 100 and having the radius as the tag information acquisition shortest distance (in step S205). NO), when the current position of the information processing apparatus 100 is not in the circle centered on the current position of the information processing apparatus 100 and whose radius is the tag information acquisition longest distance (NO in step S206), When it is not located within the angle of view obtained from the angle of view (NO in step S207), when it is determined that the arbitrary tag information read in step S204 is tag information prohibited from being acquired (YES in step S208), Similarly, for all tag information stored in the tag storage unit 201, the above steps S205 to S20 are performed. A determination is made if with the search (step S210).

When it is determined that all the tag information stored in the tag storage unit 201 has not been searched (NO in step S210), the process returns to step S204 and the above-described processing is repeated.
If it is determined that the search has been performed for all tag information stored in the tag storage unit 201 (YES in step S210), the tag search unit 202 notifies the control unit 206 that the search has been completed, The control unit 206 reads out all tag information temporarily stored in step S209 (step S211).

  Next, the control unit 206 determines whether or not there is an end instruction for the tag information search process (step S213). If there is an end instruction (YES in step S213), the control unit 206 ends the tag information search process and instructs If there is no (NO in step S213), the process returns to step S201 and the above-described processing is repeated.

  As described above, by performing the processing from step S204 to step S211, the search server 200 is included in the tag information acquisition range based on the information transmitted from the information processing apparatus 100, and acquisition is prohibited. It is possible to search for tag information that does not exist.

  Returning to the flowchart of FIG. 4, when the tag information search processing in the search server 200 is completed in step S <b> 112, the search server 200 transmits the searched tag information to the information processing apparatus 100 via the communication unit 203. 100 receives the searched tag information via the communication unit 112 (step S113).

  Next, based on an instruction from the control unit 113, the tag display image generation unit 108 uses the tag information received in step S <b> 113, the current position of the information processing apparatus 100 acquired in step S <b> 101, and the tag virtual space included in the tag information. Tag image to be used for a tag display image displayed from among three types of tag images of a front surface tag image, a back surface tag image, or a side surface tag image included in the tag information based on the arrangement direction and the tag display direction Is selected (step S114). This selection process will be described in detail below with reference to FIG.

  FIG. 10 is a top view of an image taken from the position of the point P with the information processing apparatus 100 in a horizontal posture. In FIG. 10, it is assumed that tag information TA10, TB10, and TC10 are received in step S113.

  First, the tag display image generation unit 108 acquires information on the current position of the information processing apparatus 100 detected by the position detection unit 104 via the control unit 113, and the current position and the start position of the tag included in each tag information. The straight lines LA10, LB10, and LC10 that connect are calculated. The tag display image generation unit 108 calculates angles αt1, αt2, and αt3 formed by the straight lines and the arrangement directions of the tags included in the tag information in the virtual space. Based on the relationship between each calculated angle and the display direction of the tag included in the tag information, from among the three types of tag images of the front surface tag image, the back surface tag image, or the side surface tag image included in the tag information. Select the tag display image to be displayed.

  Specifically, if the angle formed by the straight line connecting the current position of the information processing apparatus 100 and the start position of the tag and the tag placement direction is α, the surface opposite to the tag display direction when 0 ° <α <180 °, When 180 degrees <α <360 degrees, the tag display image generation unit 108 selects the tag display orientation, and when α = 0 degrees and α = 180 degrees, the tag image on the side surface is selected.

  In FIG. 10, since the angle αt1 of the tag information TA10 is 0 degree <αt1 <180 degrees, the surface opposite to the tag display direction, that is, the surface tag image is selected as the tag image based on the tag information TA10. Similarly, a front surface tag image is selected as a tag image based on the tag information TB10, and a back surface tag image is selected as a tag image based on the tag information TC10.

  When the front tag image, the back tag image, or the side tag image is selected in step S114, the tag display image generation unit 108 starts the tag included in the tag information based on an instruction from the control unit 113. From the information regarding the position, the horizontal and vertical sizes of the tag, the arrangement direction of the tag in the virtual space, the display direction of the tag in the virtual space, the shape of the tag, and the attitude of the information processing apparatus 100 detected in step S104 The display position of the display image display unit 110 on the display screen is calculated (step S114). The display position calculation process will be described in detail below with reference to FIG.

FIG. 11 shows a calculation method when a tag display image is generated based on the tag information TA10 shown in FIG.
First, a method for calculating the horizontal display position of the tag display image will be described with reference to FIG. First, the tag display image generation unit 108 acquires the posture of the information processing apparatus 100 detected by the posture detection unit 103 via the control unit 113, and the horizontal pixel count Hp of the display screen of the display unit 110 from the information regarding the posture. And the number of pixels Vp in the vertical direction is calculated.

  Next, the tag display image generation unit 108 calculates the direction θs from the information processing apparatus 100 of the point Ts1 based on the position (Xs1, Ys1) of the point Ts1 that is the tag start position related to the tag information TA10. From the difference between the calculated direction θs and the imaging direction θh of the information processing apparatus 100, the distance Ls from the information processing apparatus 100 to the center point Cs of the imaging width at the point Ts1, and from the center point Cs to the point Ts1. Is calculated with a sign.

  Next, the imaging width Hs at the point Ts1 is calculated from the obtained distance Ls and the horizontal angle of view βh of the information processing apparatus 100. Since the calculated imaging width Hs corresponds to the number of pixels Hp in the horizontal direction of the display screen, the horizontal display position of the point Ts1 on the display screen is Hp × Ws / Hs to the right of the center of the display screen. Will be displayed at the position.

  Next, the tag display image generation unit 108 calculates a point Te1 (Xe1, Ye1) that is the tag end position from the tag start position, the tag horizontal size, and the tag placement direction in the tag information. Similarly to the start position Ts1, the tag display image generation unit 108 obtains the direction θe of the point Te1 from the information processing apparatus 100, and calculates the distance Le from the information processing apparatus 100 to the center point Ce of the imaging width at the point Te1. Then, the distance We from the center point Ce to the start point Te1 and the imaging width He at the point Te1 are calculated.

  Since the calculated imaging width He corresponds to the number of pixels Hp in the horizontal direction of the display screen, the horizontal display position of the end point Te1 on the display screen is Hp × We / He from the center of the display screen. It will be displayed in the left position.

  Next, a method for calculating the display position of the tag display image in the vertical direction will be described with reference to FIG. In the present embodiment, as described above, the information processing apparatus 100 does not have a vertical inclination, the tag display image does not have a vertical inclination, and the center of the lens of the imaging unit 101 of the information processing apparatus 100 ( Since the case where the center of the tag display image is located at the same height as the optical axis) is described, in FIG. 11B, the tag display image based on the tag information TA10 has a start position Ts1 and a start position Ts1. A square formed by a point Ts1 ′ that is vertically separated from the tag image by a vertical size Vt1 and an end position Te1 and a point Te1 ′ that is vertically separated from the end position Te1 by Vt1. (The square shown by cross-hatching in FIG. 11B), the center point Cs ′ of the points Ts1 and Ts1 ′, and the center point Ce ′ of the points Te1 and Te1 ′ are the information processing apparatus. 00 will be described as being located at the same height as the optical axis of the.

  First, the tag display image generation unit 108 calculates the imaging width Vs at the start position Ts1 from the distance Ls from the information processing apparatus 100 to the start position Ts1 and the angle of view βv in the vertical direction. Since the calculated imaging width Vs corresponds to the number of pixels Vp in the vertical direction of the display screen, the vertical display position of the start position Ts1 on the display screen is Vp × Vt1 / (2 × Vs) is displayed at the upper position, and the vertical display position of the point Ts1 ′ on the display screen is displayed at a position Vp × Vt1 / (2 × Vs) below the center of the display screen. Will be.

  Similarly to the start position Ts1 and the point Ts1 ′, the tag display image generation unit 108 captures an image at the end position Te1 from the distance Le from the information processing apparatus 100 to the end position Te1 and the vertical angle of view βv. The width Ve is calculated. Since the calculated imaging width Ve corresponds to the number of pixels Vp in the vertical direction of the display screen, the vertical display position of the end position Te1 on the display screen is Vp × Vt1 / (2 × Ve) is displayed at the upper position, and the vertical display position of the point Te1 ′ on the display screen is displayed at a position Vp × Vt1 / (2 × Ve) below the center of the display screen. Will be.

  11 describes the case where the posture of the information processing apparatus 100 is a horizontal posture. However, when the information processing device 100 is a vertical posture or a posture rotated 180 degrees from the horizontal direction, the tag image generation unit 108 can operate to calculate the display position on the display screen based on the attitude of the display device 100 acquired from the attitude detection unit 103.

  Returning to FIG. 4, when the display position of the tag display image is calculated in step 115, the tag display image generation unit 108 processes the tag image selected in step S114 so as to be the display position calculated in step S115. Then, a tag display image to be output to the image composition unit 109 is generated. Then, the tag display image generation unit 108 outputs the generated tag display image to the image synthesis unit 109, and the image synthesis unit 109 synthesizes the tag display image and the captured image output from the imaging unit 101 ( Step S116).

  Then, the display unit 110 displays the image synthesized in step S116 on the display screen of the display unit 110 based on an instruction from the control unit 113 (step S117).

  FIG. 12 shows a case where the tag display image TGA10 based on the tag information TA10 in FIG. 10 whose display position is calculated in step S116 is displayed on the display screen MG12 of the display unit 110 with the information processing apparatus 100 in a horizontal posture. It is. A tag display image generated from the tag image on the surface selected by the tag display image generation unit 108 is displayed on the display screen of the display unit 110, and the photographer can acquire desired information from the tag display image.

  Further, since the information processing apparatus 100 is not located directly in front of the tag information TA10, the tag display image TGA10 based on the tag information TA10 is displayed so that the width in the vertical direction becomes narrower in the right direction of the display screen. From the shape of the tag display image, the photographer can instantly grasp that the tag information TA10 is not located directly in front of the information processing apparatus 100.

  FIG. 13 shows a case in which tag display images TGA09, TGB09, and TGC09 based on the tag information TA09, TB09, and TC09 of FIG. 9 are displayed on the display screen MG13 of the display unit 110 with the information processing apparatus 100 in a horizontal posture. It is shown. Each tag display image is displayed on the display screen in a size, shape, and orientation based on each tag information, and the photographer can instantly grasp the positional relationship of each tag information. Further, the tag display image of the tag information TC09 is generated from the tag image on the back surface, and the photographer can grasp from this image that the tag related to the tag information TC09 is in the opposite direction.

  In FIG. 13, the tag display image TGC09 generated from the back tag image is also displayed on the display screen in the same manner as the tag display images TGA09 and TGB09 generated from the front tag image. When the tag image on the back side is configured not to have any information, the tag display image displayed on the back side can be hidden. Specifically, when the tag image generation unit 108 selects a corresponding tag image from a plurality of tag images, the tag image on the back side may be processed so as not to display the tag display image of the selected tag. The same applies to the case where the side tag image is selected.

  Returning to FIG. 4, the control unit 113 determines whether or not there is an end instruction for the acquisition of tag information and the generation and display processing of the tag display image (step S118). If there is an end instruction (YES in step S118), If the process is terminated and there is no termination instruction (NO in step S118), the process returns to step S101 and the above-described process is repeated.

As described above, according to the information processing system 1 according to the embodiment of the present invention, the object display image generated based on the information acquired in the virtual reality space is appropriately associated with the subject image in the captured image. Can be superimposed.
Further, according to the information processing system 1 according to the embodiment of the present invention, the object display image generated based on the information acquired in the virtual reality space is superimposed and displayed on the subject image in the captured image. In addition, the user can easily visually recognize a desired object display image from among the plurality of superimposed object display images.

  In the present embodiment, the information processing system 1 including the information processing apparatus 100 and the search server 200 has been described. However, the information processing apparatus includes a part of the constituent members included in the search server 200, or the information processing apparatus 100 includes. The effect of the present invention can also be obtained in an information processing system configured such that a search server includes a part of the constituent members. For example, a configuration in which the search server includes a member that plays the same role as the tag display image generation unit 108, and the tag display image generated on the search server side is transmitted to the information processing apparatus, etc. According to this configuration, it is possible to reduce the processing load related to tag image generation on the information processing apparatus side. Alternatively, a configuration in which the information processing apparatus includes a member that plays the same role as the tag acquisition prohibition range storage unit 204 and the tag acquisition prohibition range specification unit 205 is also possible. According to this configuration, the search server can search for tag information after the tag acquisition prohibited range is specified by the information processing apparatus.

<Second Embodiment: Information Processing System>
In the first embodiment, an information processing system 1 in which an information processing apparatus 100 that acquires tag information and generates a tag display image and a search server 200 that searches for tag information exchange various information by wireless communication. As an example, here, an information processing apparatus 300 that performs all of acquisition of tag information, generation of a tag display image, and search of tag information will be described as an example.

FIG. 14 is a block diagram showing the configuration of the information processing apparatus 300 according to the embodiment of the present invention. The information processing apparatus 300 according to the present embodiment can capture a moving image and a still image through a built-in imaging unit.
The present invention can be applied to other electronic devices capable of imaging such as a digital still camera and a digital video camera, a mobile phone, a PHS (Personal Handyphone System), and a PDA (Personal Digital Assistant). The above is the same as the information processing apparatus 100 in the information processing system 1 described above.

<Configuration of Information Processing Device 300>
The configuration of the information processing apparatus 300 will be described below. The information processing apparatus 300 includes an imaging unit 301 that acquires a captured image of a subject, zooms the captured image, and the like, a tag search information processing unit 302 that processes information for searching tag information, and a tag search that searches tag information It is mainly composed of a processing unit 303, a tag display processing unit 304 that generates and displays a tag display image based on tag information, a control unit 313, and the like. In addition, regarding the members constituting these parts, the same symbols as those of the information processing system 1 will be assigned to those that control substantially the same processing as the members of the information processing system 1 described above.

The control unit 313 centrally controls various functions such as imaging and display of the information processing apparatus 100. For example, optical zoom magnification and digital zoom magnification corresponding to the zoom magnification setting value read from the operation unit 111 are set in the image enlargement unit 102, and optical control such as focus adjustment and exposure adjustment for the imaging unit 101 is performed. I do. Alternatively, as control related to display on the display unit 110, for example, display / non-display control, control regarding backlight brightness, power supply control of the information processing apparatus 100, and the like are performed. In addition, the above-described various members are controlled in an integrated manner.
Further, the control unit 313 performs processing of information for searching tag information by the tag search information processing unit 302, processing of searching tag information by the tag search processing unit 303, and generation of a tag display image by the tag display processing unit 304. Supervises display processing.

    The ROM 314 stores a program for the operation of the control unit 313. The RAM 315 is used as a temporary storage area for various data, or as a work area at the time of calculation related to various programs.

<Configuration of Imaging Unit 301>
The imaging unit 301 functions as a captured image acquisition unit that acquires a captured image of a subject by photoelectrically converting imaging light entering a lens system (not shown) by an imaging element (not shown) and converting the light into a video signal. Both are members that perform optical functions such as focusing the subject on the lens system.

  The image enlarging unit 102 optically zooms a lens system (not shown) with a lens driving member such as a zoom actuator (not shown) or electronically zooms a captured image held by an image sensor (not shown). In other words, it is a member that functions as a captured image magnification changing unit that enlarges the captured image captured by the imaging unit 301.

<Configuration of Tag Search Information Processing Unit 302>
The posture detection unit 103 is a member that detects the posture of the information processing apparatus 300, and includes a method of detecting a direction with respect to gravity using an element (acceleration sensor) that detects gravitational acceleration, and an acceleration sensor (gyroscope). By using a method for obtaining the rotation angle by integrating the information, information related to the posture of the information processing device 300, for example, the horizontal inclination of the information processing device 300 from the horizontal posture is detected. Information on the posture detected by the posture detection unit 103 is used for calculation of a horizontal angle of view by an angle-of-view calculation unit 106 described later, and a captured image of a tag display image generated by a tag display image generation unit 108 described later. It is used to calculate the position of superimposing on

  The position detection unit 104 is a member that detects the current position of the information processing apparatus 300 by using a GPS satellite or the like. The direction detection unit 105 is a member that detects in which direction the information processing apparatus 300 is directed by using an electronic compass or the like.

  The angle-of-view calculation unit 106 is a member that calculates the horizontal angle of view of the imaging unit 301 based on the zoom magnification that the image enlargement unit 102 enlarges and information on the posture of the information processing device 300 that the posture detection unit 103 detects. It is. The view angle calculation unit 106 calculates the horizontal view angle and the vertical view angle of the image pickup unit 301 of the information processing device 300 based on the zoom magnification that the image enlargement unit 102 enlarges. For example, when the posture of the information processing apparatus 300 detected by the posture detection unit 103 is less than 45 degrees with respect to the horizontal posture, the horizontal angle of view is set as the horizontal angle of view, and when the angle is 45 degrees or more with respect to the horizontal posture. Let the vertical angle of view be the horizontal angle of view.

  The tag information acquisition distance calculation unit 107 is a tag information acquisition shortest distance that is the shortest distance for acquiring tag information based on a zoom magnification at which the image enlargement unit 102 enlarges the captured image, and a tag that is the longest distance for acquiring tag information. It is a member that performs processing for calculating the maximum information acquisition distance in the same manner as the information processing system 1 described above.

  The tag search information processing unit 302 configured as described above acquires the current position of the information processing device 300 necessary for specifying the tag information acquisition range, the imaging direction and the horizontal angle of view, and the tag information acquisition shortest distance and tag information acquisition. The longest distance is transferred to the tag search processing unit 303 based on an instruction from the control unit 313.

<Tag search processing unit 303>
The tag search unit 202 of the tag search processing unit 303 acquires the current position of the information processing device 300 delivered from the tag search information processing unit 302, the imaging direction and the horizontal angle of view, and the tag information acquisition shortest distance and tag information acquisition. The tag information acquisition range is calculated from the longest distance. The calculation of the tag information acquisition range is performed in the same manner as the processing in the information processing system 1 described above.

  When the tag information acquisition range is calculated, the tag acquisition prohibition range specifying unit 205 next specifies tag information whose acquisition is prohibited. When tag information prohibited from being acquired is specified, the tag search unit 202 reads arbitrary tag information from all tag information stored in the tag storage unit 201, and searches for tag information to be acquired. These processes are also performed in the same manner as the information processing system 1 described above.

  When the search of the tag information to be acquired by the tag search processing unit 303 is completed, the tag information is transferred to the tag display processing unit 304 in accordance with an instruction from the control unit 313.

  Based on the instruction from the control unit 313, the tag display image generation unit 108 determines the current position of the information processing device 300 and the arrangement direction of the tag included in the tag information in the virtual space and the display direction of the tag from the delivered tag information. Based on the above, a tag display image to be displayed is selected from the three types of tag images including the front surface tag image, the back surface tag image, and the side surface tag image included in the tag information.

  Next, based on an instruction from the control unit 313, the tag display image generation unit 108 includes a tag start position, tag horizontal and vertical sizes, tag placement direction in the tag virtual space, and tag tag virtual space. The display position of the tag display image on the display screen of the display unit 110 is calculated from the information regarding the display orientation, the shape of the tag, and the attitude of the information processing apparatus 100. The calculation of the display position is also performed by the same method as in the information processing system 1 described above.

  When the display position of the tag display image is calculated, the tag display image generation unit 108 processes the selected tag image so that it becomes the calculated display position, and generates a tag display image to be output to the image composition unit 109. To do. Then, the tag display image generation unit 108 outputs the generated tag display image to the image synthesis unit 109, and the image synthesis unit 109 synthesizes the tag display image and the captured image output from the imaging unit 301. The display unit 110 displays the synthesized image on the display screen of the display unit 110 based on an instruction from the control unit 313.

As described above, according to the information processing apparatus 300 according to the embodiment of the present invention, the object display image generated based on the information acquired in the virtual reality space is appropriately associated with the subject image in the captured image. Can be superimposed.
Further, according to the information processing apparatus 300 according to the embodiment of the present invention, the object display image generated based on the information acquired in the virtual reality space is displayed in a superimposed manner on the subject image in the captured image. In addition, the user can easily visually recognize a desired object display image from among the plurality of superimposed object display images.

  In the description of each of the above embodiments, the description has been given for the sake of explanation and understanding, but the present invention is not limited to this.

  For example, taking the information processing system 1 according to the first embodiment as an example, the attitude detection unit 103 newly adds the vertical direction of the information processing apparatus 100 in addition to the attitude of the information processing apparatus 100 (tilt in the horizontal direction). The position detection unit 104 uses vertical position detection means such as an altimeter in addition to detection of the current position of the information processing apparatus 100 in the horizontal direction by horizontal position detection means such as GPS. Thus, the current position (height) in the vertical direction of the information processing apparatus 100 is newly detected.

  Next, the angle of view calculation unit 106 newly calculates the angle of view in the vertical direction in addition to the angle of view of the information processing apparatus 100 in the horizontal direction, and the communication unit 112 calculates the horizontal direction calculated by the angle of view calculation unit 106. The angle of view and the angle of view in the vertical direction, the current position in the horizontal direction of the information processing apparatus 100 detected by the position detection unit 104 and the current position (height) in the vertical direction of the information processing apparatus 100 detected by the direction detection unit 105. The current imaging direction, the tag information acquisition shortest distance and the tag information acquisition longest distance calculated by the tag information acquisition distance calculation unit 107 are transmitted to the communication unit 203.

  As tag information, in addition to the tag image, tag position, tag size, tag display direction, tag placement direction (tag horizontal tilt), tag shape, new tag vertical tilt Information about the height of the tag is stored in the tag storage unit 201, and the tag search unit 202 stores the horizontal and vertical positions of the tag, the size of the tag, the tag stored in the tag storage unit 201 Tag information is calculated which includes information on the arrangement direction of the tag, the display direction of the tag, the inclination of the tag in the horizontal and vertical directions, and the tag shape.

Furthermore, the tag search unit 202 includes a vertical inclination of the information processing apparatus 100 detected by the attitude detection unit 103, a current position in the horizontal and vertical directions of the information processing apparatus 100 detected by the position detection unit 104, and a direction detection unit 105. The current imaging direction of the information display device 100 detected by
The tag display range information is calculated from the horizontal and vertical field angles calculated by the field angle calculation unit 106, and the tag information acquisition shortest distance and tag information acquisition longest distance calculated by the tag information acquisition distance calculation unit 107. calculate.

  According to the above, the information processing apparatus has a vertical inclination, the object (tag) has a vertical inclination, and the center (optical axis) of the lens of the imaging unit included in the information processing apparatus and the object ( It is clear that the effects of the present invention can be achieved even when the centers of the tags are different.

  In each of the above-described embodiments, the tag information acquisition shortest distance and the tag information acquisition longest distance calculated by the tag information acquisition distance calculation unit 107 are both arcs centered on the position of the information processing apparatus 100 or 300, and tag information acquisition is performed. The range is the fan-shaped range formed by the arc of the information processing apparatus 100 or 300 and the arc of the tag information acquisition longest distance is removed from the fan-shaped range formed of the arc of the information processing apparatus 100 or 300 and the tag information acquisition shortest distance. Although it was described as being in the range, it is not limited to this.

  For example, the tag information acquisition shortest distance and the tag information acquisition longest distance are defined as the distance from the information processing device to the position on the optical axis, and the range formed by the angle of view of the information processing device and the tag information acquisition shortest distance is an isosceles triangle The range of the longest distance between the information processing device and the tag information acquisition is the isosceles triangle shape, and the range obtained by removing the front isosceles triangle range from the rear isosceles triangle range, that is, the trapezoidal range tag information It is good also as an acquisition range.

  In each of the above embodiments, the tag image generation unit 108 has been described as selecting a tag image based on the tag arrangement direction and the tag display direction. However, the present invention is not limited to this. The direction in which the surface of the tag is displayed is obtained from the tag arrangement direction and the tag display direction. Therefore, the two pieces of information are collectively stored in the tag information as the tag display direction, and the tag display image generation unit 108 is displayed. The tag image may be selected based on the display direction of the tag.

  Further, in each of the embodiments described above, the tag information stored in the tag storage unit 201 includes information on the tag image, the tag position, the tag size, the tag arrangement direction, the tag display direction, and the tag shape. Although described as being included, it is not limited to this. If tag information does not include information about tag size, tag placement direction, tag display orientation, and tag shape, specify the tag size, tag placement direction, tag display orientation, and tag shape. The tag display image generation unit 108 may prepare the tag display image by using this predetermined value.

Information processing system 1
Information processing apparatus 100, 300
Search server 200
Imaging unit 101
Image enlargement unit 102
Posture detection unit 103
Position detection unit 104
Direction detection unit 105
Angle of view calculation unit 106
Tag information acquisition distance calculation unit 107
Tag image generation unit 108
Image composition unit 109
Display unit 110
Communication unit 112
Control unit 113, 206
Tag storage unit 201
Tag search unit 202
Communication unit 203
Tag acquisition prohibition range storage unit 204
Tag acquisition prohibition range specifying unit 205

Claims (6)

An information processing system in which an information processing device and a search device communicate with each other via a network,
The information processing apparatus includes:
An imaging unit that images a subject and obtains a captured image;
An imaging magnification changing unit that changes an imaging magnification of the imaging unit; a position detecting unit that detects a position of the information processing device;
A direction detection unit for detecting an imaging direction of the imaging unit;
An angle-of-view calculating unit that calculates an angle of view of an image captured by the imaging unit;
The search device includes information including an imaging magnification changed by the imaging magnification changing unit, a position detected by the position detecting unit, an imaging direction detected by the direction detecting unit, and an angle of view calculated by the angle of view calculating unit. An information transmitter to transmit to
An object display image generation unit that generates an object display image using the object information;
An object information receiving unit that receives object information used by the object display image generation unit from the search device, a captured image acquired by the imaging unit, and an object display image generated by the object display image generation unit are displayed. A display unit,
The search device includes:
An information receiver that receives the information transmitted by the information transmitter included in the information processing apparatus;
A storage unit storing a plurality of pieces of object information;
The plurality of object information stored in the storage unit is searched based on the information received by the information reception unit, and the object information used by the object display image generation unit of the information processing device is specified from the storage unit A search section to
An object information transmitting unit that transmits the object information specified by the search unit to an object information receiving unit included in the information processing apparatus, and a plurality of object acquisition prohibition ranges in which object information that prohibits transmission by the object information transmitting unit exists Object acquisition prohibited range storage
A predetermined object acquisition prohibition range is specified from the position detected by the position detection unit in the information received by the information reception unit and the imaging direction detected by the direction detection unit, and the search unit acquires the specified object acquisition An information processing system comprising: an object acquisition prohibition range specifying unit that excludes object information included in a prohibition range from the object information used by the object display image generation unit of the information processing apparatus. The information processing apparatus further includes an object information acquisition distance calculation unit,
The object information acquisition distance calculation unit includes an object information acquisition shortest distance that is the shortest distance for acquiring object information based on the imaging magnification changed by the imaging magnification change unit, and object information that is the longest distance for acquiring object information. Calculate the longest acquisition distance,
The information transmission unit obtains object information acquisition range information including the object information acquisition shortest distance, the object information acquisition longest distance, the imaging direction detected by the direction detection unit, and the angle of view calculated by the angle of view calculation unit. The information processing system according to claim 1, wherein the information processing system transmits the information to the search device. The information processing apparatus further includes a posture detection unit,
The posture detection unit detects the posture of the information processing apparatus;
3. The angle of view calculation unit according to claim 1, wherein the angle of view calculation unit calculates an angle of view of an image picked up by the image pickup unit based on a posture of the information processing apparatus detected by the posture detection unit. The information processing system described. Information processing comprising: an imaging unit that images a subject and acquires a captured image; an imaging magnification changing unit that changes a magnification of imaging by the imaging unit; and a display unit that combines and displays the captured image and an object display image A search device that communicates with a device via a network,
An information receiving unit that receives information about the imaging magnification changed by the imaging magnification changing unit, the position of the information processing device, the imaging direction of the imaging unit, and the angle of view of imaging by the imaging unit from the information processing device; ,
A storage unit storing a plurality of object information for generating the object display image;
A search unit for searching the plurality of object information stored in the storage unit based on the information received by the information receiving unit, and specifying predetermined object information from the storage unit;
Object acquisition prohibition range for storing a plurality of object acquisition prohibition ranges in which object information transmission unit for transmitting predetermined object information specified by the search unit to the information processing apparatus and object information for prohibiting transmission by the object information transmission unit exist A storage unit;
A predetermined object acquisition prohibition range is specified from the position and imaging direction in the information received by the information reception unit, and the search unit transmits object information included in the specified object acquisition prohibition range to the information processing apparatus. A search device comprising an object acquisition prohibition range specifying unit that is excluded from the object information. The information processing apparatus further includes an object information acquisition distance calculation unit,
The object information acquisition distance calculation unit includes an object information acquisition shortest distance that is the shortest distance for acquiring object information based on the imaging magnification changed by the imaging magnification change unit, and object information that is the longest distance for acquiring object information. Calculate the longest acquisition distance,
Object information acquisition range information composed of the object information acquisition shortest distance and the object information acquisition longest distance calculated by the object information acquisition distance calculation unit, the imaging direction of the imaging unit, and the angle of view of imaging by the imaging unit. The search apparatus according to claim 4, wherein the information is received from the information processing apparatus as the information. A program that executes processing in which an information processing device and a search device communicate with each other via a network,
  The information processing apparatus includes:
  An imaging process for imaging a subject and acquiring a captured image;
  An imaging magnification changing process for changing an imaging magnification by the imaging process;
  Position detection processing for detecting the position of the information processing apparatus;
  Direction detection processing for detecting the imaging direction of the imaging processing;
  An angle-of-view calculation process for calculating an angle of view of imaging by the imaging process;
  The search device includes information including the imaging magnification changed by the imaging magnification changing process, the position detected by the position detection process, the imaging direction detected by the direction detection process, and the angle of view calculated by the angle of view calculation process. Information transmission processing to be transmitted to
  Object display image generation processing for generating an object display image using object information;
  An object information reception process for receiving object information used by the object display image generation process from the search device;
  Performing a display process of combining and displaying the captured image acquired by the imaging process and the object display image generated by the object display image generation process;
  The search device includes:
  An information reception process for receiving the information transmitted by the information transmission process executed by the information processing apparatus;
  A storage process for storing a plurality of object information;
  The plurality of object information stored by the storage process is searched based on the information received by the information reception process, and the object information used by the object display image generation process of the information processing apparatus is identified from the storage process Search process to
  An object information transmission process for transmitting the object information specified by the search process to an object information reception process executed by the information processing apparatus;
  Object acquisition prohibited range storage processing for storing a plurality of object acquisition prohibited ranges in which object information that prohibits transmission by the object information transmission processing exists;
A predetermined object acquisition prohibition range is specified from the position detected by the position detection process in the information received by the information reception process and the imaging direction detected by the direction detection process, and the search process is the acquisition of the specified object A program for executing object acquisition prohibition range specifying processing in which object information included in a prohibition range is excluded from the object information used by the object display image generation processing of the information processing apparatus.