JP2013105466A - Display device, display method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a display device or the like suitable for effectively displaying, on a screen, a position of a target existing at the outside of a field of vision of a camera.SOLUTION: In a display device 300, a detection part 301 detects a position and a direction of a camera. An acquisition part 302 acquires a position of a target in an investigation area defined by the detected position and the direction of the camera. A creation part 303 draws, at a photographic area in a projection surface, a photographed image obtained by photographing, by the camera, a visual field area defined by the position and the direction of the camera, draws a mark indicating the target at a position where the acquired position of the target is projected at a reporting area in the projection surface by a predetermined projection transformation, and thereby creating a display image. A display part 304 displays the created display image on a screen.

Description

  The present invention relates to a display device, a display method, and a program suitable for effectively displaying the position of a target outside the field of view of a camera on a screen.

  In recent years, augmented reality (AR) display technology that superimposes a virtual object on a real-world image photographed by a camera has been developed and is being put to practical use as a game, a communication tool, or the like. For example, in Patent Document 1, when a virtual object is superimposed on a real-world image using AR technology, the color of the virtual object is corrected according to the real-world image, thereby harmonizing with the real world. A technique for displaying a virtual object is disclosed.

JP 2010-170316 A

  In the AR technology as described above, usually, a virtual object is superimposed on a target in a captured image, that is, a target that falls within the field of view of the camera, thereby adding various information about the target in the field of view. It was what was presented. However, on the other hand, there are cases where it is desired to obtain information by displaying a virtual object in association with not only an object that falls within the visual field of the camera but also an object that is located outside the visual field. For example, when you want to get information about nearby buildings, etc., if you do not adjust the camera orientation so that the building fits in the camera's field of view, you will not be able to get the information. This is not possible when the user does not know the exact location of the building.

  The present invention solves the above-described problems, and provides a display device, a display method, and a program suitable for effectively displaying the position of a target outside the field of view of the camera on the screen. The purpose is to do.

  In order to achieve the above object, a display device according to a first aspect of the present invention is a display device including a detection unit, an acquisition unit, a generation unit, and a display unit, and is configured as follows.

  The detection unit detects the position and orientation of the camera.

  That is, the display device includes a camera that captures the real world, and the detection unit detects the position and orientation of the camera. For example, when a camera is mounted on a device that can be carried by the user, the position and orientation of the camera can change variously as the user moves and operates. In this case, the detection unit detects the current position and the current direction of the camera every time the position and the direction of the camera change variously.

  Here, the “camera position” is the position of the camera in the real world, and is represented by, for example, latitude and longitude. The “camera direction” is the direction of the line of sight of the camera at the “camera position”, and is represented by, for example, an azimuth angle based on east, west, south, and north. Note that the “camera position” may be represented by an altitude in addition to the latitude and longitude, including the position in the vertical direction in the three-dimensional space. Further, the “camera direction” may be expressed by an elevation angle with reference to the horizontal plane in addition to the azimuth angle with reference to the east, west, south, and north directions, including the vertical direction in the three-dimensional space.

  The camera is typically integrated with the display device, but may be separated from the display device. For example, a camera and a display unit that displays an image captured by the camera may be divided into different devices.

  The acquisition unit acquires the position of the target in the investigation area determined by the detected position and orientation of the camera.

  Here, the “survey area” is an area in the real world three-dimensional space, and is an area determined based on the position and orientation of the camera detected by the detection unit. For example, as an investigation area, an area surrounded by a solid body such as a pyramid or a cone opened at the camera direction as a vertex, an area surrounded by a spherical solid including the camera position, and a camera from the camera position An area surrounded by a predetermined solid at a position separated by a predetermined distance in the direction of. The acquisition unit acquires the position of the target in such a survey area.

  Here, the “target” is a building, facility, equipment, or the like that exists in the real world, and the user wants to know its position. For example, a game center, a convenience store, a vending machine, a bus stop, etc. are mentioned as a target. An acquisition part acquires the information of the position about the target in the above investigation areas among such targets.

  The generation unit draws a captured image obtained by the camera capturing a visual field region determined by the position and orientation of the camera in a photographic region within the projection plane, and the position of the acquired target is obtained by predetermined projection conversion. A display image is generated by drawing a mark representing the target at a position projected in the report area in the projection plane.

  That is, the generation unit prepares a virtual projection plane, and draws a real-world captured image by the camera and a mark representing the target on the projection plane, so that the display unit described later is displayed on the screen of the display device. A display image for display is generated. Here, out of the two-dimensional areas in the projection plane, an area where a real-world photographed image is drawn by a camera is called a photo area, and an area where a mark representing a target is drawn is called a report area.

  Among these, a real-world photographed image by the camera is obtained by the camera photographing a field of view defined by the position and orientation of the camera. Here, the “field of view” is a region in the three-dimensional space that defines the field of view of the camera. For example, typically, the field of view is a quadrangular pyramid that has the camera position at the top and opens in the direction of the camera. Determined. A real-world building, facility, or the like that falls within such a visual field area is projected onto a photographic area within a virtual projection plane, thereby being drawn as a photographed image by a camera.

  On the other hand, the mark representing the target drawn on the projection plane in addition to the image taken by the camera is projected by the predetermined projection transformation of the position of the target in the investigation area acquired by the acquisition unit. can get. Here, the “predetermined projection transformation” refers to associating an arbitrary position in the three-dimensional survey area with any position in the two-dimensional report area in the projection plane. By this projection conversion, the position of the target in the investigation area is projected onto the projection plane, and a mark representing the target is drawn at the projected position.

  The display unit displays the generated display image on the screen.

  That is, the display unit displays the display image generated by the generation unit on a screen such as a monitor provided in the display device. Since the displayed image is drawn with a camera image and a mark representing the target, the user can know where the target is based on the captured image.

  The report area includes a photographic area.

  In other words, the report area in which the mark representing the target is drawn occupies a two-dimensional area in the projection plane so as to include a photograph area in which an image captured by the camera is drawn. By taking an area wider than the photographic area as the report area, the generation unit can generate the position of the target for a range wider than the field of view of the camera as a display image. For example, for a target located within the field of view of the camera, the mark is drawn on the image captured by the camera, and for a target located outside the field of view of the camera, the mark is located outside the image captured by the camera. It is typically drawn.

  The investigation area has a portion that is not included in the visual field area.

  That is, the investigation area where the position of the target is acquired by the acquisition unit has a portion that is not included in the visual field area of the camera, that is, a three-dimensional space outside the visual field of the camera. For example, when the field of view of the camera is defined by a quadrangular pyramid solid, the survey area has an outer region that is not included in the quadrangular pyramid solid.

  With such a configuration, the display device of the present invention acquires the position of the target for an area that is not included in the visual field area of the camera, and displays it on the display image together with the image captured by the camera. Thereby, even if the target is out of the field of view of the camera, the user can recognize which position the target is on the basis of the captured image of the camera.

In the display device of the present invention,
The survey area can be configured to include a field of view area near the camera position.

  That is, the survey area has a portion that is not included in the visual field area as described above, but more specifically, here, the survey area is included in the visual field area so as to include the visual field area in the vicinity of the position of the camera. Has no part. Here, “including the visual field region near the camera position” means that the investigation region includes the visual field region in a nearby range where the distance from the camera position is within a predetermined distance. is there. That is, the survey area does not necessarily include the visual field area far from the camera position, but becomes a region in the three-dimensional space that includes the visual field area in the vicinity within a predetermined distance from the camera position. . The display device also acquires the position of the target located outside the visual field area of the camera within such a range that the distance from the camera position is within a predetermined distance, and the mark representing the target is captured by the camera. It is displayed on the screen as a display image together with the image.

  With such a configuration, the display device of the present invention takes an investigation region so as to include a visual field region in the vicinity of the position of the camera. As a result, it is possible to prevent the target from being missed because it is not in the field of view of the camera even though the target is in the vicinity of the user. Can be provided to the user.

In the display device of the present invention,
The predetermined projection transformation can be configured to match the projection transformation that projects the field of view onto the photographic region.

  That is, the generation unit sets the field of view of the camera in the projection plane as the predetermined projection conversion when projecting the position of the target in the survey area onto the report area in the projection plane. The same projection transformation as that projected onto the photograph area is used. By using the same projection transformation, the generation unit can indicate the position of the target in the camera field of view by superimposing it on the image captured by the camera, and further, the target in a position outside the camera field of view. With respect to the above, it is possible to generate a display image indicating the position at which the image taken by the camera is used as a reference.

  For example, a target that is off the camera's field of view to the left is drawn in the display image at a position deviated to the left by the amount deviated from the field of view of the captured image. The target at the position deviated from the image is drawn in a position deviated to the right by the amount deviated from the field of view with respect to the drawing position of the captured image in the display image.

  With such a configuration, the display device of the present invention displays the position of the target outside the field of view of the camera in the display image using the same projection transformation as that used for photographing a building or the like within the field of view of the camera. To do. As a result, the target in the camera field of view can be provided to the user in correspondence with the filmed building, etc., and the target that is out of the camera field of view can be selected based on the captured image of the camera. The user can recognize whether the position is present.

In the display device of the present invention,
The field of view is defined by the first cone,
The survey area is defined by the second cone,
The axis of the first cone and the axis of the second cone overlap,
The second cone can be configured to encompass the first cone.

  In other words, the field-of-view area of the camera and the survey area from which the position of the target is acquired are two cones, a first cone and a second cone (three-dimensional cones such as a quadrangular pyramid and a cone). 3D). First, the first cone defining the field of view of the camera is a cone with the camera position at the apex and a straight line extending in the direction of the camera as the axis, and the apex angle of the cone is the angle of view of the camera (field of view). Corner). In general, the field of view of a camera is often determined by a quadrangular pyramid having two angles of view, a horizontal field of view and a vertical field of view. However, the field of view is not necessarily limited to a quadrangular pyramid, and the first cone that defines the field of view of the camera. The body may be a pyramid other than a quadrangular pyramid, a cone, an elliptical cone, or the like.

  In addition, the second cone that defines the survey area from which the position of the target is acquired is a cone having a common axis with the first cone, and includes the first cone, that is, It is arranged so as to cover the entire field of view of the camera. That is, an area wider than the visual field area of the camera is taken as the investigation area, and the position of the target outside the visual field area of the camera is acquired and included in the display image.

  With such a configuration, the display device of the present invention defines the survey area with the second cone that includes the first cone that defines the field of view of the camera, and the position of the target in the survey area. To get. As a result, the position of the target outside the field of view of the camera is also displayed in the display image, and the position of the target is out of the field of view of the actual camera, as if the field of view was taken wider than the actual field of view. The user can also recognize the position of a certain target.

In the display device of the present invention,
The field of view is defined by a cone,
The survey area is defined by a quadric surface,
The axis of the cone and one of the axes of the quadric surface can be configured to overlap.

  That is, the camera field-of-view region is defined by a cone (three-dimensional solid such as a quadrangular pyramid or a cone) having a vertex at the camera position and a straight line extending in the direction of the camera as described above. On the other hand, the investigation area from which the position of the target is acquired is not determined as the second cone as described above, but is set to one of the axes common to the cone that defines the field of view of the camera. Determined by the quadric surface.

  Here, the quadric surface is a curved surface in a three-dimensional space such as a spherical surface, an ellipsoid, a paraboloid, a hyperboloid, etc., and any of the axes of the quadric surface is, for example, an ellipsoid It means one of the three symmetry axes, or the central axis if it is a paraboloid. The position of the target in the investigation area defined by such a quadric surface is acquired by the acquisition unit, and a display image on which a mark representing the acquired position of the target is drawn is generated.

  With such a configuration, the display device of the present invention takes an area defined by a quadric surface as an investigation area for acquiring the position of the target. Thereby, the position of the target to be displayed as a display image can be acquired more flexibly than, for example, the survey area defined by the second cone.

  In order to achieve the above object, a display method according to a second aspect of the present invention is a display method executed by a display device including a detection unit, an acquisition unit, a generation unit, and a display unit, and includes a detection step, an acquisition step, A generation process and a display process are provided.

In the detection step, the detection unit detects the position and orientation of the camera.
In the acquisition step, the acquisition unit acquires the position of the target in the investigation area determined by the detected position and orientation of the camera.
In the generation step, the generation unit draws a captured image obtained by the camera capturing a visual field region determined by the position and orientation of the camera in a photographic region in the projection plane, and the position of the acquired target is predetermined A display image is generated by drawing a mark representing the target object at a position projected in the report area in the projection plane by the projection conversion.
In the display step, the display unit displays the generated display image on the screen.
The report area includes a photographic area.
The investigation area has a portion that is not included in the visual field area.

  In order to achieve the above object, a program according to a third aspect of the present invention is configured to cause a computer to function as the above display device and to cause the computer to execute each step of the above display method.

  The program of the present invention can be recorded on a computer-readable information storage medium such as a compact disk, flexible disk, hard disk, magneto-optical disk, digital video disk, magnetic tape, and semiconductor memory.

  The program can be distributed and sold via a computer communication network independently of the computer on which the program is executed. The information storage medium can be distributed and sold independently from the computer.

  According to the present invention, it is possible to provide a display device, a display method, and a program suitable for effectively displaying the position of a target outside the visual field of the camera on the screen.

It is a figure which shows schematic structure of the typical information processing apparatus with which the display apparatus of this invention is implement | achieved. (A), (b) is the external view of the typical information processing apparatus with which the display apparatus of this invention is implement | achieved. It is a figure which shows the function structure of the display apparatus of this invention. It is a figure which shows a mode that the user with a display apparatus exists in the urban area of a real world. It is a figure which shows the visual field area | region of a camera. In Embodiment 1, it is a figure which shows the investigation area | region containing the visual field area | region of a camera. In Embodiment 1, it is a figure which shows a mode that the position in the visual field area | region of a camera and the position of the target object in an investigation area | region are projected on a virtual projection surface. In Embodiment 1, it is a figure which shows a display image. It is a flowchart which shows the flow of a process of the display apparatus which concerns on embodiment of this invention. In Embodiment 2, it is a figure which shows the investigation area | region defined by a paraboloid. In Embodiment 2, it is a figure which shows a mode that the position of the target object in an investigation area | region is projected on a virtual projection surface. In Embodiment 3, it is a figure which shows the investigation area | region defined by the paraboloid which includes a user.

  Embodiments of the present invention will be described below. In the following, for ease of understanding, an embodiment in which the present invention is realized using an information processing apparatus of a portable game machine type will be described, but the embodiment described below is for explanation, It is not intended to limit the scope of the present invention. Therefore, those skilled in the art can employ embodiments in which these elements are replaced with equivalent ones, and these embodiments are also included in the scope of the present invention.

  Examples of the information processing apparatus in which the display device according to the present invention can be realized include various other information processing apparatuses such as a portable game machine, a portable phone, an electronic device such as a portable camera and a smartphone. .

(Embodiment 1)
FIG. 1 is a schematic diagram showing a schematic configuration of a typical information processing apparatus in which a display device according to an embodiment of the present invention is realized. Hereinafter, a description will be given with reference to FIG.

  The information processing apparatus 1 includes a processing control unit 10, a connector 11, a cartridge 12, a wireless communication unit 13, a communication controller 14, a sound amplifier 15, a speaker 16, an operation key 17, a GPS processing unit 18, a camera 19, and a first display unit. 20, a second display unit 21, and a touch panel 22.

  The processing control unit 10 includes a CPU (Central Processing Unit) core 10a, an image processing unit 10b, a VRAM (Video Random Access Memory) 10c, a WRAM (Work RAM) 10d, an LCD (Liquid Crystal Display) controller 10e, A touch panel controller 10f.

  The CPU core 10a controls the overall operation of the information processing apparatus 1, is connected to each component, and exchanges control signals and data. Specifically, a program and data stored in a ROM (Read Only Memory) 12a in the cartridge 12 are read out with the cartridge 12 mounted in the connector 11, and predetermined processing is executed.

  The image processing unit 10b processes the data read from the ROM 12a in the cartridge 12 and the data processed by the CPU core 10a, and stores the processed data in the VRAM 10c.

  The VRAM 10c is a frame memory that stores information for display, and stores image information processed by the image processing unit 10b and the like.

  The WRAM 10d stores work data and the like necessary for the CPU core 10a to execute various processes according to the program.

  The LCD controller 10e controls the image display unit 20 to display a predetermined display image. For example, the LCD controller 10 e converts the image information stored in the VRAM 10 c into a display signal at a predetermined synchronization timing and outputs the display signal to the image display unit 20. The LCD controller 10e displays a predetermined instruction icon or the like on the image display unit 20.

  The touch panel controller 10f detects a touch (touch) on the touch panel 22 by a touch pen or a user's finger. For example, in a state where a predetermined instruction icon or the like is displayed on the image display unit 20, detection of contact or release (leaving) on the touch panel 22 and their positions are detected.

  The connector 11 is a terminal that can be detachably connected to the cartridge 12, and transmits / receives predetermined data to / from the cartridge 12 when the cartridge 12 is connected.

The cartridge 12 includes a ROM (Read Only Memory) 12a and a RAM (Random Access Memory) 12b.
The ROM 12a stores a program for realizing the game and image data, sound data, and the like accompanying the game.
The RAM 12b stores various data indicating the progress of the game.

  The wireless communication unit 13 is a unit that performs wireless communication with a wireless communication unit of another information processing apparatus, and transmits / receives predetermined data via an antenna (not shown) (built-in antenna or the like). The wireless communication unit 13 can also perform wireless LAN communication with a predetermined access point. The wireless communication unit 13 is assigned a unique MAC (Media Access Control) address.

  The communication controller 14 controls the wireless communication unit 13 and mediates communication performed between the processing control unit 10 and the processing control unit of another information processing apparatus according to a predetermined protocol. Further, when the information processing apparatus 1 is connected to the Internet via a nearby wireless access point or the like, the processing control unit 10 and the wireless access point or the like are connected according to a protocol compliant with a wireless LAN (Local Area Network) Intermediary of wireless communication performed in

  The sound amplifier 15 amplifies the audio signal generated by the processing control unit 10 and supplies it to the speaker 16. The speaker 16 is composed of, for example, a stereo speaker and outputs predetermined sound effects, music sounds, and the like according to the audio signal amplified by the sound amplifier 15.

  The operation key 17 is configured by a key switch or the like appropriately disposed in the information processing apparatus 1 and receives a predetermined instruction input according to a user operation. The operation keys 17 include buttons for adjusting the volume, knobs, and the like. Each operation key 17 is provided with a pressure sensor and can detect which key is pressed. The user inputs various operation instructions to the information processing apparatus 1 by pressing such operation keys 17.

  The GPS processing unit 18 exchanges information with a GPS (Global Positioning System) satellite, and acquires information on position coordinates (for example, latitude / longitude and altitude) of the information processing apparatus 1. The GPS processing unit 18 includes an azimuth sensor such as a compass, and acquires information on the orientation of the information processing apparatus 1 (for example, an azimuth angle with reference to east, west, south, and north). Further, the GPS processing unit 18 detects the direction of gravity using a gravity sensor or the like, and acquires inclination information such as how much the information processing apparatus 1 is inclined with respect to the direction of gravity.

  The camera 19 is incorporated in the upper chassis of the information processing apparatus 1 and photographs the back direction of the first display unit 20. For example, the camera 19 captures a landscape image to be displayed on the first display unit 20. As an example, the camera 19 includes a single-focus lens and an image sensor (CMOS, CCD, etc.) having a predetermined number of pixels, and the optical axis direction is determined at a predetermined focal length (view angle α). Take a central image.

  The 1st display part 20 and the 2nd display part 21 consist of LCD etc., and display image data suitably by control of LCD controller 10e. In addition, the second display unit 21 appropriately displays selection buttons (icons) and the like necessary for the user or the like to touch the touch panel 22 and input a selection instruction.

  The touch panel 22 is disposed so as to be superimposed on the front surface of the second display unit 21 and detects an input by a touch pen or a player's finger. For example, the touch panel 22 includes a resistive film type touch sensor panel or the like, detects a press (pressing) with a touch pen or the like, and outputs information (signals or the like) corresponding to the coordinates. Alternatively, the touch panel 22 may be a pressure-sensitive touch sensor panel and may detect pressure such as a touch pen.

  FIG. 2 is a schematic diagram showing an external appearance of a typical information processing apparatus 1 in which the display device according to the embodiment of the present invention is realized. As an example, the information processing apparatus 1 is a portable game machine, and each part is incorporated in an upper chassis Js and a lower chassis Ks as shown in FIG. The upper chassis Js and the lower chassis Ks are pivotally supported at the joint, and can be opened and closed like a normal notebook type personal computer.

  For example, the first display unit 20 and the like are incorporated in the upper chassis Js. On the other hand, the second display unit 21, the touch panel 22, and the like are incorporated in the lower chassis Ks. Further, as shown in FIG. 2B, a camera 19 is incorporated on the back side of the upper chassis Js. The camera 19 photographs the back direction of the first display unit 20 with the upper chassis Js opened. As an example, the camera 19 includes a single-focus lens, and can capture a range of an angle of view α that is somewhat wider than a human viewing angle (viewing angle when gazing).

  FIG. 3 is a diagram showing a functional configuration of the display device of the present invention realized by using the information processing device 1. The display device 300 includes a detection unit 301, an acquisition unit 302, a generation unit 303, and a display unit 304.

  Note that the display device 300 may include a storage unit and the like as appropriate. Here, the storage unit is realized by functions such as various RAMs, for example, and stores various data necessary for the operation of the display device, such as the current time, the contents of input performed by the user, and the input time.

  The detection unit 301 detects the position and orientation of the camera 19. That is, the detection unit 301 detects the position of the camera 19 in the real world by exchanging with the GPS satellite, and further detects the direction of the camera 19 at the detected position by using the function of the direction sensor. To do. Then, the detection unit 301 supplies the detected position and orientation of the camera 19 to the acquisition unit 302. Such a detection part 301 is implement | achieved when each part, such as the GPS process part 18 and WRAM 10d, cooperates under control of CPU core 10a, for example.

  The acquisition unit 302 acquires the position of the target in the survey area determined by the detected position and orientation of the camera 19. In other words, the acquisition unit 302 receives information on the position and orientation of the camera 19 detected by the detection unit 301, and acquires the position of the target in the three-dimensional survey region determined by the position and orientation. Then, the acquisition unit 302 supplies the acquired position of the target to the generation unit 303. Such an acquisition unit 302 is realized, for example, when the CPU core 10a cooperates with each unit such as the WRAM 10d.

  The generation unit 303 draws a captured image obtained by the camera 19 capturing a visual field region determined by the position and orientation of the camera 19 in a photographic region within the projection plane, and the acquired target position is a predetermined value. A display image is generated by drawing a mark representing the target object at a position projected in the report area within the projection plane by projection conversion. In other words, the generation unit 303 receives the position of the target acquired by the acquisition unit 302, and projects the position of the target on the projection plane together with the captured image by the camera 19 by a predetermined projection transformation. A display image is generated by combining the image and a mark representing the target. Then, the generation unit 303 supplies the generated display image to the display unit 304. Such a generation unit 303 is realized, for example, by the cooperation of the units such as the image processing unit 10b and the VRAM 10c under the control of the CPU core 10a.

  The display unit 304 displays the generated display image on the screen. That is, the display unit 304 receives the display image generated by the generation unit 303 and displays the display image on the first display unit 20. Such a display unit 304 is realized, for example, by the cooperation of each unit such as the LCD controller 10e and the VRAM 10c under the control of the CPU core 10a.

  Hereinafter, how the display device 300 acquires the position of the target and displays the position on the screen will be specifically described. Here, specifically, it is assumed that the target is a game center equipped with a game machine that the user wants to play. That is, a case where the user wants to play a certain game and obtains information about where the game center where the game machine is installed by using the display device 300 will be described below.

  FIG. 4 is a diagram illustrating a state in which a user having the display device 300 is in a real world city area. In this urban area, a plurality of buildings are set up facing the central street, and there are trees and cars beside the buildings. The user 400 is standing at the center position 401 of the street and holds the display device 300 so that the camera 19 of the display device 300 faces in the direction 402 at the back of the street. The position 401 of the user 400 is detected as the position 401 of the display device 300 held by the user 400 by the function of the GPS satellite 420 orbiting the sky.

  Among the buildings facing the street, the buildings 405a and 405b have game centers in which game machines that the user 400 wants to play are installed, that is, the targets 410a and 410b. On the other hand, it is assumed that the user 400 does not know in which building the game center where the game can be played, that is, the target objects 410a and 410b are located.

  At this time, when the user 400 wants to know the position of the target 410 and inputs an instruction to that effect to the display device 300 using an input device such as the touch panel 22, first, the detection unit 301 of the display device 300 The position 401 and the direction 402 of the camera 19 are detected by the function of the GPS processing unit 18. Specifically, the detection unit 301 exchanges information with the GPS satellite 420 and detects the current position 401 of the camera 19. In addition, the detection unit 301 uses the azimuth sensor and the gravity sensor provided in the GPS processing unit 18 to determine the direction 402 of the camera 19, that is, the direction in which the display device 300 is currently directed, and the degree of inclination from the direction of gravity. Is detected. At this time, the detection unit 301 acquires latitude / longitude / altitude information as the position 401 of the camera 19, and as the orientation 402 of the camera 19, the azimuth angle information with respect to east, west, south, and north and the elevation angle with respect to the horizontal plane. Get information.

  FIG. 5 is a diagram showing the field of view of the camera 19 defined by the detected position 401 and orientation 402 of the camera 19 when viewed from directly above the user 400. The field of view 500 of the camera 19 provided in the display device 300 held by the user 400 is formed by a line of sight that extends radially from the position 401 of the camera 19 to the direction 402 of the camera 19 in the range of the angle of view α of the camera 19. That is, as shown by the hatched portion in FIG. 5, the visual field region 500 has a vertical line corresponding to the angle of view α of the camera 19 with a straight line extending from the position 401 of the camera 19 as the apex to the direction 402 of the camera 19 as an axis. It is defined by a square pyramid 510 having a corner.

  At this time, of the two targets 410a and 410b existing around the user 400, the target 410a in the building 405a is in the visual field area 500, but the target 410b in the building 405b is It does not enter the visual field area 500. That is, since the user 400 points the camera 19 in the back direction 402 of the street, the target object 410a located in the back of the street is within the field-of-view area 500, but is immediately to the left of the user 400. The target object 410b has deviated from the visual field region 500.

  For this reason, since the orientation of the camera 19 is not directed so that the target object 410b enters the visual field region 500 even though it is closer to the user 400, the target object 410b is included in the captured image by the camera 19. The building 405b is not projected, and the user 400 can know only the position of the target object 410a farther away from the building displayed in the captured image. In view of this, the display apparatus 300 of the present embodiment avoids overlooking the target 410b that is closer to the display device 300 as much as possible, even if the target 410 is out of the field of view 500 of the camera 19. It can be displayed and provided to the user 400.

  FIG. 6 is a diagram illustrating a survey area including the visual field area 500 of the camera 19 when viewed from directly above the user 400. Similar to the quadrangular pyramid 510 that defines the field-of-view area 500 of the camera 19, the survey area 600 is another axis that includes the quadrangular pyramid 510 with the position 401 of the camera 19 as the axis and a straight line extending in the direction 402 of the camera 19. It is defined by a quadrangular pyramid 610. That is, like the hatched portion in FIG. 6, the survey region 600 becomes a region in a three-dimensional space defined by the second quadrangular pyramid 610 including the first quadrangular pyramid 510 that defines the visual field region 500, It becomes an area as if the angle of view of the camera 19 is further expanded with respect to the visual field area 500 of the camera 19 defined by the angle of view α.

  The acquisition unit 302 of the display device 300 acquires not only the position of the target object 410a in the visual field area 500 but also the position of the target object 410b in the survey area 600 extending to the outside of the visual field area 500 of the camera 19 as described above. To do. That is, when the detection unit 301 detects the position 401 and orientation 402 of the camera 19, the acquisition unit 302 obtains the space occupied by the survey area 600 defined by the detected position 401 and orientation 402 of the camera 19. It is determined whether or not the position of the target 410 is within the survey area 600, and the position of the target 410 that is determined to be within is acquired.

  Specifically, in FIG. 6, the target object 410 b on the left side of the user 400 is out of the visual field area 500 of the camera 19, but is within the survey area 600, and thus the target object in the visual field area 500. The position is acquired by the acquisition unit 302 together with 410a.

  Information on the position of each target 410 used for determining whether or not it is within the survey area 600 is stored in advance in various storage units in the display device 300 and read to the acquisition unit 302 as necessary. For example, for the user 400 who wants to play the game A, the location information of the game center where the game machine of the game A is installed is played, and the game B is played. The user 400 who wants to acquire information on the location of the game center where the game machine of the game B is installed may be acquired through the wireless communication unit 13 as necessary.

  When the position of the target 410 in the survey area 600 is acquired in this way, the generation unit 303 of the display device 300 next displays a display image for displaying the acquired position of the target 410 as a camera. 19 is generated in correspondence with the photographed image.

  FIG. 7 is a diagram showing a state in which the position of the camera 19 in the visual field area 500 and the position of the target in the investigation area 600 are projected on a virtual projection plane. FIG. 7 shows a state when the visual field region 500 of the camera 19 and the survey region 600 including the same are viewed from directly above the user 400, as in FIG.

  In order to generate a display image, the generation unit 303 prepares a virtual projection plane 700 in a storage unit such as the VRAM 10c. Specifically, as illustrated in FIG. 7, the generation unit 303 performs a three-dimensional operation in which the position 401 of the user 400 (camera 19) is the origin (= (0, 0, 0)) and the direction 402 is the z-axis. Taking the coordinates, the projection plane 700 is taken at a position away from the position 401 by a distance a in the z direction, that is, at a position of z = a. In the projection plane 700, a photographic area 701 where an image taken by the camera 19 is drawn, and a report area 702 where a mark representing the acquired target object 410 is drawn so as to include the photographic area 701, Is prepared.

  Here, the photograph area 701 is a two-dimensional area for drawing an image captured by the camera 19 (so-called photograph). The generation unit 303 draws an image captured by the camera 19 by projecting the position of a building, a tree, or the like stored in the field of view 500 of the camera 19 onto the photograph area 701. That is, the generation unit 303 projects a line of sight that extends radially from the position 401 of the camera 19 that is the origin in the range of the angle of view α, and projects a position in the visual field 500 onto the projection plane 700 along the projection line. To do. More specifically, using coordinates, an arbitrary position in the field of view 500 whose three-dimensional coordinates are represented by (x, y, z) is xy coordinates (x It is projected at a position represented by (a / z), y (a / z)).

  On the other hand, the report area 702 is a two-dimensional area for reporting the position of the target 410 located in the survey area 600 to the user 400 by drawing a corresponding mark, and includes the photo area 701. In addition, a wider two-dimensional region in the projection plane 700 is taken. The generation unit 303 projects the positions of the targets 410a and 410b acquired by the acquisition unit 302 on the report region 702 using the same projection transformation as that for projecting a building, a tree, or the like in the visual field region 500. . That is, the positions of the targets 410a and 410b are projected onto the report area 702 in the projection plane 700 along a projection line that extends radially from the position 401 of the camera 19 that is the origin.

  Specifically, the target 410a at the position where the three-dimensional coordinates are represented by (x1, y1, z1) has the xy coordinates of (x1 (a / z1), y1 (a / z1) along the projection line 705a. )) And projected onto a position in the projection plane 700. Similarly, the target 410b whose three-dimensional coordinates are represented by (x2, y2, z2) has xy coordinates (x2 (a / z2), y2 (a / z2)) along the projection line 705b. ) Is projected to a position in the projection plane 700 represented by

  Then, the generation unit 303 draws marks 710a and 710b representing the targets 410a and 410b at the positions of the projected targets 410a and 410b, and matches the captured image drawn in the photo area 701 with the display image. Generate.

  FIG. 8 is a diagram showing the generated display image. As shown in FIG. 8, the display image 800 includes an image 810 captured by the camera 19 drawn in a photographic area 701 in the projection plane 700 and targets 410 a and 410 b drawn in a report area 702 in the projection plane 700. Mark 710a, 710b (specifically, a star-shaped mark in FIG. 8). The display unit 304 of the display device 300 displays such a display image 800 on the screen of the first display unit 20 or the like.

  That is, in the display image 800, a building, a tree, a car, and the like housed in the field-of-view area 500 of the camera 19 are drawn as a captured image 810 as if viewed from the position 401 of the user 400, and there When the marks 710a and 710b representing the objects 410a and 410b are drawn so as to overlap each other, the user 400 viewing the display image 800 recognizes where the target objects 410a and 410b are based on the captured image 810. Will be able to.

  Specifically, as shown in FIG. 8, the mark 710a representing the target object 410a located in the visual field area 500 of the camera 19 is a position in the captured image 810 where the building 405a containing the target object 410a is drawn. Drawn on. On the other hand, the mark 710b representing the target object 410b located outside the visual field area 500 of the camera 19 is not drawn in the captured image 810 because the building 405b containing the target object 410b is not drawn in the captured image 810. Is not drawn, and is drawn at a position deviated to the left from the range of the captured image 810.

  Thereby, the user 400 can know that the target 410a located in the visual field area 500 of the camera 19 is in the captured building 405a while viewing the captured image 810. Further, the user 400 cannot know the target object 410 b located outside the visual field area 500 of the camera 19 until it is in any building, but which area is based on the current position 401 and the direction 402. You can find out if there is. Therefore, the user 400 can obtain the target 410b at a closer position without missing the position.

  That is, the user 400 who is looking for a game center that can play a desired game misses information on the position of a closer game center (target 410b) because the camera 19 happens to be pointed away. Without having to go to a distant game center (target 410a).

  FIG. 9 is a flowchart showing a flow of processing according to the display device 300 of the present embodiment. The processing flow realized in the present embodiment that has been described so far will be summarized below.

  When the processing of the display device 300 is started, after performing various initialization processes, the display device 300 first determines whether or not it has received an instruction from the user 400 to display the position of the target 410 (step S901). That is, the display device 300 determines whether or not an instruction for displaying the position of the target 410 from the user 400 who wants to know the position of the target 410 is input via the input device such as the touch panel 22. judge.

  When the instruction from the user 400 has not been received (step S901; NO), the process remains in step S901 and waits until the instruction from the user 400 is received.

  On the other hand, when an instruction from the user 400 is received (step S901; YES), the detection unit 301 detects the position 401 and the orientation 402 of the camera 19 (step S902). That is, the detection unit 301 obtains the current position 401 of the camera 19 by communicating with the GPS satellite 420, and obtains the current direction 402 of the camera 19 by an orientation sensor.

  When the position 401 and the direction 402 of the camera 19 are detected, the acquisition unit 302 then acquires the position of the target 410 within the investigation area 600 (step S903).

  That is, the acquisition unit 302 obtains the survey area 600 defined by the detected current position 401 and orientation 402 of the camera 19 and acquires the position of the target object 410 located in these. Specifically, as shown in FIG. 6, the survey area 600 is defined as a second quadrangular pyramid 610 that includes a quadrangular pyramid 510 that defines the visual field area 500 of the camera 19. Then, the position of the target 410 located in the survey area 600 is acquired.

  When the position of the target object 410 is acquired, the generation unit 303 then projects the visual field region 500 of the camera 19 (step S904), and further projects the acquired position of the target object 410 on the projection plane 700. (Step S905), the display image 800 is generated.

  That is, as illustrated in FIG. 7, the generation unit 303 projects the position in the visual field region 500 of the camera 19 onto the photographic region 701 in the projection plane 700 to draw the photographed image 810, and performs the same projection conversion. Using the acquired position of the target object 410 in the survey area 600, the position is projected onto the report area 702 in the projection plane 700. Then, a mark 710 representing the target 410 is drawn on the projected position so as to overlap the captured image 810, thereby generating a display image 800 as shown in FIG.

  When the display image 800 is generated, the display unit 304 waits for a predetermined synchronization timing at which the display image 800 should be output on the screen, and displays the generated display image 800 on the screen (step S906). . That is, the display unit 304 provides the user 400 with information on the position of the target 410 by displaying the generated display image 800 on the screen of the first display unit 20 or the like.

  Thereafter, the process returns to step S901, and waits until an instruction to display the position of the target object 410 is received from the user 400 again.

  With such a configuration, the display device 300 according to the present embodiment includes the field-of-view area 500 of the camera 19 using the same projection transformation as that projected from the field-of-view area 500 of the camera 19 together with the captured image 810 of the camera 19. The position of the target 410 in the survey area 600 to be displayed is displayed in the display image 800. As a result, since the position of the target 410 within the field of view of the camera 19 is displayed superimposed on the captured image 810 of the camera 19, the position of the target 410 is determined by the user corresponding to the captured building or the like. 400, and the user 410 can recognize the position of the target 410 that is out of the field of view of the camera 19 based on the captured image 810 of the camera.

(Embodiment 2)
Next, Embodiment 2 of the present invention will be described. In the display device 300 according to the first embodiment, the survey area 600 in which the position of the target 410 is acquired is determined by the second quadrangular pyramid 610 including the first quadrangular pyramid 510 that defines the visual field area 500 of the camera 19. It was. On the other hand, the display device 300 according to the present embodiment defines the investigation region by a paraboloid having a common axis with the quadrangular pyramid 510 that defines the visual field region 500 of the camera 19. This will be described below.

  FIG. 10 is a diagram showing a survey area defined by a paraboloid in the present embodiment. In FIG. 10, as in FIG. 6 in the first embodiment, the user 400 in the urban area of the real world points the camera 19 of the display device 300 from the middle position 401 in the street to the back direction 402 in the street. It shows how it is. As in FIG. 6, the field-of-view area 500 of the camera 19 is defined by a quadrangular pyramid 510 with the position 401 of the camera 19 as a vertex and a straight line extending in the direction 402 as an axis.

  On the other hand, the survey area 1000 in which the acquisition unit 302 should acquire the position of the target object 410 in the present embodiment is not the second square pyramid 610 as shown in FIG. It is defined by a paraboloid 1010 that is a quadric surface. Specifically, this paraboloid 1010 has the position 401 of the camera 19 as the apex, similarly to the quadrangular pyramid 510 that defines the visual field region 500, and extends in the same axis as the quadrangular pyramid 510 that defines the visual field region 500, that is, the direction 402 of the camera 19. It becomes a paraboloid of revolution with a straight line as its axis of rotation.

  When the detection unit 301 detects the position 401 and the direction 402 of the camera 19, the acquisition unit 302 obtains a three-dimensional space occupied by the survey region 1000 defined by the paraboloid 1010, and obtains each target 410. It is determined whether or not the position is within the survey area 1000, and the position of the target 410 that is determined to be within is acquired.

  Specifically, in FIG. 10, the target object 410 b on the left side of the user 400 is out of the visual field area 500 of the camera 19, but is within the investigation area 1000 defined by the paraboloid 1010. The position is acquired by the acquisition unit 302 together with the target 410a in the visual field region 500.

  By using the paraboloid 1010 for the investigation region 1000, the investigation region 1000 is more sufficiently wider than the visual field region 500, particularly in the vicinity of the position 401 of the camera 19, as compared with the case where the second square pyramid 610 is used. Will take up space. As a result, the display device 300 according to the present embodiment acquires the position of the target object 410 that is out of the visual field area 500 of the camera 19 even though it is near the user 400 like the target object 410b. It is possible to more effectively prevent being missed.

  When the position of the target 410 within the survey area 1000 defined by the paraboloid 1010 is acquired in this way, the generation unit 303 of the display device 300 displays the acquired position of the target 410 next. A display image for this purpose is generated in correspondence with an image captured by the camera 19.

  FIG. 11 is a diagram illustrating a state in which the position of the target in the survey area 1000 is projected onto the virtual projection plane 700. FIG. 11 shows a state when the visual field region 500 and the survey region 1000 are viewed from directly above the user 400, as in FIG.

  In order to generate the display image, the generation unit 303 sets the position 401 of the user 400 (camera 19) as the origin (= (0, 0, 0)) and sets the direction of the direction 402 as in FIG. Taking a three-dimensional coordinate as the z axis, a virtual projection plane 700 is taken at a position a distance a from the position 401 in the z direction by a distance a, that is, a position of z = a. In the projection plane 700, a photographic area 701 where an image taken by the camera 19 is drawn, a report area 702 where a mark representing the acquired target 410 is drawn so as to include the photographic area 701, Prepare.

  Here, in the photographic area 701 in the projection plane 700, as in the first embodiment, the position of a building, a tree, or the like housed in the visual field area 500 is projected along the line of sight of the camera 19. The photographed image by 19 is drawn. On the other hand, in the report area 702 in the projection plane 700, the position of the target object 410 located in the survey area 1000 is projected, and a mark representing the target object 410 is drawn at the projected position. At the same time, in the first embodiment, the same projection transformation as that for drawing a captured image by the camera 19 is used, whereas in this embodiment, the projection transformation using a parabola as a projection line is used.

  Specifically, as shown in FIG. 11, a parabola whose axis is a straight line extending in the direction 402 of the camera 19 is a projection line. Then, an arbitrary position in the survey area 1000 is projected onto the projection plane 700 at the position z = a along this projection line. More specifically, using coordinates, an arbitrary position in the investigation area 1000 whose three-dimensional coordinates are represented by (x, y, z) is xy coordinates on the projection plane 700 with z = a (x Projected to a position represented by √ (a / z), y√ (a / z)).

  In other words, the target 410a whose three-dimensional coordinates are represented by (x1, y1, z1) has xy coordinates (x1√ (a / z1), y1√ (a / z1) along the projection line 1105a. )) And projected onto a position in the projection plane 700. Similarly, the target 410b whose three-dimensional coordinates are represented by (x2, y2, z2) has xy coordinates (x2√ (a / z2), y2√ (a /) along the projection line 1105b. Projected at a position in the projection plane 700 represented by z2)).

  Then, the generation unit 303 draws marks 710a and 710b representing the targets 410a and 410b at the positions of the projected targets 410a and 410b, and matches the captured image drawn in the photo area 701 with the display image. Generate. The generated display image is displayed on the screen of the display device 300 by the display unit 304. Thereby, the user 400 can recognize which position the target objects 410a and 410b are on the basis of the image captured by the camera 19.

  In addition, the position of the target 410 is projected onto the projection plane 700 by the projection conversion using the parabola as the projection line in this way, so that the projected position of the target 410 is projected with the line of sight of the camera 19 as the projection line. Deviation occurs with respect to the drawing position of the captured image. That is, for example, the target object 410a in the visual field 500 of the camera 19 is projected to a position shifted from the position of the building 405a in the drawn photographed image even though it is actually in the building 405a. become. However, since the parabola used as the projection line is close to a straight line near the center of the photographed image, the deviation between the position of the projected target 410 and the drawing position of the photographed image becomes small. A certain error may also be included in the position 401 and orientation 402 of the camera 19 detected by the above, and the position of the target 410 acquired by the acquisition unit 302. Therefore, the display device 300 according to the present embodiment can provide the position of the target 410 without causing the user 400 to feel a great difference between the position of the projected target 410 and the drawing position of the captured image.

  With such a configuration, the display device 300 according to the present embodiment takes an area defined by the paraboloid 1010 as the survey area 1000 for acquiring the position of the target 410. Then, the acquired position of the target 410 is projected onto the projection plane 700 using projection transformation with a parabola as a projection line, and is displayed on the screen as a display image together with an image captured by the camera 19. As a result, particularly in the vicinity of the position 401 of the camera 19, the survey area 1000 for acquiring the position of the target 410 can be widened, and it is effective to acquire and miss the position of the target 410 near the user 400. The position information can be provided to the user 400 while preventing it.

(Embodiment 3)
Next, a third embodiment of the present invention will be described. In the display device 300 according to the second embodiment, the survey area 1000 where the position of the target 410 is acquired is defined by the paraboloid 1010 having the position 401 of the user 400 (camera 19) as a vertex. On the other hand, the display device 300 according to the present embodiment defines the survey area by a paraboloid that includes the user 400 with the position behind the user 400 as a vertex. This will be described below.

  FIG. 12 is a diagram illustrating a survey area defined by a paraboloid that includes the user 400. In FIG. 12, as in FIG. 10 in the second embodiment, the user 400 in the urban area of the real world directs the camera 19 of the display device 300 from the middle position 401 of the street to the back direction 402 of the street. It shows how it is. The field-of-view area 500 of the camera 19 at this time is defined by a quadrangular pyramid 510 with the position 401 of the camera 19 as a vertex and a straight line extending in the direction 402 as an axis, as in FIG.

  On the other hand, the survey area 1200 in which the acquisition unit 302 should acquire the position of the target 410 in the present embodiment is defined by a paraboloid 1210 that includes the user 400 as shown by the hatched portion in FIG. . Specifically, the paraboloid 1210 extends from the position of the user 400 (camera 19) at a distance b to the apex, and extends in the same axis as the quadrangular pyramid 510 that defines the visual field 500, that is, the direction 402 of the camera 19. It becomes a paraboloid of revolution with a straight line as its axis of rotation.

  In this way, by taking the survey area 1200 to the rear area with respect to the direction 402 of the camera 19, the position of the target object 410 positioned not only in front of the camera 19 but also in the rear is acquired. Specifically, in FIG. 12, the target object 410 c in the building 405 c is behind the right side of the user 400 and is out of the visual field area 500 of the camera 19, but in the investigation area 1200 defined by the paraboloid 1210. Therefore, the position is acquired by the acquisition unit 302 together with the other targets 410a and 410b.

  The acquired positions of the target objects 410a, 410b, and 410c are projected onto a virtual projection plane along the parabolic projection line, as in the second embodiment. The projected position will be specifically described using coordinates. Similarly to FIG. 11 in the second embodiment, the position 401 of the user 400 (camera 19) is the origin (= (0, 0, 0)), and the direction 402 If the virtual projection plane is taken at a position a distance a from the position 401 in the z direction, the three-dimensional coordinates are represented by (x, y, z). An arbitrary position in the survey area 1200 is projected on the projection plane at a position where the xy coordinates are represented by (x√ (a + b) / √ (z + b), y√ (a + b) / √ (z + b)). It will be.

  Then, marks representing the target objects 410a, 410b, and 410c are drawn at the positions of the target objects 410a, 410b, and 410c projected on the projection plane, and are displayed on the screen as a display image together with the images captured by the camera 19. Thereby, the user 400 can obtain information on which position the target objects 410a, 410b, and 410c are located with reference to an image captured by the camera 19.

  With such a configuration, the display device 300 according to the present embodiment defines the paraboloid 1210 that includes the user 400 as the survey area 1200 for acquiring the position of the target 410. As a result, particularly in the vicinity of the position 401 of the camera 19, it is possible to take the survey area 1200 for acquiring the position of the target object 410 to the rear of the user 400. Can be provided to the user 400.

(Other embodiments)
As mentioned above, although embodiment of this invention was described, this invention is not limited to embodiment mentioned above, A various deformation | transformation and application are possible. Moreover, it is also possible to freely combine the constituent elements of the above-described embodiments.

  For example, in the above-described embodiment, the target 410 acquired by the acquisition unit 302 has been described as a game center provided with a game machine that the user wants to play. For example, a convenience store, a vending machine, a bus stop, or the like may be used as long as it is a building, facility, equipment, or the like that you want to know.

  In the above embodiment, the detection unit 301 of the display device 300 acquires latitude / longitude / altitude information as the position 401 of the camera 19, and azimuth angle information with reference to east, west, south, and north as the direction 402 of the camera 19. And the elevation angle information with reference to the horizontal plane was acquired. However, the present invention is not limited to this, and the position 401 of the camera 19 may be acquired by a coordinate system other than latitude and longitude, or the direction 402 of the camera 19 may be acquired by information other than the azimuth angle with reference to east, west, south, and north. .

  In addition, the detection unit 301 does not acquire vertical information, that is, altitude and elevation angle, as the position 401 and orientation 402 of the camera 19, and detects only horizontal information, that is, latitude / longitude and azimuth. Also good. By omitting the information in the vertical direction from the information to be detected, the accuracy in the vertical direction of the position of the target 410 displayed in the display image, that is, the information on how high the target 410 is located is lowered. However, in order for the user 400 to know the position of the target 410, there are many cases where it is only necessary to recognize the position information in the horizontal direction. Therefore, by omitting the information in the vertical direction from the information to be detected, the position of the target object 410 outside the camera can be provided to the user 400 with the simplified configuration of the display device 300.

  In the second and third embodiments, the survey areas 1000 and 1200 in which the position of the target 410 is acquired have a paraboloid of revolution having a common axis with the quadrangular pyramid 510 that defines the visual field area 500 of the camera 19. 1010, 1210. However, the investigation area is not limited to the paraboloid of revolution, and may be defined by other quadric surfaces. For example, it is not limited to a paraboloid, but may be defined by a spherical surface, an elliptical surface, a hyperboloid, a conical surface, or even a paraboloid, not limited to a rotationally symmetric one such as a rotating paraboloid, It may be determined by a three-dimensional shape such as a parabolic column surface obtained by extending a parabola in a columnar shape in the vertical direction. The display device 300 of the present invention can flexibly design such various quadric surfaces according to the region where the position of the target 410 is desired to be acquired.

  As described above, according to the present invention, it is possible to provide a display device, a display method, and a program suitable for effectively displaying the position of a target outside the field of view of the camera on the screen. it can.

DESCRIPTION OF SYMBOLS 1 Information processing apparatus 10 Processing control part 10a CPU core 10b Image processing part 10c VRAM
10d WRAM
10e LCD controller 10f Touch panel controller 11 Connector 12 Cartridge 12a ROM
12b RAM
DESCRIPTION OF SYMBOLS 13 Wireless communication part 14 Communication controller 15 Sound amplifier 16 Speaker 17 Operation key 18 GPS processing part 19 Camera 20 1st display part 21 2nd display part 22 Touch panel 300 Display apparatus 301 Detection part 302 Acquisition part 303 Generation part 304 Display part 400 User 401 (Camera) position 402 (Camera) orientation 405a, 405b, 405c Building 410a, 410b, 410c Target 420 GPS satellite 500 Field of view 510 Square pyramid 600, 1000, 1200 Investigation area 610 Second square pyramid 700 Projection surface 701 Photo area 702 Report area 705a, 705b, 1105a, 1105b Projection line 710a, 710b Mark 800 Display image 810 Captured image 1010, 1210 Paraboloid

Claims (7)

A detector that detects the position and orientation of the camera,
An acquisition unit for acquiring a position of a target in an investigation region determined by the detected position and orientation of the camera;
A photographed image obtained by photographing the visual field region defined by the position and orientation of the camera is photographed in a photographic region within a projection plane, and the position of the acquired target is determined by a predetermined projection conversion. A generation unit that generates a display image by drawing a mark representing the target object at a position projected in a report area in the projection plane;
A display unit for displaying the generated display image on a screen;
The reporting area includes the photographic area;
The survey area has a portion not included in the visual field area. The display device according to claim 1,
The investigation area includes the visual field area in the vicinity of the position of the camera. The display device according to claim 1 or 2,
The predetermined projection conversion coincides with a projection conversion for projecting the visual field area onto the photographic area. The display device according to any one of claims 1 to 3,
The field of view is defined by a first cone;
The survey area is defined by a second cone;
The axis of the first cone and the axis of the second cone overlap,
The display device characterized in that the second cone includes the first cone. The display device according to any one of claims 1 to 3,
The field of view is defined by a cone;
The survey area is defined by a quadric surface,
The display device, wherein the axis of the cone and one of the axes of the quadric surface overlap. A display method executed by a display device including a detection unit, an acquisition unit, a generation unit, and a display unit,
A detection step in which the detection unit detects the position and orientation of the camera;
An acquisition step in which the acquisition unit acquires a position of a target in a survey area determined by the detected position and orientation of the camera;
The generation unit draws a photographed image obtained by the camera photographing a visual field region determined by the position and orientation of the camera in a photographic region within a projection plane, and the position of the acquired target is predetermined. Generating a display image by drawing a mark representing the target at a position projected in the report area in the projection plane by the projection conversion of
The display unit includes a display step of displaying the generated display image on a screen;
The reporting area includes the photographic area;
The survey method has a portion not included in the visual field region. Computer
A detector that detects the position and orientation of the camera,
An acquisition unit for acquiring a position of a target in an investigation region determined by the detected position and orientation of the camera;
A photographed image obtained by photographing the visual field region defined by the position and orientation of the camera is photographed in a photographic region within a projection plane, and the position of the acquired target is determined by a predetermined projection conversion. A generation unit that generates a display image by drawing a mark representing the target object at a position projected in a report area in the projection plane;
Function as a display unit for displaying the generated display image on a screen;
The reporting area includes the photographic area;
The research area is made to function so as to have a portion not included in the visual field area.

Images