JP2011227759A - Image display device and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image display device capable of switching a view point position with more intuitive operation when observing an object in three-dimensional virtual space.SOLUTION: An image display device comprises an image acquisition section, an display control section and an attitude detection section for detecting an attitude change of the device. The image acquisition section acquires a display image corresponding to a three-dimensional object observed in a predetermined view point position. The display control section makes the display device display a display image. The image acquisition section, when an attitude change is detected, shifts its view point position toward the attitude changing direction in such a way as to follow the change to acquire a display image again.

Description

  The present invention relates to an image display device and a program.

  Conventionally, various techniques for displaying an object in a three-dimensional virtual space on a monitor have been proposed (see, for example, Patent Document 1).

JP2009-211423

  When observing an object in a three-dimensional virtual space with an image display device, there is a demand for a user interface that can switch the viewpoint position with a more intuitive operation.

  In view of the above circumstances, an image display device capable of switching the viewpoint position with a more intuitive operation when observing an object in a three-dimensional virtual space is provided.

  An image display apparatus according to one aspect includes an image acquisition unit, a display control unit, and a posture detection unit that detects a change in the posture of the device. The image acquisition unit acquires a display image corresponding to a state in which a three-dimensional object is observed at a predetermined viewpoint position. The display control unit displays the display image on the display device. Then, when a change in posture is detected, the image acquisition unit shifts the viewpoint position so as to follow the direction in which the posture changes, and re-acquires the display image.

  In the one aspect described above, the image acquisition unit may acquire a two-dimensional image corresponding to a state in which a three-dimensional object is observed at a predetermined viewpoint position as a display image.

  The image display device according to the one aspect may further include an imaging unit that captures an image of the subject via the imaging optical system. Further, the image acquisition unit may set the viewpoint position with reference to the optical axis of the photographing optical system. Furthermore, the image acquisition unit may change the magnification of the object in the display image according to the zoom state of the imaging unit.

  Further, the display control unit may display a pointer for designating a position on the screen when displaying the display image. Then, the image acquisition unit may enlarge or rotate the display image with reference to the position designated by the pointer on the display image.

  The image display device according to the one aspect includes a current position acquisition unit that acquires the coordinates of the current position of the device, an orientation detection unit that detects the orientation of the observation target, and a distance acquisition unit that calculates a distance from the observation target. And may be further provided. Then, when the information of the three-dimensional object is associated with the position coordinates, the image acquisition unit obtains the position coordinates of the observation target using the coordinates of the current position, the orientation of the observation target, and the distance to the observation target. The display image may be acquired from information of a three-dimensional object corresponding to the position coordinates of the observation target.

  Also, an aspect in which each operation in the image display apparatus according to one aspect described above is expressed as an image display method by a computer, a program that causes the computer to function as an image display apparatus according to one aspect, and a program storage medium storing the program This configuration is also effective as a specific embodiment of the present invention.

  According to the present invention, when a change in the posture of the apparatus is detected, the display position is acquired by shifting the viewpoint position so as to follow the direction in which the posture changes, and the viewpoint position is switched by a more intuitive operation. Can do.

1 is a block diagram illustrating a configuration example of an image display device according to an embodiment; A flowchart showing an example of a display operation of a three-dimensional object in one embodiment Schematic diagram showing an example of the relationship between the 3D object and the viewpoint position in the virtual space (A)-(c): The figure which shows the example of a change of a projection image at the time of shaking an electronic camera to the left direction (A)-(c): The figure which shows the example of a change of a projection image at the time of shaking an electronic camera to a downward direction (A): a diagram showing an example in which a focus selector is superimposed on a projected image, (b): a diagram showing an example in which a projected image is enlarged and displayed with reference to the focus selector in FIG. 6 (a), (c): a diagram The figure which shows the example which rotated and displayed the projection image on the basis of the focus selector of 6 (a)

<Configuration example of image display device>
FIG. 1 is a block diagram illustrating a configuration example of an electronic camera including an image display device according to an embodiment. Here, the image display apparatus according to an embodiment has a function of reading modeling data of a three-dimensional object generated in advance by a computer or the like and displaying a two-dimensional projection image of the object.

  An electronic camera 1 according to an embodiment includes a photographing lens 11, an imaging unit 12, a camera microcomputer 13, a recording I / F 14, a monitor 15, an attitude detection unit 16, a memory 17, and a current position acquisition unit 18. , An orientation detection unit 19, a distance acquisition unit 20, a release button 21, a zoom switch 22, and a selection button 23. Here, the imaging unit 12, recording I / F 14, monitor 15, posture detection unit 16, memory 17, current position acquisition unit 18, direction detection unit 19, distance acquisition unit 20, release button 21, zoom switch 22, selection button 23 Are respectively connected to the camera microcomputer 13.

  The taking lens 11 is composed of a plurality of lenses including a zoom lens and a focusing lens. Each lens position of the photographing lens 11 is adjusted in the optical axis direction by a lens driving unit (not shown). For the sake of simplicity, FIG. 1 shows the photographing lens 11 as a single lens.

  The imaging unit 12 is a module that captures (captures) an image of a subject formed by the photographing lens 11. For example, the imaging unit 12 includes an imaging element that performs photoelectric conversion, and an image processing circuit that performs A / D conversion and various types of image processing.

  The camera microcomputer 13 is a processor that comprehensively controls the operation of the electronic camera 1. For example, the camera microcomputer 13 executes well-known autofocus control and automatic exposure control at the time of shooting. The camera microcomputer 13 may perform autofocus control at the time of shooting so as to focus on a subject specified by a focus selector described later.

  The camera microcomputer 13 functions as an image acquisition unit 31 and a display control unit 32 by executing a program.

  The image acquisition unit 31 acquires a two-dimensional projection image using the modeling data of the three-dimensional object. This projection image is an image observed by projecting a three-dimensional object in the virtual space onto a projection plane based on a predetermined viewpoint position (projection center). In one embodiment, the viewpoint position with respect to the object in the virtual space is set based on the optical axis of the photographing lens 11.

  The display control unit 32 outputs data of a display target image (such as a projection image) to the monitor 15 and controls the display state of the image on the monitor 15.

  The recording I / F 14 has a connector for connecting a nonvolatile storage medium 33. The recording I / F 14 executes data writing / reading with respect to the storage medium 33 connected to the connector. The storage medium 33 includes a hard disk, a memory card incorporating a semiconductor memory, or the like. In FIG. 1, a memory card is illustrated as an example of the storage medium 33. Note that the modeling data of the three-dimensional object may be recorded in the storage medium 33.

  The monitor 15 displays various images according to instructions from the display control unit 32. In addition, the monitor 15 in one embodiment is composed of, for example, a liquid crystal display panel disposed on the back of the casing of the electronic camera 1 (see FIG. 3).

  The attitude detection unit 16 detects changes in the attitude of the electronic camera 1 (yaw, pitch, and angle in the rolling direction of the electronic camera 1 with respect to the optical axis of the photographing lens 11). For example, the posture detection unit 16 includes a known three-axis acceleration sensor that supports measurement of gravitational acceleration (static acceleration).

  The memory 17 is composed of a nonvolatile storage medium such as a flash memory. The memory 17 stores a program executed by the camera microcomputer 13 and data of a three-dimensional object. An operation example in the display mode by this program will be described later. Note that the modeling data of the three-dimensional object may be recorded in the memory 17 in advance.

  The current position acquisition unit 18 acquires coordinate information including the latitude, longitude, and altitude of the current position of the electronic camera 1. As an example, the current position acquisition unit 18 is configured by a known positioning device using GPS (Global Positioning System).

  The azimuth detecting unit 19 detects the azimuth of the observation target facing the photographing lens 11. For example, the azimuth detector 19 includes a triaxial magnetic sensor that detects geomagnetic components in directions orthogonal to each other. Then, the azimuth detecting unit 19 detects the direction of the optical axis of the photographic lens 11 from the magnetic field detection value by each magnetic sensor.

  The distance acquisition unit 20 obtains a distance from the observation target. For example, the distance acquisition unit 20 is a laser distance meter that measures the distance to the observation target by receiving laser light reflected by the observation target.

  The release button 21 is a switch that accepts an image capturing instruction to the imaging unit 12. The zoom switch 22 is a switch that accepts an optical zoom operation by moving the zoom lens and an electronic zoom operation by changing the resolution of the image. The selection button 23 is, for example, a cross key and is a switch that accepts a user's selection operation. For example, the selection button 23 is used for an operation of moving a pointer (focus selector) indicating a position (focus area) to be focused on the shooting screen during shooting.

  Next, a display operation example of a three-dimensional object in the electronic camera of one embodiment will be described with reference to the flowchart of FIG. Here, the process of the flowchart of FIG. 2 is started in response to a program start instruction from the user.

  Step # 101: The image acquisition unit 31 reads modeling data of a three-dimensional object from the storage medium 33 or the memory 17.

  Here, when there are a plurality of modeling data of a three-dimensional object associated with position coordinates (for example, there is a database in which modeling data of an existing building is associated with position coordinates of the building) ), The image acquisition unit 31 in # 101 may read modeling data corresponding to the observation target to which the electronic camera 1 is directed.

  At this time, the image acquisition unit 31 acquires the coordinate information of the current position of the electronic camera 1, the orientation of the observation target, and the distance from the observation target. Next, the image acquisition part 31 calculates | requires the position coordinate of observation object from each said information. Then, the image acquisition unit 31 performs matching between the position coordinates on the modeling data side and the position coordinates obtained by the calculation, and reads out modeling data in which the deviation between the coordinates falls within the threshold range as an observation target.

  Step # 102: The image acquisition unit 31 sets the viewpoint position in the initial state.

  As an example, the image acquisition unit 31 acquires the current posture information of the electronic camera 1 from the posture detection unit 16. Then, the image acquisition unit 31 associates the current yaw angle of the electronic camera 1 with the front of the three-dimensional model in the virtual space. Further, the image acquisition unit 31 shifts the viewpoint position in the virtual space in the vertical direction according to the pitch angle of the electronic camera 1 with respect to the horizontal plane. Further, the image acquisition unit 31 rotates the projection surface of the display image according to the rolling angle of the electronic camera 1. Further, the image acquisition unit 31 sets the display magnification of the display image to a state without enlargement. The image acquisition unit 31 stores the posture information of the electronic camera 1 detected in # 102 in the memory 17 in order to detect a change in the posture of the electronic camera 1.

  Here, FIG. 3 schematically shows a relationship example between the three-dimensional object and the viewpoint position in the virtual space. In the following description, the case where the viewpoint position in the initial state is a position where the projection plane faces the front of the object in the virtual space will be described.

  Step # 103: The image acquisition unit 31 sets a projection plane on the virtual space with reference to the current viewpoint position, and generates a projection image in which the three-dimensional object is projected on the projection plane. For example, the image acquisition unit 31 may generate a projection image by performing a known geometric transformation on the modeling data.

  Step # 104: The display control unit 32 displays the projection image acquired by the process of # 103 on the monitor 15. Note that the camera microcomputer 13 may generate attention information indicating the viewpoint position of the currently displayed projection image in response to pressing of the release button 21. By using this attention information, it is possible to generate a projection image at a preferred angle when displaying a three-dimensional object.

  Step # 105: The image acquisition unit 31 determines whether or not there is a change in the posture (yaw angle, pitch angle, rolling angle) of the electronic camera 1. As an example, the image acquisition unit 31 compares the posture information acquired from the posture detection unit 16 with the posture information stored in the memory 17, and changes to the posture of the electronic camera 1 if there is a difference larger than the threshold value. Judge that there is.

  If the above requirement is satisfied (YES side), the process proceeds to # 106. On the other hand, if the above requirement is not satisfied (NO side), the process proceeds to # 107. Note that the image acquisition unit 31 may overwrite the attitude information of the electronic camera 1 in the memory 17 on the NO side of # 105.

  Step # 106: The image acquisition unit 31 shifts the viewpoint position so as to follow the direction in which the posture of the electronic camera 1 changes. Thereafter, the image acquisition unit 31 overwrites the attitude information of the electronic camera 1 on the memory 17, and returns to the process of # 103, and the camera microcomputer 13 repeats the above operation.

  As an example, consider a case where the yaw angle is changed by the user shaking the electronic camera 1 leftward with reference to FIG. At this time, the image acquisition unit 31 in # 106 rotates the viewpoint position to the right by the same angle as the yaw angle around the reference point in the virtual space. As another example, a case where the pitch angle is changed by the user shaking the electronic camera 1 downward will be considered with reference to FIG. At this time, the image acquisition unit 31 in # 106 rotates the viewpoint position upward by the same angle as the pitch angle around the reference point in the virtual space.

  And in the process of # 103 looped from # 106, the image acquisition part 31 produces | generates a projection image again according to a new viewpoint position. FIG. 4 shows a change example of the projected image when the electronic camera 1 is shaken in the left direction. FIG. 5 shows an example of a change in the projected image when the electronic camera 1 is shaken downward.

  Thus, the viewpoint position in the virtual space moves with the same movement as when the subject is photographed with the electronic camera 1. Therefore, the user can intuitively change the viewpoint position in the virtual space by changing the posture of the electronic camera 1.

  Step # 107: The image acquisition unit 31 determines whether or not a zoom operation by the zoom switch 22 has been accepted. If the above requirement is satisfied (YES side), the process proceeds to # 108. On the other hand, if the above requirement is not satisfied (NO side), the process proceeds to # 109.

  Step # 108: The image acquisition unit 31 changes the magnification of the object in the virtual space according to the zoom operation. The image acquisition unit 31 in # 108 changes the enlargement magnification greatly when the zoom operation (zoom-in) to the telephoto side is performed, and decreases the enlargement magnification when the zoom operation (zoom-out) to the wide angle side is performed. To do. Thereafter, returning to the process of # 103, the camera microcomputer 13 repeats the above operation.

  In the process of # 103 looped from # 108, the image acquisition unit 31 adjusts the range of the projection plane in the virtual space in accordance with the enlargement magnification, and the resolution (image size) of the monitor 15 is changed to the image of the projection plane. You may convert according to.

  Further, the display control unit 32 may display a pointer for designating a position on the screen in the same display format as the focus selector when displaying the projection image (see FIG. 6A). Then, the image acquisition unit 32 enlarges the projection image (FIG. 6B) or rotates the projection image (FIG. 6C) with reference to the position specified by the pointer (focus selector) on the projection image. You may make it perform.

  Step # 109: The camera microcomputer 13 determines whether an instruction to end the display operation of the three-dimensional object has been received. If the above requirement is satisfied (YES side), the camera microcomputer 13 ends the display operation. On the other hand, if the above requirement is not satisfied (NO side), the camera microcomputer 13 returns to # 105 and repeats the above operation. Above, description of the flowchart of FIG. 2 is complete | finished.

  When detecting a change in the attitude of the apparatus (# 105), the electronic camera 1 according to the embodiment shifts the viewpoint position so as to follow the direction in which the attitude changes (# 106). Further, the electronic camera 1 changes the magnification of the object in accordance with the zoom operation by the zoom switch 22 (# 108). Thereby, the user can change the image of the three-dimensional object displayed on the monitor 15 with the same operation feeling as when photographing the subject with the electronic camera 1.

<Supplementary items of the embodiment>
(1) In the above embodiment, the example in which the image display device is incorporated in the electronic camera 1 has been described. However, the image display device of the present invention can be widely applied to other electronic devices having an image display function (such as a photo viewer, a mobile phone, a portable game machine, an electronic book viewer, and a head-mounted display). In the case of these electronic devices, the viewpoint position may be set with reference to the display surface of the monitor.

  (2) In the above embodiment, the example in which the modeled three-dimensional object is a display target has been described. However, the present invention can also be applied to display of panoramic images in all directions, for example.

  (3) In the above embodiment, the example in which the projection image of the object is a two-dimensional image has been described. However, the image display device of the present invention may display a stereo pair image for stereoscopic viewing on the monitor as the projected image of the object. The stereo pair format may be a known method. For example, the image display device may display in parallel a left-eye image and a right-eye image that are observed by the parallel method or the intersection method. Alternatively, the image display apparatus may cause the left-eye image and the right-eye image having different polarization directions to be observed using polarized glasses with different polarizing angles of both eyes. Further, when the liquid crystal shutter is used, the image display device may alternately display the left-eye image and the right-eye image. Further, the image display device may convert the left-eye image and the right-eye image into red-blue anaglyph images having different colors. Further, the image display device may output a stereoscopic television image in which left-eye images and right-eye images are alternately arranged in a predetermined stripe pattern.

  (4) In the above embodiment, the program executed by the camera microcomputer 13 of the electronic camera 1 may be a firmware program that is updated by version upgrade or the like. That is, the function of the image display apparatus of the present invention may be provided by updating the firmware program of the electronic camera 1. When the image display device is a device other than the electronic camera 1 (such as a photo viewer or a mobile phone), the program of the above embodiment may be provided as an optional application program.

  (5) In the above embodiment, the distance acquisition unit may obtain the subject distance using the position of the photographing lens at the time of focusing.

  From the above detailed description, features and advantages of the embodiments will become apparent. It is intended that the scope of the claims extend to the features and advantages of the embodiments as described above without departing from the spirit and scope of the right. Further, any person having ordinary knowledge in the technical field should be able to easily come up with any improvements and modifications, and there is no intention to limit the scope of the embodiments having the invention to those described above. It is also possible to use appropriate improvements and equivalents within the scope disclosed in.

DESCRIPTION OF SYMBOLS 1 ... Electronic camera, 11 ... Shooting lens, 12 ... Imaging part, 13 ... Camera microcomputer, 14 ... Recording I / F, 15 ... Monitor, 16 ... Attitude detection part, 17 ... Memory, 18 ... Current position acquisition part, 19 ... Direction detection unit, 20 ... distance acquisition unit, 21 ... release button, 22 ... zoom switch, 23 ... selection button, 31 ... image acquisition unit, 32 ... display control unit, 33 ... storage medium

Claims (7)

An image acquisition unit that acquires a display image corresponding to a state in which a three-dimensional object is observed at a predetermined viewpoint position;
A display control unit for displaying the display image on a display device;
An attitude detection unit that detects a change in the attitude of the device,
The image acquisition unit is configured to re-acquire the display image by shifting the viewpoint position so as to follow the direction in which the posture changes when the change in the posture is detected. The image display device according to claim 1,
The image acquisition unit is an image display device that acquires, as the display image, a two-dimensional image corresponding to a state in which a three-dimensional object is observed at a predetermined viewpoint position. The image display device according to claim 1 or 2,
An imaging unit that captures an image of the subject via the imaging optical system;
The image acquisition unit is an image display device that sets the viewpoint position with reference to an optical axis of the photographing optical system. The image display device according to claim 3,
The image acquisition unit is an image display device that changes an enlargement magnification of the object in the display image according to a zoom state of the imaging unit. In the image display device according to claim 3 or 4,
The display control unit displays a pointer for designating a position on the screen when the display image is displayed,
The image acquisition unit is an image display device that enlarges or rotates the display image with reference to a position designated by the pointer on the display image. The image display device according to any one of claims 1 to 5,
A current position acquisition unit for acquiring coordinates of the current position of the device;
An azimuth detector that detects the azimuth of the observation target;
A distance acquisition unit for obtaining a distance to the observation target;
When the information of the three-dimensional object is associated with the position coordinates, the image acquisition unit obtains the position coordinates of the observation target using the coordinates of the current position, the azimuth, and the distance, and the observation target An image display device that acquires the display image from the information of the three-dimensional object corresponding to the position coordinates. Processing for obtaining a display image corresponding to a state in which a three-dimensional object is observed at a predetermined viewpoint position;
Processing for displaying the display image on a display device;
Processing to detect changes in the posture of the device;
A program for causing a computer to execute a process of re-acquiring the display image by shifting the viewpoint position so as to follow the direction in which the posture changes when the change in posture is detected.

Images