JP2012205153A - Photographing device, control method, and virtual information display program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable plural pieces of information added to an image to be easily selected.SOLUTION: A mobile phone terminal (photographing device) 1 comprises: a photographing unit 40; a display unit 2 that displays an image acquired by the photographing unit; and a control unit 22 that superimposes objects related to the image acquired by the photographing unit 40 on the image, and causes the display unit 2 to display them. The control unit 22 performs selection processing of an object overlapping a display region of a predetermined object included in the image among the objects.

Description

  The present invention relates to a photographing apparatus, a control method, and a virtual information display program.

  In recent years, augmented reality (AR) technology that adds an additional information by processing a real-space image with a computer has attracted attention (for example, see Patent Document 1). There is also known a technique for displaying more detailed information by selecting information added to an image in the real space.

JP 2002-027349 A

  However, conventionally, there has not been provided an operation method capable of selecting a plurality of pieces of information added to an image in the real space collectively. For this reason, when the user does not know which of a plurality of pieces of information added to the real space image is the desired information, the user selects information one by one and confirms the associated detailed information. I had to do it.

  An object of the present invention is to provide a photographing apparatus, a control method, and a virtual information display program that can easily select a plurality of information added to an image.

  An imaging apparatus according to the present invention includes an imaging unit, a display unit that displays an image acquired by the imaging unit, and an object related to the image acquired by the imaging unit that is superimposed on the image on the display unit. A control unit for displaying, and the control unit selects an object that overlaps a display area of a predetermined object included in the image among the objects.

  Here, it is preferable that the control unit performs the selection process in response to a change in display mode of the predetermined object. Note that the change in the display mode of the predetermined object may include the disappearance of the protrusion included in the predetermined object.

  Moreover, it is preferable that the said control part displays the said object selected by the said selection process as a list on the said display part. The control unit may display the list of the selected objects by rearranging the selected objects according to the distance from the own device. The control unit may classify and display the objects selected by the selection process for each category.

  Further, it is preferable that the display area of the predetermined object includes an inner side of an area where ends of a plurality of protrusions constituting the predetermined object are connected to each other.

  In addition, when the display area of the predetermined object moves before the display mode of the predetermined object changes, the control unit preferably further performs a selection process on the object displayed on the movement locus. .

  In order to solve the above-described problems and achieve the object, the control method according to the present invention includes a step of displaying an image acquired by the imaging unit, and an object related to the image acquired by the imaging unit. Over the image and displayed on the display unit, and among the objects, a step of selecting an object that overlaps a display area of a predetermined object included in the image.

  In order to solve the above-described problems and achieve the object, a virtual information display program according to the present invention is obtained by a step of displaying an image acquired by an imaging unit on an information processing apparatus and the imaging unit. A step of causing the display unit to display an object related to the image on the image and a step of selecting an object that overlaps a display area of a predetermined object included in the image. .

  The present invention has an effect that a plurality of information added to an image can be easily selected.

FIG. 1 is a front view showing an appearance of the mobile phone terminal according to the present embodiment. FIG. 2 is a block diagram illustrating a functional configuration of the mobile phone terminal according to the present embodiment. FIG. 3 is a diagram illustrating an example of virtual information. FIG. 4 is a diagram illustrating an example of selection of an AR tag. FIG. 5 is a diagram for explaining the selected region. FIG. 6 is a diagram illustrating another example of selection of an AR tag. FIG. 7 is a diagram showing a processing procedure of AR tag selection processing by the mobile phone terminal. FIG. 8 is a diagram illustrating an example in which AR tags are displayed by category. FIG. 9 is a diagram illustrating an example of a scene where the hand first overlaps the AR tag. FIG. 10 is a diagram for explaining the reference distance and the conversion coefficient. FIG. 11 is a diagram illustrating an example of a threshold value for selecting an AR tag.

  Hereinafter, the present invention will be described in detail with reference to the drawings. The present invention is not limited to the following description. In addition, constituent elements in the following description include those that can be easily assumed by those skilled in the art, those that are substantially the same, and those in a so-called equivalent range. In the following, a mobile phone terminal will be described as an example of a photographing device. However, the application target of the present invention is not limited to a mobile phone terminal. For example, a PHS (Personal Handyphone System), a PDA, a portable navigation device, a notebook The present invention can also be applied to various information processing apparatuses such as personal computers and game machines.

  First, the external appearance of the mobile phone terminal 1 according to the present embodiment will be described with reference to FIG. FIG. 1 is a front view showing the appearance of the mobile phone terminal 1. As shown in FIG. 1, the casing 1 </ b> C of the mobile phone terminal 1 includes a first casing 1 </ b> CA and a second casing 1 </ b> CB connected to each other by a hinge mechanism 8 so as to be opened and closed. That is, the mobile phone terminal 1 has a foldable casing.

  Note that the casing of the mobile phone terminal 1 is not limited to such a structure. For example, the housing of the mobile phone terminal 1 may be a sliding housing that allows one housing and the other housing to slide from each other in a state where both housings are overlapped, A rotary type in which one casing is rotated around an axis along the overlapping direction, or two casings connected via a biaxial hinge may be used. Further, the case of the mobile phone terminal 1 may be a so-called straight type (slate type) case made of one case.

  1st housing | casing 1CA has the display part 2, the receiver 16, and the imaging | photography part 40. FIG. The display unit 2 includes a display device such as a liquid crystal display (LCD) or an organic EL (Organic Electro-Luminescence) panel, and displays various types of information such as characters, figures, and images. The display unit 2 can also display an image acquired by the imaging unit 40. The receiver 16 outputs the other party's voice during a call.

  The imaging unit 40 captures an image by an imaging unit such as an imaging sensor. The photographing window for guiding external light to the photographing means of the photographing unit 40 is provided on the surface opposite to the surface on which the display unit 2 of the first housing 1CA is provided. That is, the first casing 1CA is configured such that when the user views the display unit 2 from the front, an image on the opposite side to the first casing 1CA acquired by the photographing unit 40 is displayed on the display unit 2. Has been.

  The second casing 1CB includes an operation key 13A composed of a numeric keypad, a function key, etc., a direction and determination key 13B for executing menu selection and determination, screen scrolling, and the like, and voice acquisition means for acquiring voice during a call. And a microphone 15. The operation key 13A and the direction and determination key 13B constitute the operation unit 13 of the mobile phone terminal 1. Note that the operation unit 13 may include a touch sensor superimposed on the display unit 2 instead of the operation key 13A or the like or in addition to the operation key 13A or the like.

  Next, a functional configuration of the mobile phone terminal 1 will be described with reference to FIG. FIG. 2 is a block diagram showing a functional configuration of the mobile phone terminal 1. As shown in FIG. 2, the mobile phone terminal 1 includes a communication unit 26, an operation unit 13, a voice processing unit 30, a display unit 2, a photographing unit 40, a position / attitude detection unit 36, and a control unit 22. And a storage unit 24.

  The communication unit 26 includes an antenna 26a, establishes a radio signal line by a CDMA (Code Division Multiple Access) method or the like with a base station via a channel assigned by the base station, and communicates with other devices through the base station. Phone communication and information communication. When the operation key 13A or the direction and determination key 13B is operated by the user, the operation unit 13 outputs a signal corresponding to the operation content to the control unit 22.

  The sound processing unit 30 converts the sound input from the microphone 15 into a digital signal and outputs the digital signal to the control unit 22. In addition, the audio processing unit 30 decodes the digital signal output from the control unit 22 and outputs the decoded digital signal to the receiver 16. The display unit 2 displays various information according to the control signal input from the control unit 22. The imaging unit 40 converts the acquired image into a digital signal and outputs the digital signal to the control unit 22.

  The position / posture detection unit 36 detects the position and posture of the mobile phone terminal 1 and outputs the detection result to the control unit 22. The position / attitude detection unit 36 detects the position of the mobile phone terminal 1 based on, for example, a GPS (Global Positioning System) receiver or a base station with which the communication unit 26 has established a radio signal line. In addition, the position / posture detection unit 36 detects the posture of the mobile phone terminal 1 based on, for example, a three-axis acceleration sensor, an orientation sensor, and a gyro sensor.

  The control unit 22 includes a CPU (Central Processing Unit) that is a calculation unit and a memory that is a storage unit, and implements various functions by executing programs using these hardware resources. Specifically, the control unit 22 reads a program or data stored in the storage unit 24 and expands it in a memory, and causes the CPU to execute instructions included in the program expanded in the memory. And the control part 22 reads / writes data with respect to a memory and the memory | storage part 24 according to the execution result of the command by CPU, or controls operation | movement of the communication part 26, the display part 2, etc. FIG. When the CPU executes an instruction, data developed in the memory, a signal input from the position / posture detection unit 36, and the like are used as parameters.

  The storage unit 24 includes a nonvolatile storage device such as a flash memory, and stores various programs and data. The programs and data stored in the storage unit 24 include a virtual information display program 24a, one or more virtual information 24b, and shape information 24c. In addition, the memory | storage part 24 may be comprised by the combination of portable storage media, such as a memory card, and the reading / writing apparatus which reads / writes with respect to a storage medium. In this case, the program and data that are supposed to be stored in the storage unit 24 are stored in a portable storage medium. Further, the program and data that are supposed to be stored in the storage unit 24 may be acquired from another device such as a server device by wireless communication by the communication unit 26.

  Based on augmented reality (AR) technology, the virtual information display program 24a superimposes virtual information corresponding to actual facilities and articles on the image acquired by the imaging unit 40 and displays the virtual information on the display unit 2. Provide functionality. Specifically, the virtual information display program 24a controls the virtual information 24b corresponding to the facility or article in the shooting range of the shooting unit 40 based on the position and posture detected by the position / posture detection unit 36. To get. Then, the virtual information display program 24a displays an object called an AR tag corresponding to the acquired virtual information 24b so as to overlap the image.

  Further, the virtual information display program 24a provides a function that allows the user to select an arbitrary AR tag and a function that executes an operation associated with the selected AR tag. The virtual information display program 24a enables the user to select an arbitrary AR tag even when a plurality of AR tags are overlapped.

  The virtual information 24b holds information related to the AR tag that is displayed over the image acquired by the imaging unit 40. FIG. 3 is a diagram illustrating an example of the virtual information 24b. As shown in FIG. 3, the virtual information 24b includes information on classification items such as latitude, longitude, height, image, comment, and URL (Uniform Resource Locator). The latitude, longitude, and height indicate the positions of actual facilities and articles corresponding to the virtual information 24b. The image shows the substance of the object displayed as the AR tag. The comment is a descriptive text regarding the actual facility or article corresponding to the virtual information 24b. The URL indicates information associated with the AR tag. Note that the classification items included in the virtual information 24b are not limited to the above-described classification items, and may not include heights or comments.

  In addition, not all the virtual information 24b to be displayed is stored in the storage unit 24 in advance, but based on the position and orientation of the mobile phone terminal 1, the shooting range of the shooting unit 40 and the facilities and articles existing in the vicinity thereof The virtual information 24b corresponding to may be acquired from the server device as needed. In this case, a predetermined amount or a predetermined number of pieces of virtual information 24b may be cached in the storage unit 24. The server device here is, for example, a device operated by a trader who provides AR services.

  The shape information 24c holds information for detecting a predetermined object used for selecting an AR tag from the image acquired by the photographing unit 40. In this embodiment, it is assumed that the predetermined object used for selecting the AR tag is a human hand. It is assumed that the shape of the human hand is registered in the shape information 24c. The shape of the human hand here is, for example, a shape having a predetermined number or more of protrusions. In addition, the hand shape registered in the shape information 24c is not limited to any image of a hand that has been photographed or registered in advance, or hand shape feature information.

  Next, an AR tag selection process executed by the control unit 22 based on the function provided by the virtual information display program 24a will be described with reference to FIGS. FIG. 4 is a diagram illustrating an example of selection of an AR tag. FIG. 5 is a diagram for explaining the selected region. FIG. 6 is a diagram illustrating another example of selection of an AR tag.

  As shown in FIG. 4, when the photographing unit 40 acquires the image P, the control unit 22 displays the image P on the display unit 2 and tries to detect the shape of the hand by comparing the image P with the shape information 24c. . When the hand shape is detected, the control unit 22 extracts the contour of the hand by binarization or the like. In addition, you may implement | achieve the detection of the shape of a hand, for example using the technique etc. which are disclosed by Unexamined-Japanese-Patent No. 2003-346162. In addition, the hand recognition method is not limited to the above-described method, and for example, the hand shape stored in advance and the image P may be matched to detect the same hand shape. Further, in the same manner, two patterns of images in which the hand is opened (open state) and closed (closed state) are stored in advance, and the two patterns of images and the hand included in the image P are stored. The shape of the hand included in the image P may be detected by comparing the shapes of

  Further, the control unit 22 acquires virtual information 24b corresponding to the facility or article in the shooting range of the shooting unit 40 based on the position and posture of the mobile phone terminal 1 detected by the position / posture detection unit 36. The AR tag corresponding to the acquired virtual information 24b is displayed on the display unit 2 so as to overlap the image P.

  Specifically, the control unit 22 calculates which position of the image P acquired by the imaging unit 40 the position indicated by the acquired virtual information 24b corresponds to, and corresponds to the calculated position of the virtual information 24b. Display the AR tag. Where the position indicated by the virtual information 24b corresponds to in the image P depends on the position and orientation of the mobile phone terminal 1 detected by the position / posture detection unit 36, the positional information included in the virtual information 24b, and the imaging surface of the photographing unit 40 Is calculated on the basis of the size and angle of view.

  In addition, in the image P, an AR tag having a region where at least a part of another AR tag overlaps is displayed so that the position indicated by the corresponding virtual information 24 b is closer to the mobile phone terminal 1 and is positioned in front.

  In step S11 shown in FIG. 4, since the hand H is shown in the image P, the shape of the hand H included in the image P is detected by comparing the image P with the shape information 24c. In step S11, four AR tags A1 to AR4 A4 corresponding to facilities and articles in the photographing range of the photographing unit 40 are displayed superimposed on the image P. The AR tags A2 to A4 are displayed so as to overlap each other because the positions indicated by the corresponding virtual information 24b are close to each other when viewed from the focus of the photographing unit 40.

  In this scene, the hand H is in a closed state, and the display area of the hand H does not overlap with the display position of any AR tag. When the hand H detected in this way is in the closed state and the display area does not overlap with the display position of any AR tag, no AR tag is selected and the unselected state is maintained.

  Subsequently, in step S12, the hand H has moved to the open state after actually moving to a position corresponding to the display area including the display positions of the AR tags A2 to A4. In step S13, the hand H shifts to the closed state while maintaining the display area. As described above, when the hand H shifts from the open state to the closed state, the control unit 22 sets the display area of the hand H when it is open as the selection area, and selects the AR tag whose display position is included in the selection area. Put it in a state. In this example, in step S13, the three AR tags A2 to A4 are selected.

  As described above, when an operation of opening and closing the hand H is detected, the user can select a plurality of AR tags by selecting the AR tag including the display position in the display area of the hand H when the hand H is open. Tags can be easily selected. In addition, since the operation of opening and closing the hand H is the same as the operation of grasping the desired thing, the method of selecting the AR tag by such an operation is very intuitive and easy for the user to understand.

  Note that the selected region may be a region along the shape of the actual hand H, or may be an inner region where the tips (protrusions) of fingers are connected as in the region R1 shown in FIG. 5 may be a minimum rectangle including the shape of the hand H as in a region R2 shown in FIG. By setting the region R1 as the selection region, the AR tag displayed between the fingers is also selected, and a selection range suitable for the user's intention can be set. By setting the region R2 as the selected region, the amount of calculation for calculating the region can be reduced.

  When the display position of the AR tag is included in the display area of the hand H, it is preferable to display the shape of the hand H in front of the AR tag as shown in steps S12 and S13. By controlling in this way, the position of the hand H becomes clear and the user can easily understand which AR tag is selected. In addition, an AR tag in which only a part of the ideographic position is included in the selection area may be selected or not selected.

  When the AR tag is selected in this way, the control unit 22 displays the selected AR tag as a menu M1 on the display unit 2 as a list. In step S14 shown in FIG. 4, the AR tags that have been selected are displayed in a list in association with the comments included in the corresponding virtual information 24b. Thus, by displaying the AR tags in a list, the AR tags that have been overlapped can be arranged and displayed at a predetermined interval. By arranging and displaying the AR tags at a predetermined interval, the user can clearly identify each AR tag and easily select one of the AR tags. Become.

  The order in which the AR tags in the selected state are arranged may be, for example, the order in which the position indicated by the corresponding virtual information is closer to the mobile phone terminal 1 or the update date / time of the corresponding virtual information is the newest. There may be. By arranging the AR tags in the order in which the position indicated by the corresponding virtual information is closer to the mobile phone terminal 1, the arrangement order of the AR tags matches the order in which the corresponding facilities and articles can be seen. It becomes easy to grasp the correspondence with facilities and articles.

  Subsequently, as shown in step S15, when the user selects one of the AR tags on the menu M1, the control unit 22 executes an operation corresponding to the selected AR tag as shown in step S16. . In the example shown in FIG. 4, the control unit 22 accesses the URL included in the virtual information 24b corresponding to the AR tag A2 and displays the WEB page W1 on the display unit 2 as the operation corresponding to the selected AR tag A2. The operation to be performed is executed.

  Note that the user may select an AR tag on the menu M1 by operating the operation unit 13, or may perform a predetermined operation on the image acquired by the photographing unit 40 to perform the selection on the menu M1. AR tag selection may be performed. For example, the predetermined operation here refers to an operation of moving a finger up and down in the AR tag display area so as to tap, an operation of moving a finger so as to repel the AR tag, or a movement of a finger in the AR tag display area. This is an operation that stops for a certain period of time. Note that the finger recognition method may be realized by a detection method similar to the hand shape detection method.

  Further, when displaying the menu M1, it is preferable to display the menu M1 on the back of the hand shape so that the user can easily confirm the position of the hand selecting the AR tag. Alternatively, the menu M1 may be displayed translucently so that the user can confirm the position of the hand behind.

  FIG. 4 shows an example in which the AR tag is selected by moving the open hand to the closed state while maintaining the display area. However, the user can move to a wider range by moving the open hand. An AR tag can be selected. For example, as shown in FIG. 6, the hand H that has shifted to the open state in step S13 moves in the lower left direction to a position in which the display position of the AR tag A1 is included in the display area in step S17, and then enters the closed state. Assume that it has migrated. In this case, the control unit 22 places the AR tag A1 in a selected state in addition to the AR tags A2 to A4. Then, as shown in step S18, the control unit 22 displays the selected AR tags A1 to A4 as a menu M2 on the display unit 2 as a list.

  As described above, when the hand H in the open state moves, the control unit 22 monitors the movement trajectory, and the display position is included on the trajectory from the selection start area R3 before the movement to the selection completion area R4 after the movement. All AR tags to be selected are selected. By controlling in this way, the user can select a plurality of AR tags displayed at distant positions at a time. Here, the case where the hand H in the open state moves includes the case where the hand H moves while shifting from the open state to the closed state.

  Next, the operation of the mobile phone terminal 1 will be described with reference to FIG. FIG. 7 is a diagram showing a processing procedure of AR tag selection processing by the mobile phone terminal 1. The processing procedure shown in FIG. 7 is repeatedly executed until the user performs a predetermined end operation after the virtual information display program 24a is started. The AR tag selection process is realized by the control unit 22 executing the virtual information display program 24a.

  As shown in FIG. 7, the control unit 22 first sets a monitoring flag provided in the memory to OFF in step S101. The monitoring flag is a flag that is set to ON when the movement of the hand that has changed to the open state should be monitored. Then, the control part 22 displays the image P acquired by the imaging | photography part 40 on the display part 2 as step S102. In step S103, the control unit 22 causes the position / posture detection unit 36 to acquire the position and posture of the own apparatus.

  Subsequently, in step S104, the control unit 22 acquires virtual information 24b corresponding to the shooting range of the shooting unit 40 based on the position and posture of the own device. In step S105, the control unit 22 causes the display unit 2 to display the AR tag corresponding to the acquired virtual information 24b on the image P.

  Subsequently, the control unit 22 detects a hand included in the image P as step S106. When a hand is not detected (step S107, No), the control part 22 sets a monitoring flag to OFF as step S108, and performs again after step S102. On the other hand, when the hand contained in the image P is detected (step S107, Yes), the control part 22 determines whether the monitoring flag is ON as step S109. When the monitoring flag is not on (step S109, No), the control unit 22 determines whether the hand is in an open state as step S110. Whether the hand is in an open state is determined based on, for example, the inclusion of a shape corresponding to a finger.

  When the hand is in the open state (step S110, Yes), the control unit 22 sets the monitoring flag to ON as step S111. In step S112, the control unit 22 sets a selection start area based on the hand area at that time, and re-executes step S102 and subsequent steps. On the other hand, if the hand is not open (step S110, No), the control unit 22 re-executes step S102 and subsequent steps without changing the monitoring flag.

  If the monitoring flag is turned on in step S109, that is, if the hand has changed to an open shape (step S109, Yes), the control unit 22 determines whether the hand is in a closed shape as step S113. Determine. Whether the hand is in a closed shape is determined based on, for example, that a protrusion corresponding to a finger is not included in the shape. When the hand has not changed to the closed shape (No at Step S113), the control unit 22 re-executes Step S102 and the subsequent steps.

  When the hand has changed to a closed shape (step S113, Yes), the control unit 22 sets a selection completion region based on the region of the hand at that time as step S114. The selection completion area may be set based on the hand area of the closed shape, or may be set based on the assumed hand area assuming the shape when the closed hand is open. Good.

  Subsequently, in step S115, the control unit 22 selects all AR tags located on the locus from the selection start area to the selection completion area. The trajectory here may be a trajectory when moving linearly from the selection start area to the selection completion area, or may be a trajectory set along a path that the hand has actually reached. Good. Then, the control unit 22 displays a list of virtual information 24b corresponding to the selected AR tag as step S116.

  Subsequently, in step S117, the control unit 22 determines whether a user selection operation for the virtual information 24b displayed as a list is detected. When the selection operation is not detected (No at Step S117), the control unit 22 re-executes the determination at Step S117. When the selection operation is detected (step S117, Yes), the control unit 22 executes an operation corresponding to the virtual information 24b on which the selection operation is performed as step S118. The operation corresponding to the virtual information 24b is, for example, an operation of accessing a URL included in the virtual information 24b and displaying a WEB page.

  As described above, in the present embodiment, since the AR tags displayed at the position where the hand-gripping operation is detected can be selected together, a plurality of AR tags added to the image can be easily selected. You can choose.

  In addition, the aspect of this invention shown by said Example can be arbitrarily changed in the range which does not deviate from the summary of this invention. For example, in the above embodiment, the mobile phone terminal 1 executes the process related to the selection of the AR tag. However, the server apparatus that can communicate with the mobile phone terminal 1 may execute this process. In this case, the mobile phone terminal 1 transmits the image P acquired by the photographing unit 40 and the position and posture of the mobile phone terminal 1 detected by the position / posture detection unit 36 to the server device, and responds from the server device. The processed image is displayed on the display unit 2.

  In the above embodiment, a list of AR tags selected in accordance with the detection of the hand closing operation is displayed. However, how to display the AR tags selected in accordance with the detection of the hand closing operation is as follows. Any method can be used as long as each AR tag can be easily selected. For example, as in the menu M3 shown in FIG. 8, the selected AR tags may be classified into categories, and tabs such as tabs C1 to C3 may be provided for each category so that the AR tags are displayed for each category. .

  Further, when the AR tag is selected in accordance with the detection of the hand closing operation, the distance between the device itself and the AR tag in the depth direction may be taken into consideration. An example of selecting an AR tag in consideration of the distance in the depth direction with the own device will be described with reference to FIGS. 9 to 11. FIG. 9 is a diagram illustrating an example of a scene where the hand first overlaps the AR tag. FIG. 10 is a diagram for explaining the reference distance and the conversion coefficient. FIG. 11 is a diagram illustrating an example of a threshold value for selecting an AR tag.

  If the hand moves upward in the scene of step S11 shown in FIG. 4, as shown in FIG. 9, a part of the hand overlaps the end of the AR tag A3. As described above, when the hand first overlaps any one of the AR tags, the control unit 22 measures the distance between the mobile phone terminal 1 and the hand at that time and stores the measured distance in the memory as a reference distance. . The measurement of the distance between the mobile phone terminal 1 and the hand is realized by a focusing function used by the photographing unit 40 to focus on the object. The distance between the mobile phone terminal 1 and the hand may be measured using, for example, a distance measuring sensor that measures the distance using infrared rays, sound waves, or the like, or the shape information 24c is associated with a predetermined distance in advance. May be calculated based on the ratio between the size of the hand held in the image P and the size of the hand in the image P.

  Subsequently, the control unit 22 calculates the distance between the position of the mobile phone terminal 1 detected by the position / posture detection unit 36 and the position indicated by the virtual information 24b corresponding to the AR tag A3. Then, the control unit 22 converts the distance in the depth direction between the position indicated by the virtual information 24b corresponding to the AR tag A3 and the position indicated by the other virtual information 24b into the movement distance of the hand based on the obtained two distances. A conversion coefficient for calculating the value is calculated.

  As shown in FIG. 10, when the reference distance is DS and the distance between the position P3 indicated by the virtual information 24b corresponding to the AR tag A3 and the mobile phone terminal 1 is D3, the conversion coefficient is DS / D3. . By using this conversion coefficient, the relative position in the depth direction of each AR tag based on the AR tag A3 can be made to correspond to the position of the hand.

  For example, if DS is 10 [cm], that is, 0.1 [m], and D3 is 100 [m], the conversion coefficient is 0.001. In this case, if the position P2 indicated by the virtual information 24b corresponding to the AR tag A2 is 50 [m] before the position P3, 50 × 0.001 [m], that is, the position where the hand is moved 5 cm before Is the position of the AR tag A2 in the depth direction.

  Similarly, if the position P1 indicated by the virtual information 24b corresponding to the AR tag A1 is 10 [m] before the position P3, 10 × 0.001 [m], that is, the position where the hand is moved 1 cm before Is the position of the AR tag A1 in the depth direction. Also, assuming that the position P4 indicated by the virtual information 24b corresponding to the AR tag A4 is 10 [m] behind the position P3, 10 × 0.001 [m], that is, the position where the hand is moved 1 cm behind. This is the position in the depth direction of the AR tag A4.

  In this way, by associating the relative position of the AR tag in the depth direction with the position of the hand, when selecting the AR tag along with the detection of the closing operation of the hand, the distance between the own device and the AR tag in the depth direction is determined. It becomes possible to consider. That is, when an operation of closing the hand is detected, it is possible to perform control such that an AR tag located outside the range that can be grasped by the hand is not selected even if the AR tag is included in the open hand region. .

  Whether it is within a range that can be grasped by hand may be determined by setting a threshold as shown in FIG. 11, for example. In the example illustrated in FIG. 11, the position PS where the hand opening motion is detected and the range S from the position PE where the hand closing motion is detected to the position away from the threshold DT in the depth direction are determined to be grasped by the hand. . In the example shown in FIG. 11, the AR tag A2 positioned outside the range S is not selected even if it overlaps the range of the open hand.

DESCRIPTION OF SYMBOLS 1 Cellular phone terminal 2 Display part 13 Operation part 22 Control part 24 Storage part 24a Virtual information display program 24b Virtual information 24c Shape information 26 Communication part 30 Voice processing part 36 Position and attitude | position detection part 40 Imaging | photography part

Claims (10)

A shooting section;
A display unit for displaying an image acquired by the photographing unit;
A control unit that causes the display unit to display an object related to the image acquired by the imaging unit so as to overlap the image;
The control unit performs a selection process on an object that overlaps a display area of a predetermined object included in the image among the objects.   The imaging apparatus according to claim 1, wherein the control unit performs the selection process when a display mode of the predetermined object is changed.   The imaging apparatus according to claim 2, wherein the change in the display mode of the predetermined object includes a disappearance of a protrusion included in the predetermined object.   4. The photographing apparatus according to claim 1, wherein the control unit displays a list of the objects selected by the selection process on the display unit. 5.   The imaging apparatus according to claim 4, wherein the control unit displays a list display of the selected objects by rearranging the selected objects according to a distance from the own apparatus.   The imaging apparatus according to claim 4 or 5, wherein the control unit classifies and displays the objects selected by the selection process for each category.   7. The display device according to claim 1, wherein the display area of the predetermined object includes an inner side of an area where ends of a plurality of protrusions constituting the predetermined object are connected to each other. Shooting device.   When the display area of the predetermined object moves before the display mode of the predetermined object changes, the control unit further performs a selection process on the object displayed on the movement trajectory. The imaging device according to any one of claims 1 to 7. Displaying an image acquired by the imaging unit;
Causing the display unit to display an object related to the image acquired by the photographing unit on the image;
And a step of selecting an object that overlaps a display area of a predetermined object included in the image among the objects. In the information processing device,
Displaying an image acquired by the imaging unit;
Causing the display unit to display an object related to the image acquired by the photographing unit on the image;
A virtual information display program that executes a process of selecting an object that overlaps a display area of a predetermined object included in the image among the objects.

Images