JP6550766B2 - AR apparatus, AR implementation method, and computer program - Google Patents

Description

  The present invention relates to AR (Augmented Reality) technology.

  In recent years, AR (Augmented Reality) technology has begun to spread. AR is generally translated as “augmented reality”.

  According to the AR technology, when a printed matter to which an AR marker is attached is photographed by a smartphone or the like, content (object) corresponding to the AR marker is displayed on the image of the photographed printed matter. As a result, the user can feel as if the real space has been expanded.

  In addition, by printing an AR marker instead of printing confidential content, confidential content can be concealed.

  In addition, the following techniques related to AR have been proposed. An image processing apparatus described in Patent Literature 1 includes an image acquisition unit that acquires an input image that constitutes a video that reflects a real space, and the image capturing apparatus that has captured the input image by analyzing the input image. An analysis unit that recognizes at least one of a position and a posture in space, and a detection that detects an object that is associated with content arranged in the augmented reality space and exists in the real space in the input image A management unit that causes a storage medium to store detection information including at least one of a position and an attitude of the object detected by the detection unit and the detection unit; Associated with the detected object based on at least one of a relative position and orientation of the imaging device with respect to the detected object. It comprises a content control unit for controlling the behavior in the augmented reality space of content and.

JP, 2013-92964, A

  However, according to the technology described in Patent Document 1, even if the AR marker goes out of the shooting range, it is possible to display the corresponding object. However, in order to make an object such as an animal drawn in a picture book appear as if it pops out of the picture book, the relative relationship between the position and orientation of the imaging device and the object must always be calculated.

  If AR technology is used to conceal confidential content, it is required to more easily improve the visibility of the content.

  An object of the present invention is to improve the visibility of content displayed by AR more easily than the conventional one in view of such a subject.

An AR device according to an aspect of the present invention is an AR device that realizes an AR (Augmented Reality) by displaying an image so as to overlap with an object to be marked, and continuously displays the AR device. A photographing unit for photographing, a detection unit for detecting the marker from a photographed image obtained by the photographing unit for photographing, and a reception unit for receiving a corresponding image corresponding to the marker detected by the detection unit And, first display control means for causing the display means to display a focused portion which is all or part of the captured image each time the captured image is obtained by the imaging means, and the marker is included in the focused portion is the case are, the corresponding image is displayed on the display unit, the first portion of the other determined by the operation of the zoom be any portion of the captured image Is a portion for changing the portion of interest by scrolling to the second part, when the said marker by by that operation to the user has not passed the predetermined time after no longer included in the portion of interest, the The display of the corresponding image on the display means is ended when the corresponding image is continuously displayed on the display means and the predetermined time elapses after the marker is not included in the target portion by the operation. , And second display control means.

  According to the present invention, the visibility of the content to be displayed by the AR can be easily improved as compared with the prior art.

It is a figure which shows the example of the whole structure of a document browsing system. It is a figure which shows the example of the hardware constitutions of a document server. It is a figure which shows the example of the hardware constitutions of a tablet type terminal device. It is a figure which shows the example of the hardware constitutions of a spectacles type terminal device. It is a figure which shows the example of printed matter. It is a figure which shows the example of a functional structure of a document server and a terminal device. It is a figure which shows the example of the marker for a call preserve | saved at a content preservation | save part, and image data. It is a flowchart explaining the example of the flow of a superimposition necessity determination process. It is a figure for demonstrating the example which overlaps and displays a content on the attention partial image. It is a figure for demonstrating the example which overlaps and displays a content on the attention partial image. It is a figure for demonstrating the example which overlaps and displays a content on the attention partial image. It is a flowchart explaining the example of the flow of the whole process in a tablet type terminal device. It is a flowchart explaining the example of the flow of the whole process in a tablet type terminal device.

  FIG. 1 is a diagram illustrating an example of the overall configuration of the document browsing system 1. FIG. 2 is a diagram illustrating an example of a hardware configuration of the document server 2. FIG. 3 is a diagram illustrating an example of a hardware configuration of the tablet terminal device 3T. FIG. 4 is a diagram illustrating an example of a hardware configuration of the glasses-type terminal device 3M.

  As shown in FIG. 1, the document browsing system 1 includes a document server 2, a terminal device 3, a communication line 4 and the like. The document browsing system 1 manages data for displaying a document (document), and provides a user with a service of browsing the document.

  The document server 2 and the terminal device 3 can communicate with each other via the communication line 4. As the communication line 4, the Internet, Wi-Fi, LAN (Local Area Network), a cellular phone line, or the like is used.

  The document server 2 stores data for displaying a document and provides the data to the terminal device 3 as appropriate. A so-called server machine is used as the document server 2. Alternatively, a multifunction peripheral provided with functions such as copying, scanning, and faxing may be used, or a so-called cloud server may be used.

  As shown in FIG. 2, the document server 2 includes a CPU (Central Processing Unit) 20a, a RAM (Random Access Memory) 20b, a ROM (Read Only Memory) 20c, a mass storage device 20d, and a NIC (Network Interface Card) 20e. Composed of

  The NIC 20 e communicates with a device such as the terminal device 3 according to a protocol such as Transmission Control Protocol / Internet Protocol (TCP / IP).

  A content providing program 2P is stored in the ROM 20c or the mass storage device 20d. The content providing program 2P is loaded into the RAM 20b and executed by the CPU 20a. A hard disk, SSD (Solid State Drive), or the like is used as the mass storage device 20d.

  The terminal device 3 displays a document based on data provided from the document server 2. As the terminal device 3, a personal computer, a tablet computer, a smartphone, a glasses-type computer, or the like is used.

  The “glasses-type computer” is a kind of wearable computer, and is a computer that is worn on the face like glasses such as Google Glass of Google.

  Hereinafter, a case where a tablet computer or a glasses-type computer is used as the terminal device 3 will be described as an example. The former and the latter may be described separately as “tablet-type terminal device 3T” and “eyeglass-type terminal device 3M”, respectively.

  As shown in FIG. 3, the tablet terminal device 3T includes a CPU 30a, a RAM 30b, a ROM 30c, a flash memory 30d, a touch panel display 30e, an operation button group 30f, a digital camera 30g, a wireless communication device 30h, an acceleration sensor 30i, and the like. .

  The touch panel display 30e is configured by a flat display (for example, a liquid crystal display) and a touch panel. The flat display displays various images. It is used specifically to display documents. The touch panel detects the position touched by the user and notifies the CPU 30a of the position.

  The operation button group 30f includes a button for returning to the home screen, a button for confirming the input content, a button for adjusting the volume, a button for switching on / off the power, and the like.

  The digital camera 30g captures various images. In particular, it is used to capture a sheet on which a document is printed. The digital camera 30g has a digital zoom function.

  The wireless communication device 30h communicates with a device such as the document server 2 through a base station using a protocol such as TCP / IP. As a communication method of the wireless communication device 30h, for example, Wi-Fi, Bluetooth (registered trademark), LTE (Long Term Evolution), W-CDMA (Wideband-Code Division Multiple Access), PHS (Personal Handy-phone System), etc. Is used.

  The acceleration sensor 30i detects the acceleration and posture of the tablet terminal device 3T. Further, a change in position is detected based on a change in acceleration.

  A client program 3P is stored in the ROM 30c or the flash memory 30d. The client program 3P is loaded into the RAM 30b and executed by the CPU 30a.

  As shown in FIG. 4, the eyeglass-type terminal device 3M uses a CPU 31a, a RAM 31b, a ROM 31c, a flash memory 31d, a display 31e, an operation button group 31f, a digital camera 31g, a wireless communication device 31h, an acceleration sensor 31i, and the like.

  The functions of the CPU 31a, RAM 31b, ROM 31c, flash memory 31d, display 31e, operation button group 31f, digital camera 31g, wireless communication device 31h, acceleration sensor 31i are the CPU 30a, RAM 30b, ROM 30c, and flash memory of the tablet terminal device 3T, respectively. The operation is basically the same as 30d, touch panel display 30e, operation button group 30f, digital camera 30g, wireless communication device 30h, and acceleration sensor 31i.

  However, the method for reproducing an image is different between the touch panel display 30e and the display 31e. As described above, a flat display such as a liquid crystal display is used for the touch panel display 30e. On the other hand, the display 31e reproduces an image by emitting light from a projector provided at the root of the handle to a prism provided near the end of the lens.

  The client program 3P is stored in the ROM 31c or the flash memory 31d, loaded into the RAM 31b, and executed by the CPU 31a.

  FIG. 5 is a view showing an example of the printed matter 5. FIG. 6 is a diagram illustrating an example of functional configurations of the document server 2 and the terminal device 3. FIG. 7 is a diagram illustrating an example of the call marker 51Y and the image data 55 stored in the content storage unit 201. FIG. 8 is a flowchart for explaining an example of the flow of the necessity determination process for superposition.

  Incidentally, the document browsing system 1 can not only simply provide a file such as PDF (Portable Document Format) but also reproduce the document by AR (Augmented Reality) technology. At this time, the printed matter 5 is used.

  As shown in FIG. 5A, the printed matter 5 is printed with a calling marker 51Y in addition to the character string. The call marker 51Y is for calling the content of which data is stored in the document server 2.

  When the user wears the glasses-type terminal device 3M and sees the printed matter 5, the data of the content 51C is downloaded from the document server 2 based on the calling marker 51Y, and the content 51C is transferred to the printed matter 5 as shown in FIG. Looks as if it were printed. The same appears when the printed product 5 is viewed while being photographed with the tablet terminal device 3T.

  Thus, according to the document browsing system 1, AR is realized. As the calling marker 51Y, a marker in a predetermined format (for example, an AR marker) is used. The AR marker is sometimes called "AR code".

  This mechanism is realized by the content providing program 2P and the client program 3P. Hereinafter, this mechanism will be described by way of example where the user views the printed matter 5 with the tablet terminal device 3T.

  By executing the content providing program 2P by the CPU 20a, the content storage unit 201 and the content transmission unit 202 shown in FIG. 6 are realized in the document server 2.

  When the client program 3P is executed by the CPU 30a, the marker type detecting unit 301, the content acquiring unit 302, the content storing unit 303, the display portion determining unit 304, the comparison image storing unit 305 shown in FIG. The photographed image display control unit 306, the timer 307, the superposition necessity determination unit 308, the content superposition control unit 309, and the like are realized.

  As shown in FIG. 7, in the content storage unit 201, the image data 55 for each content 51C is stored in advance in association with the unique call marker 51Y.

  The user causes the tablet terminal device 3T to start the client program 3P. Then, the digital camera 30g starts shooting. Then, photographing is performed every predetermined time Ta (for example, 0.1 second).

  Here, the user adjusts the position and attitude of the digital camera 30g so that the printed matter 5 is photographed. Then, an image of a part or all of the printed matter 5 is captured, and image data of the image is generated.

  The marker detection unit 301 tries to detect the calling marker 51Y from the image each time an image is captured by the digital camera 30g. That is, a marker of a predetermined format is searched from the image, and if such a marker is found, it is extracted as a calling marker 51Y. Note that the detection of the calling marker 51Y may not be performed while the user is performing an operation of enlargement or scrolling such as pinch out or drag. The same applies to the case where the position or posture of the tablet terminal device 3T is changing at a certain speed (for example, while the user pans the tablet terminal device 3T).

  When the call detection marker 51Y is detected (extracted) by the marker detection unit 301, the content acquisition unit 302 acquires the content 51C corresponding to the call marker 51Y as follows.

  The content acquisition unit 302 transmits the calling marker 51Y to the document server 2, and requests the content 51C corresponding thereto.

  Then, in the document server 2, the content transmission unit 202 calls the image data 55 corresponding to the transmitted call marker 51Y from the content storage unit 201 and transmits it to the tablet terminal device 3T.

  Then, the content acquisition unit 302 receives the image data 55 from the document server 2.

  When the content acquisition unit 302 acquires the image data 55, the content storage unit 303 temporarily stores the image data 55.

  The same call marker 51Y may be detected again by the marker detection unit 301. In this case, if the image data 55 corresponding to the calling marker 51Y is stored in the content storage unit 303, the content acquisition unit 302 does not acquire the image data 55.

  By the way, at the beginning of starting the client program 3P, the digital zoom of the digital camera 30g is set to equal magnification. On the touch panel display 30e, all or part of an image (hereinafter, referred to as "captured image 60") captured by the digital camera 30g is displayed. Which part is displayed is determined by the display part determination unit 304 as follows according to the magnification of the digital zoom and the user's operation.

  The display part determination unit 304 initially determines that the entire captured image 60 is to be displayed on the touch panel display 30e (hereinafter referred to as “display part”) when the client program 3P is activated.

  Thereafter, when the user pinches out (pinch-opens) at an arbitrary position on touch panel display 30e, display portion determining unit 304 sets the display portion to a range corresponding to the distance the finger has moved. decide. The range becomes narrower as the distance that the finger moves is longer. As described later, this range is enlarged and displayed so as to fit the entire touch panel display 30e. That is, when the pinch out is performed, the captured image 60 is enlarged and displayed at a magnification according to the distance of movement of the finger, centering on the position where the pinch out is performed.

  After that, when the user drags the touch panel display 30e in an arbitrary direction by an arbitrary distance, the display part determination unit 304 determines a part moved from the current display part by that distance in the direction as a new display part. Do.

  In the comparison image storage unit 305, a photographed image 60 when the call marker 51Y is first detected by the marker detection unit 301 is stored. Thereafter, when a pinch out is performed, an image of a display portion of the captured image 60 immediately after the pinch out is performed is stored instead of the captured image 60. Hereinafter, an image stored in the comparison image storage unit 305 is referred to as a “comparison image 61”.

  The captured image display control unit 306 controls the touch panel display 30 e so that the display portion determined by the display portion determination unit 304 in the captured image 60 is displayed on the whole.

  The timer 307 measures the time that has elapsed since there is no matching part between the comparison image 61 and a target partial image 62 described later. If there is a matching part again after there is no matching part, stop timing and reset.

  The superimposition necessity determination unit 308 detects the calling marker 51Y by the marker detection unit 301, acquires the image data 55 by the content acquisition unit 302, and then obtains a new captured image 60 by the digital camera 30g. Based on the image data 55, it is determined whether the content 51C should be displayed superimposed on the captured image 60 or not. The determination process is performed according to the procedure shown in FIG.

  If the calling marker 51Y is included in the image of the display portion of the new captured image 60 (hereinafter referred to as “target portion image 62”) (Yes in # 701). ), It is determined that the content 51C should be displayed in an overlapping manner (# 706).

  If the attention partial image 62 does not include the call marker 51Y (No in # 701), the processing is further continued as follows.

  The superposition necessity determination unit 308 determines whether there is a matching (common) portion by comparing the notable partial image 62 with the comparison image 61 (# 702).

  If there is a matching portion, the content superposition control unit 308 determines that the content 51C should be displayed overlappingly (# 706).

  If there is no matching portion (No in # 703), if the user is performing an operation of enlarging or scrolling the captured image 60 (Yes in # 704), the overlap necessity determination unit 308 Then, it is determined that the content 51C should be displayed in an overlapping manner (# 706).

  Specifically, the determination is made as follows. If the user starts to pinch out or drag the touch panel display 30e while the content 51C is displayed, the captured image 60 is enlarged or scrolled. As a result, the calling marker 51Y may not be included in the target partial image 62. If the finger is not yet away from the touch panel display 30e, the pinch out or drag is in progress. Therefore, if the finger 60 is still touching the touch panel display 30e after the captured image 60 starts to be enlarged or scrolled, the superposition necessity determination unit 308 superimposes the content 51C regardless of the presence or absence of the call marker 51Y. Determine that it should be displayed.

  If there is no part that matches the notable partial image 62 and the comparison image 61 (No in # 703), and the user has not performed an operation to magnify or scroll the captured image 60 (No in # 704), they match If a predetermined time Tb (for example, 3 seconds) has not yet elapsed since the last determination that there is a portion (No in # 705), the overlap necessity determination unit 308 displays the content 51C in an overlapping manner. If it should be, it is determined (# 706). On the other hand, if the predetermined time Tb has already passed (Yes in # 705), it is determined that the display should not be made (# 707). The predetermined time Tb can be arbitrarily set by the user.

  The process of determination is not always performed each time a new captured image 60 is obtained, and may be performed at predetermined intervals. For example, it may be performed every one second.

  The content superimposition control unit 309 causes the image data 55 to be displayed so that the content 51C is displayed on the target partial image 62 when the superimposition necessity determination unit 308 determines that the content 51C should be displayed in an overlapping manner. Based on the touch panel display 30e is controlled.

  Note that if the superposition-necessary content superimposition control unit 309 determines that the content 51C should not be displayed in an overlapping manner, the image data 55 of the content 51C is deleted from the content storage unit 303.

  FIGS. 9 to 11 are diagrams for explaining an example in which the content 51 </ b> C is displayed over the target partial image 62.

  According to each of the above-described units, the image displayed on the touch panel display 30 e transitions as illustrated in, for example, FIGS. 5 and 9 to 11 according to the operation by the user.

  When the client program 3P is activated and the digital camera 30g starts shooting, the digital zoom is set to equal magnification. Here, when the position and orientation of the tablet terminal device 3T are adjusted so that the entire printed matter 5 falls within the photographing range, the entire printed matter 5 (see FIG. 5A) is displayed on the touch panel display 30e. Further, since the call marker 51Y is detected, the corresponding image data 55 is downloaded from the document server 2 to the tablet terminal 3T, and based on the image data 55, the content 51C is an image of the printed matter 5, that is, a photographed image. It is displayed on 60 as shown in FIG. Since the digital zoom is the same magnification, the captured image 60 is the target partial image 62 at this time. Then, the captured image 60 (target partial image 62) when the calling marker 51Y is first detected is stored in the comparative image storage unit 305 as the comparative image 61.

  When the user performs a pinch out on the touch panel display 30e, a part of the captured image is enlarged and displayed as shown in FIG. 9A. At this point, since the calling partial image 62 includes the calling marker 51Y, the content 51C is continuously displayed on the captured image as shown in FIG. 9B.

  Further, in accordance with the enlargement, the attention partial image 62 is newly stored in the comparison image storage unit 305 as the comparison image 61 instead of the previous comparison image 61.

  After enlargement, the captured image 60 may be scrolled by the user dragging, and the calling marker 51Y may not be included in the target partial image 62 as shown in FIG. Alternatively, due to enlargement, the call marker 51Y may not be included in the target partial image 62 without being accompanied by scrolling. In these cases, if the finger is in contact with the touch panel display 30e, that is, if the pinch out or drag operation is continued, the content 51C continues to be displayed on the captured image as shown in FIG. It is displayed superimposed on.

  Here, when the user's finger is moved away from the touch panel display 30e, the comparison image 61 and the target partial image 62 are compared, and if there is a matching (common) part, then as shown in FIG. Content 51C is displayed.

  Furthermore, when the user performs a drag operation, the photographed image 60 is scrolled and the focused partial image 62 changes as shown in FIG. 11A, and when there is no part matching the comparison image 61, the predetermined time Tb elapses. Until then, the content 51C is continuously displayed as shown in FIG. However, when the predetermined time Tb elapses, the content 51C is not displayed.

  When the user moves the tablet terminal device 3T, for example, when the tablet terminal device 3T is panned, the focused partial image 62 changes from the state shown in FIG. 10A to the state shown in FIG. 11A. There is a case. Also in this case, the content 51C is displayed until a predetermined time Tb elapses after there is no part matching the comparison image 61 and the target partial image 62. When the predetermined time Tb has elapsed, the display is ended.

  12 and 13 are flowcharts illustrating an example of the overall processing flow in the tablet terminal device 3T.

  Next, an overall processing flow when the tablet terminal device 3T displays content will be described with reference to a flowchart.

  The tablet terminal device 3T executes processing in the procedure shown in FIGS. 12 and 13 based on the client program 3P.

  The tablet terminal device 3T performs shooting and acquires a shot image 60 (# 11 in FIG. 12). When the user performs an operation such as pinch out or drag (Yes in # 12), the tablet terminal device 3T determines a portion (display portion) to be displayed on the touch panel display 30e of the captured image 60 according to the operation. To do (# 13). The initial display portion is the entire photographed image 60.

  When the captured image 60 including the call marker 51Y is acquired for the first time or enlargement operation is performed (Yes in # 14), the document browsing system 1 displays the image of the display portion of the captured image 60. The (target partial image 62) is stored as the comparison image 61 (# 15).

  Then, the tablet terminal device 3T displays the attention partial image 62 (# 16). At the same time or in parallel with this, detection of the calling marker 51Y from the photographed image 60 is tried (# 17).

  Note that the detection of the calling marker 51Y may not be performed while the user is performing an operation of enlargement or scrolling such as pinch out or drag. The same applies while the position or posture of the tablet terminal device 3T is changing at a certain speed or higher (for example, while the user pans the tablet terminal device 3T).

  If the call marker 51Y can be detected (Yes in # 18), and the image data 55 of the content 51C corresponding to the call marker 51Y is not included (No in # 19), the tablet type terminal The apparatus 3T requests the image data 55 by transmitting the calling marker 51Y to the document server 2 (# 20).

  When the image data 55 of the content 51C is received from the document server 2 (Yes in # 21 of FIG. 13), the tablet terminal device 3T stores this (# 22).

  Furthermore, the tablet terminal device 3T determines whether or not the content 51C should be displayed superimposed on the target partial image 62 (# 23). The method of determination is as described above in the flowchart of FIG.

  When it is determined that they should be superimposed (Yes in # 24), the tablet terminal device 3T displays the content 51C on the target partial image 62 (# 25). Note that if the download of the image data 55 of the content 51C has not yet been completed, step # 25 may be skipped.

  If a part matching the comparison image 61 and the target partial image 62 is not included (No in # 26), if the timer 307 has not been started yet (No in # 27), the timer 307 is started (#) 28). As a result, the time after the matching portion is not included is measured.

  On the other hand, if the comparison image 61 and the target partial image 62 include the matching part (Yes in # 26), the timer 307 is stopped if the timer 307 has already been started (Yes in # 29). Reset (# 30).

  When the predetermined time Tb is measured by the timer 307 (Yes in # 31), if the image data 55 is stored (Yes in # 32), it is deleted (# 33).

  Until the client program 3P is terminated, the processes of steps # 11 to # 33 are appropriately executed at every predetermined time Ta.

  By the way, the client program 3P is realized by the CPU 31a, so that the marker detection unit 301, the content acquisition unit 302, the content storage unit 303, the display part determination unit 304, the comparison image storage unit 305, the timer 307 are added to the glasses-type terminal device 3M. , A superposition necessity determination unit 308 and a content superposition control unit 309 are realized.

  The photographed image display control unit 306 is not realized. This is because the user can view the printed matter 5 through the lens, and it is not necessary to display the image captured by the digital camera 31g by the display 31e. Thus, zooming and scrolling of the captured image 60 also does not work. Therefore, in the glasses-type terminal device 3M, a portion that matches the comparison image 61 and the target partial image 62 is not included when the glasses-type terminal device 3M is moved by moving the head of the user or the like. When the time Tb elapses, the content 51C is not displayed. The content 51C is displayed until the predetermined time Tb elapses.

  According to the present embodiment, the content 51C is displayed even if a part of the call marker 51Y is missing from the display range without calculating the relationship between the position or orientation of the terminal device 3 and the call marker 51Y. Also, even if all parts are missing, the content 51C is displayed under certain conditions. Thereby, the visibility of the content 51C can be improved more easily than before.

  In the present embodiment, the case where only one call marker 51Y is printed on the printed matter 5 has been described as an example. However, when a plurality of print markers 51Y are printed, the document server 2 and the terminal device 3 each call the call marker 51Y. The above-described process may be performed independently. Therefore, for example, if the terminal device 3 is panned while the content 51C corresponding to a certain call marker 51Y is being played and detects another call marker 51Y, an image is generated based on the other call marker 51Y. The data 55 may be downloaded and the content 51C corresponding to the other call marker 51Y may be displayed together.

  In the present embodiment, the case where the tablet type terminal device 3T has a digital zoom function has been described as an example. When the (part) disappears, the content 51C may not be displayed after the elapse of the predetermined time Tb.

  In the present embodiment, the timer 307 measures the time after there is no matching part between the comparison image 61 and the target partial image 62. After that, when the matching part occurs again, the clocking is stopped and reset, but the clocking may be stopped and reset also when the user performs an operation of enlargement or scrolling (movement) such as pinch out or drag.

  In the present embodiment, the terminal device 3 continues to display the content 51 </ b> C as long as there is a portion that matches the comparison image 61 and the target partial image 62. If it is less, the display of the content 51C may be stopped when a predetermined time Tb elapses after the predetermined area or less than a predetermined ratio of the area of the comparison image 61.

  In addition, the configuration of the whole or each part of the document browsing system 1, the document server 2, and the terminal device 3, the processing content, the processing order, the data configuration, and the like can be appropriately changed in accordance with the spirit of the present invention.

3 Terminal equipment (AR equipment)
3M glasses-type terminal device (AR device)
3T tablet type terminal device (AR device)
30e Touch panel display (display means)
30g, 31g digital camera (shooting means)
301 Marker detection unit (detection means)
302 Content acquisition unit (reception means)
306 Photographed image display control unit (first display control means)
308 Superposition Necessity Determination Unit (Second Display Control Means)
309 Content overlay control unit (second display control means)
31e Display 5 Printed matter (object)
51Y Call marker (marker)
51C content (corresponding image)
62 Part of interest image (part of interest)

Claims (4)

An AR device that realizes AR (Augmented Reality) by displaying an image so that the marker is overlaid on an object,
Display means,
Photographing means for continuously photographing;
Detecting means for detecting the marker from a photographed image obtained by photographing by the photographing means;
Receiving means for receiving a corresponding image corresponding to the marker detected by the detecting means;
Each time the captured image is obtained by the imaging unit, a first display control unit that causes the display unit to display a portion of interest that is all or part of the captured image;
When the marker includes the marker in the target portion, the corresponding image is displayed on the display means, and the first portion is an arbitrary portion of the photographed image and is determined by the zoom operation. When the predetermined part of time has not elapsed since the marker is no longer included in the part of interest by an action by a user, the part of interest is changed. A second display unit configured to continue to display the corresponding image on the display unit when the predetermined time elapses after the marker is not included in the target portion by the operation; Display control means;
An AR device comprising: The second display control means performs an operation of changing the target portion by the user, a zoom operation, or the like until the predetermined time elapses after the marker is not included in the target portion by the operation. When the AR device is panned, the corresponding image is continuously displayed on the display means even after a predetermined time has elapsed.
The AR device according to claim 1. An AR realization method for realizing AR by displaying an image on a display means so as to overlap an object on which a marker is written,
Shoot continuously by the shooting means,
The marker is detected from a photographed image obtained by photographing by the photographing means,
Receiving a corresponding image corresponding to the detected marker;
Each time the photographed image is obtained by the photographing means, the display means is controlled to display a part of interest that is all or part of the photographed image,
When the marker includes the marker in the target portion, the corresponding image is displayed on the display means, and the first portion is an arbitrary portion of the photographed image and is determined by the zoom operation. When the predetermined part of time has not elapsed since the marker is no longer included in the part of interest by an action by a user, the part of interest is changed. The display of the corresponding image on the display unit is ended when the display unit continues to display and the predetermined time has elapsed after the marker is not included in the target portion by the operation.
AR realization method characterized by this. A computer program used in a computer that realizes AR by displaying an image on a display unit so as to overlap an object on which a marker is written,
On the computer
Execute a shooting process of shooting continuously by the shooting means ,
Causing the imaging means to perform detection processing for detecting the marker from a captured image obtained by performing imaging,
Causing a receiving process to receive a corresponding image corresponding to the detected marker,
The first control process is performed to control the display unit to display a target portion which is all or part of the captured image each time the captured image is obtained by the imaging unit.
When the marker includes the marker in the target portion, the corresponding image is displayed on the display means, and the first portion is an arbitrary portion of the photographed image and is determined by the zoom operation. When the predetermined part of time has not elapsed since the marker is no longer included in the part of interest by an action by a user, the part of interest is changed. A second display unit configured to continue to display the corresponding image on the display unit when the predetermined time elapses after the marker is not included in the target portion by the operation; To execute control processing,
A computer program characterized by

Images