JP4834116B2 - Augmented reality display device, augmented reality display method, and program - Google Patents

Description

  The present invention provides an augmented reality display device, an augmented reality display method, and a method suitable for correcting a color of a virtual object in accordance with a real world image in a technique for superimposing a virtual object on a real world image. The present invention relates to a program realized by a computer.

  In recent years, augmented reality display technology for superimposing a virtual object on a photographed real-world image has been developed and is being put to practical use as a game or communication tool. Such a technique is generally realized by a system including a camera, a monitor, and a marker indicating a position for displaying a virtual object.

  Non-Patent Document 1 discloses a marker position detection method in augmented reality display technology. By converting the vertex information of the marker in the two-dimensional space into the three-dimensional space coordinate system and further converting the coordinates into the coordinate system on the monitor, the virtual object can be displayed at an appropriate position in the real world.

Hirokazu Kato, Mark Billinghurst, Koichi Asano, Keihachiro Tachibana, Augmented Reality System Based on Marker Tracking and its Calibration, Transactions of the Virtual Reality Society of Japan, Vol.4, No.4, 1999

  However, when superimposing a real-world image and a virtual object, the color tone of the real-world image is not taken into consideration, so there is a problem that the virtual object appears too bright or too dark compared to the background. It was.

  The present invention solves the above-described problems, and an augmented reality display device and an augmented reality display method suitable for correcting the color tone of a virtual object in order to harmonize a real-world image and a virtual object image , As well as providing a program.

  An augmented reality display device according to a first aspect of the present invention includes an imaging unit, an estimation unit, a calculation unit, a conversion unit, a drawing unit, and a display unit, and is configured as follows.

  The photographing unit photographs the state of the real world where the marker is arranged.

  Here, a predetermined pattern is drawn on the marker, and it is placed in the real world. For example, it is affixed to a wall or floor like a sticker or sticker, or placed on a table for a coaster. it can.

  The pattern drawn on the marker is preferably an asymmetric pattern, for example, in order to determine the orientation. The user registers in advance an arbitrary asymmetric pattern and an object associated with the pattern, and places a marker at a place where a virtual object is to be superimposed.

  A marker is arranged in the real world, and the marker is photographed in the direction in which the user is looking. The photographing unit is provided in, for example, an HMD (Head Mounted Display) worn by the user in order to photograph the viewpoint of the user.

  The estimation unit estimates the position of the marker in the image captured by the imaging unit.

  For example, when the marker has a square shape, a square portion is searched from the captured image, and coordinate information of the vertex of the matched portion is obtained. Furthermore, the pattern drawn on the marker is matched with a previously registered marker pattern to identify the pattern.

  The calculation unit calculates a conversion parameter from the color of the pattern drawn on the marker to the color of the pattern of the marker in the image photographed by the photographing unit.

  The color information of the marker pattern registered in advance is compared with the color information of the marker pattern whose color has been changed by placing it in the real world, and the conversion parameter is calculated, for example, by taking the ratio of the two values.

  The conversion unit converts the color of the object associated with the marker using the calculated conversion parameter.

  The color information of the object associated with the marker is registered in advance, and the color information of the object is read based on the identified pattern. The read color is converted using conversion parameters.

  The drawing unit draws the object whose color has been converted at the estimated position in the image photographed by the photographing unit.

  Based on the marker pattern, the shape information of the object is read, and the object is drawn at the position of the marker coordinate received from the estimation unit with the color converted by the conversion unit.

  The display unit displays an image on which the object is drawn.

  The object image drawn by the drawing unit is superimposed on the real-world video and displayed on a monitor or the like.

  According to the present invention, it is possible to estimate a change in color of a virtual object due to placement in the real world by measuring a change in the color of a marker that is a real object, and thereby a virtual that is in harmony with the real world. An object can be displayed.

  Further, the augmented reality display device of the present invention can be configured to further include the marker.

  In the augmented reality display device of the present invention, the color of the pattern drawn on the marker can be configured to be the average or mode value of the colors of the objects associated with the marker.

  That is, it is possible to know what color the average color of the object or the color occupying the widest range changes in the real world.

  The present invention relates to a preferred embodiment of the above-described invention, and since it is possible to know what color the color used for drawing changes, it is possible to perform color conversion more harmonized with real-world images. Become.

  An augmented reality display method according to another aspect of the present invention includes an imaging unit, an estimation unit, a calculation unit, a conversion unit, a drawing unit, and a display unit, and is configured as follows.

  In the photographing process, the photographing unit photographs the state of the real world where a marker on which a predetermined pattern is drawn is arranged.

  In the estimation step, the estimation unit estimates the position of the marker in the image captured by the imaging unit.

  In the calculation step, the calculation unit calculates a conversion parameter from the color of the pattern drawn on the marker to the color of the marker pattern in the image photographed by the photographing unit.

  In the conversion step, the conversion unit converts the color of the object associated with the marker with the calculated conversion parameter.

  In the drawing step, the drawing unit draws the object whose color has been converted at the estimated position in the image photographed by the photographing unit.

  In the display step, the display unit displays an image on which the object is drawn.

  A program according to another aspect of the present invention is configured to cause a computer to function as the augmented reality display device and to cause the computer to execute the augmented reality display method.

  The program of the present invention can be recorded on a computer-readable information storage medium such as a compact disk, flexible disk, hard disk, magneto-optical disk, digital video disk, magnetic tape, and semiconductor memory.

  The above program can be distributed and sold via a computer communication network independently of the computer on which the program is executed. The information storage medium can be distributed and sold independently from the computer.

  According to the present invention, it is possible to provide an augmented reality display device, an augmented reality display method, and a program suitable for correcting a virtual object color tone in order to harmonize a real-world image and a virtual object image.

It is a mimetic diagram showing the outline composition of the typical information processor which fulfills the function of the augmented reality display device of the present invention by running the program. It is explanatory drawing which shows schematic structure of the augmented reality display apparatus of embodiment. It is a figure which shows the example of the correspondence of the pattern of the marker used in embodiment, and an object. (A), (b) is a figure which shows the example of the marker used in embodiment. It is a flowchart for demonstrating the process which each part of the augmented reality display apparatus of embodiment performs. It is a figure which shows the image | video of the real world image | photographed by the imaging | photography part. It is a figure which shows the image which superimposed the object matched with the marker image | photographed in FIG. It is a figure which shows the object which converted the color according to the color of the marker image | photographed in FIG. It is a figure which shows the image | video of the real world when illumination is turned off. It is a figure which shows the image which superimposed the object matched with the marker image | photographed in FIG. It is a figure which shows the object which converted the color according to the color of the marker image | photographed in FIG.

  Embodiments of the present invention will be described below. In the following, for ease of understanding, an embodiment in which the present invention is realized using a game information processing device will be described. However, the embodiment described below is for explanation, and the present invention is described. It does not limit the range. Therefore, those skilled in the art can employ embodiments in which each or all of these elements are replaced with equivalent ones, and these embodiments are also included in the scope of the present invention.

(Embodiment)
FIG. 1 is a schematic diagram showing a schematic configuration of a typical information processing apparatus that functions as an augmented reality display apparatus according to an embodiment of the present invention by executing a program. Hereinafter, a description will be given with reference to FIG.

  The information processing apparatus 100 includes a CPU (Central Processing Unit) 101, a ROM 102, a RAM (Random Access Memory) 103, an interface 104, a controller 105, an external memory 106, an image processing unit 107, and a DVD-ROM. (Digital Versatile Disc ROM) drive 108, NIC (Network Interface Card) 109, audio processing unit 110, monitor 111, and camera 112.

  The augmented reality display device according to the present embodiment is executed by mounting a DVD-ROM storing a game program and data in the DVD-ROM drive 108 and turning on the information processing apparatus 100 to execute the program. Is realized.

  The CPU 101 controls the overall operation of the information processing apparatus 100 and is connected to each component to exchange control signals and data. Further, the CPU 101 uses arithmetic operations such as addition / subtraction / multiplication / division, logical sum, logical product, etc. using an ALU (Arithmetic Logic Unit) (not shown) for a storage area called a register (not shown) that can be accessed at high speed. , Logic operations such as logical negation, bit operations such as bit sum, bit product, bit inversion, bit shift, and bit rotation can be performed. In addition, the CPU 101 itself is configured so that saturation operations such as addition / subtraction / multiplication / division for multimedia processing, vector operations such as trigonometric functions, etc. can be performed at a high speed, and those provided with a coprocessor. There is.

  The ROM 102 records an IPL (Initial Program Loader) that is executed immediately after the power is turned on, and when this is executed, the program recorded on the DVD-ROM is read out to the RAM 103 and execution by the CPU 101 is started. The The ROM 102 stores an operating system program and various data necessary for operation control of the entire information processing apparatus 100.

  The RAM 103 is for temporarily storing data and programs, and holds programs and data read from the DVD-ROM and other data necessary for game progress and chat communication. Further, the CPU 101 provides a variable area in the RAM 103 and performs an operation by directly operating the ALU on the value stored in the variable, or temporarily stores the value stored in the RAM 103 in the register. Perform operations such as performing operations on registers and writing back the operation results to memory.

  The controller 105 connected via the interface 104 receives an operation input performed when the user executes the game.

  The external memory 106 detachably connected via the interface 104 stores data indicating game play status (past results, etc.), data indicating game progress, and log of chat communication in the case of a network match ( Data) is stored in a rewritable manner. The user can record these data in the external memory 106 as appropriate by inputting an instruction via the controller 105.

  A DVD-ROM mounted on the DVD-ROM drive 108 stores a program for realizing the game and image data and audio data associated with the game. Under the control of the CPU 101, the DVD-ROM drive 108 performs a reading process on the DVD-ROM loaded therein, reads out necessary programs and data, and these are temporarily stored in the RAM 103 or the like.

  The image processing unit 107 processes the data read from the DVD-ROM by an image arithmetic processor (not shown) included in the CPU 101 or the image processing unit 107, and then processes the processed data on a frame memory ( (Not shown). The image information recorded in the frame memory is converted into a video signal at a predetermined synchronization timing and output to the monitor 111 connected to the image processing unit 107. Thereby, various image displays are possible.

  The image calculation processor can execute a two-dimensional image overlay calculation, a transmission calculation such as α blending, and various saturation calculations at high speed.

  In addition, when the virtual space is configured in three dimensions, polygon information that is arranged in the three-dimensional space and to which various texture information is added is rendered by the Z buffer method, and a predetermined viewpoint position is used. It is also possible to perform a high-speed execution of a calculation for obtaining a rendering image obtained by looking down at a polygon arranged in the virtual space in a predetermined line-of-sight direction.

  Further, the CPU 101 and the image arithmetic processor cooperate to draw a character string as a two-dimensional image in the frame memory or draw it on the surface of each polygon according to the font information that defines the character shape. Is possible.

  The NIC 109 is used to connect the information processing apparatus 100 to a computer communication network (not shown) such as the Internet, and is based on the 10BASE-T / 100BASE-T standard used when configuring a LAN (Local Area Network). To connect to the Internet using an analog modem, ISDN (Integrated Services Digital Network) modem, ADSL (Asymmetric Digital Subscriber Line) modem, cable television line A cable modem or the like, and an interface (not shown) that mediates between them and the CPU 101 are configured.

  The audio processing unit 110 converts audio data read from the DVD-ROM into an analog audio signal and outputs the analog audio signal from a speaker (not shown) connected thereto. Further, under the control of the CPU 101, sound effects and music data to be generated during the progress of the game are generated, and sound corresponding to this is output from the speaker.

  When the audio data recorded on the DVD-ROM is MIDI data, the audio processing unit 110 refers to the sound source data included in the audio data and converts the MIDI data into PCM data. If the compressed audio data is in ADPCM format or Ogg Vorbis format, it is expanded and converted to PCM data. The PCM data can be output by performing D / A (Digital / Analog) conversion at a timing corresponding to the sampling frequency and outputting it to a speaker.

  The monitor 111 is connected to the image processing unit 107, and displays image information through the cooperative operation of the CPU 101 and the image arithmetic processor. The monitor 111 is composed of an LCD (Liquid Crystal Display), an organic EL display (organic Electro-Luminescence display), and the like, and is used in a general display type or a wearable HMD.

  The camera 112 is connected via the interface 104, captures the real world, and converts the captured image into an electrical signal. It consists of a CCD (Charge Coupled Device) camera, a CMOS (Complimentary MOS) sensor, and the like. The camera 112 may be worn by the user in order to shoot from the user's viewpoint, or may be provided on a stationary display or the like in order to project the user.

In addition, the information processing apparatus 100 uses a large-capacity external storage device such as a hard disk so as to perform the same function as the ROM 102, the RAM 103, the external memory 106, the DVD-ROM mounted on the DVD-ROM drive 108, and the like. You may comprise.

  The information processing apparatus 100 described above corresponds to a so-called “consumer video game apparatus”, but the present invention can be realized as long as it performs image processing to display a virtual space. Therefore, the present invention can be realized on various computers such as a mobile phone, a portable game device, a karaoke apparatus, and a general business computer.

  For example, a general computer includes an CPU, a RAM, a ROM, a DVD-ROM drive, and an NIC as in the information processing apparatus 100, and an image processing unit having a simpler function than the information processing apparatus 100. In addition to having a hard disk as an external storage device, a flexible disk, a magneto-optical disk, a magnetic tape, and the like can be used. Further, not the controller 105 but a keyboard or a mouse is used as an input device.

  FIG. 2 is an explanatory diagram showing an outline configuration of an augmented reality display device that is an augmented reality display device according to one embodiment of the present invention and is realized on the information processing apparatus 100. Hereinafter, a description will be given with reference to FIG.

  The augmented reality display device 201 according to the present embodiment includes an imaging unit 203, an estimation unit 204, a calculation unit 205, a conversion unit 206, a drawing unit 207, a display unit 208, and a storage unit 209. Also, typically, the augmented reality display device 201 is provided or sold with the marker 202.

  The marker 202 is drawn in a predetermined pattern and is arranged at a place where a virtual object is displayed on the video by the user. The pattern drawn on the marker 202 and the object associated with the pattern are registered in the storage unit 209 in advance. An example in which the correspondence between the pattern of the marker 202 and the object is tabulated is shown in FIG. The pattern is asymmetric to determine the orientation of the object to be drawn. Assume that the object has a three-dimensional structure. Further, the storage unit 209 stores the color used for the marker 202, the color information of the object associated with the marker 202, and the like.

  FIG. 4 shows examples of markers. In the example shown in FIG. 4A, an L-shaped pattern is drawn on the marker, and only two colors of white and black are used. The object associated with the pattern is determined, and the direction in which the object is drawn is determined by the direction of the pattern. The average color or the mode color of the object color may be used as the color of the marker pattern. Further, as shown in FIG. 4B, several kinds of colors may be used. Further, in addition to the color information, information (character or the like) of the displayed object may be written on the marker.

  The image capturing unit 203 captures the state of the real world where the marker 202 is arranged, and sends an image signal to the estimation unit 204 and the calculation unit 205. For example, the photographing unit 203 is provided in an HMD worn by the user, and photographs the real world as viewed by the user. Therefore, the CPU 101 functions as the photographing unit 203 in cooperation with the camera 112.

  The estimation unit 204 receives the image signal captured by the imaging unit 203, estimates the position of the marker 202 on the image, and identifies the pattern drawn on the marker 202. The estimation unit 204 first searches the marker 202 from the video imaged by the imaging unit 203 and extracts a marker image. The coordinate information of the marker 202 is obtained from the extracted marker image. Next, the pattern of the extracted marker image and the pattern registered in the storage unit 209 are matched, the pattern drawn on the marker 202 is identified, and the direction of the pattern is obtained. The estimation unit 204 sends information on the estimated position of the marker 202, the identified pattern, and its orientation to the calculation unit 205 and the drawing unit 207. For example, when the real world 210 of FIG. 2 is photographed, a marker image beside the vase on the table is extracted, and the two-dimensional coordinates of the four vertices of the marker 202 are obtained. Next, the extracted marker 202 is determined to be a marker having the pattern of FIG. 4A, and the pattern of the marker 202 is estimated to face the front. Therefore, the CPU 101 cooperates with the RAM 103 and functions as the estimation unit 204.

  The calculation unit 205 receives the image signal from the imaging unit 203 and information on the position, pattern, and orientation of the marker 202 from the estimation unit 204, and refers to the color information of the marker 202 stored in the storage unit 209. To calculate the conversion parameters. For example, the calculation unit 205 acquires RGB values other than the pattern and the pattern of the marker 202 from the image signal, the position, pattern, and orientation information of the marker 202, and the pattern and pattern of the marker 202 stored in the storage unit 209. A conversion parameter is obtained by obtaining a ratio with other RGB values. In place of the RGB value, an HSB value that expresses a color by hue, saturation, and brightness may be used. The calculation unit 205 sends the obtained conversion parameter and the pattern information of the marker 202 to the conversion unit 206. Therefore, the CPU 101 functions as the calculation unit 205.

  For example, it is assumed that the marker in FIG. 4A is arranged in the real world 210 and the white portion other than the pattern is orange due to illumination or the like. Based on the image signal and information on the position and orientation of the marker 202, the RGB values of portions other than the pattern of the marker 202 are detected (for example, R value = r, G value = g, B value = b). On the other hand, the color information of the marker 202 stored in the storage unit 209 is acquired. Since the part other than the pattern of the marker 202 is originally white, R value = 255, G value = 255, and B value = 255. The ratio of the RGB values of the two is obtained (R: r / 255, G: g / 255, B: b / 255), and that value is used as the RGB value conversion parameter. When calculating the conversion parameters using the markers shown in FIG. 4B, conversion parameters for each color may be obtained, and the most suitable conversion parameter may be selected when converting the color of the object.

  The conversion unit 206 receives the conversion parameter and the pattern information obtained from the calculation unit 205, and converts the color of the object based on the conversion parameter. The conversion unit 206 reads the color information of the object associated with the received pattern information from the storage unit 209 and performs conversion using the conversion parameter on the color of the object. The converted color information is sent to the drawing unit 207. Therefore, the CPU 101 cooperates with the image processing unit 107 and functions as the conversion unit 206.

  For example, the conversion parameters are R: r / 255, G: g / 255, B: b / 255, and the RGB values of the original color of the object are R value = r ′, G value = g ′, and B value. = B ′, RGB values of the converted object are obtained as R value = (r / 255) r ′, G value = (g / 255) g ′, and B value = (b / 255) b ′. It is done. In addition, a conversion parameter is obtained from the marker in FIG. 4B, and when the original color to be converted is a color close to blue, the conversion parameter may be converted using the RGB value of the blue pattern portion. Good.

  The drawing unit 207 receives the marker position, the pattern of the identified marker 202 and information on the orientation thereof from the estimation unit 204, and the color information of the converted object from the conversion unit 206. The drawing unit 207 reads object shape information from the pattern of the marker 202 and draws the object at the coordinate position of the marker 202 with the color converted by the conversion unit 206. Further, the direction of the object is adjusted so as to correspond to the direction of the pattern of the marker 202. The drawing unit 207 sends image information of the drawn object to the display unit 208. Therefore, the CPU 101 cooperates with the image processing unit 107 and functions as the drawing unit 207.

  The display unit 208 receives the object image information from the drawing unit 207 and displays an image superimposed on the video of the real world 210 on the monitor 111. Accordingly, the CPU 101 functions as the display unit 208 in cooperation with the image processing unit 107 and the monitor 111.

  The storage unit 209 stores, for example, a list indicating the correspondence between the marker and the object in FIG. 3 and color information of the marker and the object. The information stored in the storage unit 209 is referred to by the estimation unit 204, the calculation unit 205, the conversion unit 206, and the drawing unit 207. Therefore, the CPU 101 cooperates with the RAM 103 and functions as the storage unit 209.

  FIG. 5 is a flowchart showing the flow of augmented reality display executed by the augmented reality display device 201. Hereinafter, a description will be given with reference to FIG.

  Photographing by the photographing unit 203 is started in response to an instruction to start photographing by the user (step S501). Here, FIG. 6 shows an example in which the marker of FIG. 4A registered in the storage unit 209 is projected in orange by illumination.

  The estimation unit 204 receives the image signal from the imaging unit 203 and estimates the position coordinates of the marker 202 on the real world image 610 (step S502). Next, the estimation unit 204 detects an L-shaped pattern drawn on the marker 202 and performs matching with the pattern registered in the storage unit 209. As a result, the pattern drawn on the marker 202 is identified and compared with the registered marker direction to determine the direction of the L-shaped pattern (step S503). From the marker of FIG. 4A registered in the storage unit 209, it is estimated that the marker 202 of the real-world image 610 faces the front.

  The calculation unit 205 determines the color of the marker 202, and reads color information of the marker registered in advance from the storage unit 209. In the case of the real-world image 610, the color used other than the pattern is determined to be “orange”. Further, the color other than the marker pattern registered in advance from the pattern of the marker 202 is determined to be “white”. Next, the RGB value ratio between the registered marker color and the color of the portion changed from the registered marker color is obtained, and a conversion parameter is calculated (step S504).

  The conversion unit 206 changes the color of the original object associated with the marker 202 (here, “white”) using the conversion parameter obtained by the calculation unit 205 (step S505). The color of the object associated with the marker 202 is changed to “orange”.

  The drawing unit 207 arranges the object at a predetermined position and orientation on the image from the position coordinates of the marker 202 and the pattern orientation obtained by the estimation unit 204. In the case of the marker 202, like the object 711 in FIG. 7, the marker 202 is arranged facing the front. Further, an object image is drawn based on the color information of the object obtained by the conversion unit 206 (step S506). In the case of the marker 202, it is drawn in “orange” like the object 811 in FIG.

  The display unit 208 superimposes the object image drawn by the drawing unit 207 on the real world image 610 and displays the image of FIG. 8 on the monitor 111 (step S507).

  FIG. 9 shows an example in which the markers in FIG. 4A are arranged in different directions and projected in gray under dark illumination.

  In this case, the estimation unit 204 estimates that the marker 902 is directed in the direction of 45 degrees to the left as compared to the registered marker orientation of the real-world image 910 (step S503). Next, the calculation unit 205 determines that the marker of the real-world image 910 is “gray”, and calculates a conversion parameter from the determined color and the registered marker color (step S504). The conversion unit 206 changes the color of the object associated with the marker 902 to “gray” using the conversion parameter obtained by the calculation unit 205 (step S505). The drawing unit 207 is arranged at the position of the marker 902 facing left 45 degrees like the object 1011 in FIG. 10, and based on the color information of the object obtained by the conversion unit 206, as shown in the object 1111 in FIG. 11. Is drawn in “gray” (step S506). The display unit 208 superimposes the object image drawn by the drawing unit 207 on the real world image 910, and displays the image of FIG. 11 on the monitor 111 (step S507).

  In this embodiment, since the color of the virtual object superimposed on the real world on the video is converted based on the color information of the marker arranged in the real world, it is possible to reduce the sense of incongruity in the color. In this embodiment, the color of the marker pattern is averaged or the mode value of the object color associated with the marker, thereby measuring the color that has changed when the color used for the object is arranged in the real world. Color conversion accuracy can be increased.

  As described above, according to the present invention, an augmented reality display device, an augmented reality display method, and a program suitable for correcting a virtual object color tone are provided in order to harmonize a real-world image and a virtual object image. Can be provided.

100 Information processing apparatus 101 CPU
102 ROM
103 RAM
104 Interface 105 Controller 106 External Memory 107 Image Processing Unit 108 DVD-ROM Drive 109 NIC
DESCRIPTION OF SYMBOLS 110 Audio | voice processing part 111 Monitor 112 Camera 201 Augmented reality display apparatus 202,902 Marker 203 Image pick-up part 204 Estimation part 205 Calculation part 206 Conversion part 207 Drawing part 208 Display part 209 Storage part 210 Real world 610 Real world where the marker 202 is arrange | positioned Image 710 An image 711 showing an object before color conversion associated with the marker 202 An object 810 before color conversion associated with the marker 202 An object after color conversion associated with the marker 202 is displayed Image 811 Color-converted object 910 associated with the marker 202 Real-world image 1010 in which the marker 902 is arranged Image 1011 associated with the marker 902 and displaying the object before color conversion 1011 Marker 902 Objects before color conversion Associated with the preparative 1110 marker 902, the object of the color conversion associated with the image 1111 marker 902 to display the object after the color conversion

Claims (4)

A shooting unit that shoots a state of the real world in which a marker is associated with an object, a predetermined pattern is drawn, and the marker in which the color of the pattern is the mode value of the color of the object ;
An estimation unit that estimates the position of a marker in an image captured by the imaging unit;
A calculation unit for calculating a conversion parameter from the color of the pattern drawn on the marker to the color of the marker pattern in the image captured by the imaging unit;
A conversion unit that converts the color of the object to be associated to said marker, by the calculated transformation parameters,
A drawing unit for drawing the object whose color has been converted at the estimated position in the image photographed by the photographing unit;
A display unit for displaying an image in which the object is drawn;
An augmented reality display device comprising: A marker to which the object is associated, a predetermined pattern is drawn, the color of the pattern is the mode value of the color of the object, and a marker placed in the real world,
A photographing unit for photographing the state of the real world where the marker is arranged;
An estimation unit that estimates the position of a marker in an image captured by the imaging unit;
A calculation unit for calculating a conversion parameter from the color of the pattern drawn on the marker to the color of the marker pattern in the image captured by the imaging unit;
A conversion unit that converts the color of the object to be associated to said marker, by the calculated transformation parameters,
A drawing unit for drawing the object whose color has been converted at the estimated position in the image photographed by the photographing unit;
A display unit for displaying an image in which the object is drawn;
An augmented reality display device comprising: An augmented reality display method executed in an augmented reality display device having an imaging unit, an estimation unit, a calculation unit, a conversion unit, a drawing unit, and a display unit,
Shooting in which the shooting unit is a marker to which an object is associated, a predetermined pattern is drawn, and a state of the real world is arranged in which the marker is arranged where the color of the pattern is the mode value of the color of the object Process,
An estimation step in which the estimation unit estimates a position of a marker in an image photographed by the photographing unit;
The calculation unit calculates a conversion parameter from the color of the pattern drawn on the marker to the color of the marker pattern in the image photographed by the photographing unit;
A conversion step in which the conversion unit converts the color of the object associated with the marker according to the calculated conversion parameter;
A drawing step in which the drawing unit draws the object whose color has been converted at the estimated position in the image photographed by the photographing unit;
A display step in which the display unit displays an image in which the object is drawn; and
An augmented reality display method comprising: Computer
An imaging unit that captures a state of the real world in which a marker is associated with an object , a predetermined pattern is drawn , and the marker in which the color of the pattern is the mode of the color of the object is arranged,
An estimation unit for estimating the position of the marker in the image captured by the imaging unit;
A calculation unit for calculating a conversion parameter from the color of the pattern drawn on the marker to the color of the pattern of the marker in the image captured by the imaging unit;
A conversion unit that converts the color of the object associated with the marker according to the calculated conversion parameter;
A drawing unit for drawing the object whose color has been converted at the estimated position in the image photographed by the photographing unit;
A display unit for displaying an image in which the object is drawn;
A program characterized by functioning as

Images