KR101576538B1 - Apparatus for stereogram of ground plan - Google Patents

Apparatus for stereogram of ground plan Download PDF

Info

Publication number
KR101576538B1
KR101576538B1 KR1020150065969A KR20150065969A KR101576538B1 KR 101576538 B1 KR101576538 B1 KR 101576538B1 KR 1020150065969 A KR1020150065969 A KR 1020150065969A KR 20150065969 A KR20150065969 A KR 20150065969A KR 101576538 B1 KR101576538 B1 KR 101576538B1
Authority
KR
South Korea
Prior art keywords
image
virtual image
sketch
virtual
color
Prior art date
Application number
KR1020150065969A
Other languages
Korean (ko)
Inventor
최우철
Original Assignee
주식회사 아이아라
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 주식회사 아이아라 filed Critical 주식회사 아이아라
Priority to KR1020150065969A priority Critical patent/KR101576538B1/en
Application granted granted Critical
Publication of KR101576538B1 publication Critical patent/KR101576538B1/en
Priority to PCT/KR2016/000534 priority patent/WO2016182167A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G06T19/006—Mixed reality
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computer Graphics (AREA)
  • Software Systems (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Geometry (AREA)
  • Computer Hardware Design (AREA)
  • General Engineering & Computer Science (AREA)
  • Processing Or Creating Images (AREA)

Abstract

The present invention relates to an apparatus for realizing a planar drawing stereoscopic image, and more particularly, to an apparatus for realizing stereoscopic drawing of a planar drawing and displaying a 3D drawing. An embodiment of the present invention is a 2D sketch having a sketch drawing, a marker surrounding a 2D sketch drawing printed thereon, and receiving a color overlay on the 2D sketch drawing from a user; And generating a captured image by capturing a 2D sketch picture in the marker printed on the 2D plane sheet, extracting a 3D virtual image matching with the captured image, recording the captured image in color texture information in the 3D virtual image, And a user terminal for generating and displaying the added 3D virtual image.

Description

[0001] Apparatus for stereogram of ground plan [0002]

The present invention relates to an apparatus for realizing a planar drawing stereoscopic image, and more particularly, to an apparatus for realizing stereoscopic drawing of a planar drawing and displaying a 3D drawing.

AR (Augmented Reality) is one of the virtual reality in which the user views the real world viewed by the user and the virtual world having the additional information as one image. AR, a concept that complements the real world with a virtual world, uses a virtual environment created by computer graphics, but the protagonist is a real environment. Computer graphics serve to provide additional information needed for the real world. By overlapping the three-dimensional virtual image on the real image that the user is viewing, it means that the distinction between the real environment and the virtual screen becomes blurred.

Augmented reality, a concept that complements the real world with a virtual world, uses a virtual environment created by computer graphics, but the protagonist is a real environment. Computer graphics serve to provide additional information needed for the real world. It means that the distinction between the real environment and the virtual screen is blurred by overlapping the three-dimensional (3D) virtual image with the real image that the user is viewing.

Virtual reality technology allows a user to be immersed in a virtual environment, and the real environment can not be seen. However, augmented reality technology, which is a mixture of real environment and virtual objects, allows the user to see the real environment and provides better realism and additional information. For example, if you use a smartphone camera to illuminate the surroundings, information such as the location of a nearby store, phone number, and the like is displayed as a stereoscopic image.

However, in the related art, only three-dimensional (3D) virtual images registered in advance in the real-time video of the real world are overlaid and the user can not construct the three-dimensional virtual images. For example, it is impossible for each individual to color the surface of a three-dimensional virtual image, so all users can only view three-dimensional (3D) virtual images of the same color.

Korean Patent Laid-Open No. 10-2012-0136950

An object of the present invention is to provide an apparatus for realizing a three-dimensional reinforcement of a three-dimensional representation of a plan view. Further, the technical problem of the present invention is to enable each user to configure his / her own three-dimensional color.

An embodiment of the present invention is a 2D sketch having a sketch drawing, a marker surrounding a 2D sketch drawing printed thereon, and receiving a color overlay on the 2D sketch drawing from a user; And generating a captured image by capturing a 2D sketch picture in the marker printed on the 2D plane sheet, extracting a 3D virtual image matching with the captured image, recording the captured image in color texture information in the 3D virtual image, A user terminal for generating and displaying an added 3D virtual image; Wherein the user terminal comprises: a 3D virtual image database storing a plurality of 3D virtual images together with a unique number; A 2D sketch picture database storing images of the 3D virtual images viewed from the front together with respective unique numbers as 2D sketch pictures; A camera for photographing the 2D flat paper; A capture image generation unit for capturing a sketch picture in the marker of the 2D plane sheet to generate a captured image; A 3D virtual image acquisition unit for acquiring a 3D virtual image matching the 2D sketch image from the 3D virtual image database when there is a 2D sketch image having the same shape as the captured image; A color-added 3D virtual image generation unit for generating a color-added 3D virtual image by recording the above-mentioned chapter image in color texture information in a provided 3D virtual image; An image synthesizer for synthesizing and displaying the color added 3D virtual image on a background image photographed by the camera; And a plan view stereoscopic application execution unit for performing stereoscopic driving by controlling a captured image generation unit, a 3D virtual image acquisition unit, a color added 3D virtual image generation unit, and an image synthesis unit.

delete

The image combining unit may determine a size, a direction, and a position of the color-added 3D virtual image to be overlapped based on the marker recognized by the camera in real time.

Wherein the user terminal comprises: a three-dimensional location sensor for capturing three-dimensional moving coordinates of a user terminal, wherein the image combining unit is operable to change a perspective change of the color added 3D virtual image according to three- Can be displayed.

And a 3D virtual image registration server for assigning and registering a 3D virtual image for each 2D sketch picture, wherein when the 2D sketch picture having the same shape as the captured image exists, A unique number of the 2D sketch figure is transmitted to the 3D virtual image registration server, and a 3D virtual image matched with the 2D sketch image is received from the 3D virtual image registration server.

The 3D virtual image registration server includes a 3D virtual image storage unit for assigning and registering a 3D virtual image for each unique number of each 2D sketch image; And a 3D option image storage unit in which a 3D option image whose display appearance is changed by a user operation is registered for each unique number of each 2D sketch image, wherein the user terminal provides a 3D virtual image matched with the 2D sketch image And the 3D option image assigned to the 2D sketch drawing is received together with the 3D option image, and the color added 3D virtual image and the 3D option image can be synthesized and displayed on the background image photographed by the camera.

The user terminal can move the 3D virtual image or change the 3D option image when the user touches any of the 3D virtual image and the 3D option image displayed on the touch screen.

According to the embodiment of the present invention, the color painted by the user can be painted on the plane drawing to be displayed in a three-dimensional form, which can cause the user's interest. In addition, individual users can paint their own colors and solidify them to enjoy art education and hobbies.

BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a photograph showing a 2D plane drawing depicting a 2D sketch drawing according to an embodiment of the present invention; Fig.
FIG. 2 is a photograph showing a user painting a 2D sketch picture with coloring according to an embodiment of the present invention; FIG.
FIG. 3 is a photograph showing a state in which a flat-screen stereoscopic application installed in a user terminal is driven according to an embodiment of the present invention; FIG.
4 is a photograph showing a state in which a 2D flat plane is photographed through a user terminal according to an embodiment of the present invention,
5 is a photograph showing a 3D virtual image displayed on a display window of a user terminal according to an embodiment of the present invention.
6 is a photograph showing a 3D virtual image displayed on a display window of a user terminal according to an embodiment of the present invention;
FIG. 7 is a system configuration diagram of an apparatus for realizing a planar visualization according to an embodiment of the present invention; FIG.
8 is a block diagram of a configuration of a user terminal according to an embodiment of the present invention.
Figure 9 is an exemplary photograph of a function called UV in a 3D production tool.
FIG. 10 to FIG. 12 are views showing a state in which color-added 3D virtual images are superimposed on real-time background images taken by a camera according to an embodiment of the present invention.
FIG. 13 is a system configuration diagram of an apparatus for realizing a planar visualization provided with a 3D virtual image registration server according to an embodiment of the present invention; FIG.
FIG. 14 is a block diagram of a configuration of a 3D virtual image registration server according to an embodiment of the present invention; FIG.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings, in order to explain the present invention in detail so that those skilled in the art can easily carry out the present invention. . Other objects, features, and operational advantages of the present invention, including its effects and advantages, will become more apparent from the description of the preferred embodiments. It should be noted that the same reference numerals are used to denote the same or similar components in the drawings.

FIG. 1 is a photograph showing a 2D plane drawing in which a 2D sketch drawing is drawn according to an embodiment of the present invention, FIG. 2 is a photograph showing a state in which a user paints a 2D sketch drawing according to an embodiment of the present invention, FIG. 3 is a photograph showing how a stereoscopic application for plan view installed in a user terminal is driven according to an embodiment of the present invention. FIG. 4 is a view for photographing a 2D flat surface through a user terminal according to an embodiment of the present invention. FIG. 5 is a photograph showing a 3D virtual image displayed on a display window of a user terminal according to an embodiment of the present invention. FIG. 6 is a diagram illustrating a 3D virtual image displayed on a display window of a user terminal according to an embodiment of the present invention. FIG. 7 is a system configuration diagram of an apparatus for realizing a planar stereoscopic image according to an embodiment of the present invention.

In the present invention, when a real-time image photographed through a camera 110 and displayed on a screen is displayed on a screen, a specific marker is recorded in a 2D flat surface 10 on which a picture painted by a user is drawn When a specific marker is recognized in the photographed image, a 3D virtual image, which is a three-dimensional graphic model linked to a sketch drawing drawn in the marker, is displayed so that the actual image is superimposed on the photographed image of the 2D plane sheet 10.

In detail, as shown in Fig. 1, a 2D sketch drawing is printed on a 2D plane sheet 10 such as paper. For example, a sketch of a 2D character such as a dinosaur or a dog is printed on the 2D flat surface 10 as a 2D sketch figure. In addition to the sketch figure, a marker in the form of surrounding the 2D sketch figure is printed on the 2D flat sheet 10. The 2D plane sheet 10 can be sold or can be provided free of charge by printing printed sketches.

Here, the 2D sketch drawing printed on the 2D flat surface 10 is a picture registered in the user terminal 100. [ Also, a marker is to provide size, direction, and position information of a 3D virtual image, which is a three-dimensional graphic model that exists on a two-dimensional plane and is connected to the three-dimensional graphic model. Since the marker has a shape surrounding the 2D sketch picture, it can have various shapes such as a square, a prototype, and a triangle. In some cases, it is only necessary to have a shape that can be displayed and identified in a part without surrounding the 2D sketch figure.

2, the user can color a 2D sketch drawing printed on the 2D flat surface 10 using crayons, paints, color pens, highlight pens, or the like. Therefore, each user can complete the painting by coloring his / her desired color.

Then, as shown in FIG. 3, a flat-screen stereoscopic application installed in the user terminal 100 is touched and driven to display a 2D sketch picture 10 of a 2D painted background 10 painted by the user As the camera 110 of the user terminal 100. [

5, the user terminal 100 recognizes the 2D sketch drawing of the 2D flat sheet 10 painted by the user as the camera 110 of the user terminal 100 as shown in FIG. As shown in the figure, a 3D virtual image having a color matching the drawn 2D sketch image is displayed in three dimensions. Accordingly, when the user looks at the display window of the user terminal 100, the 3D virtual images associated with the 2D sketch images are superimposed on the photographed image of the 2D plane sheet 10 as shown in FIG.

Therefore, the user can paint each color and view the result as a 3D virtual image, which is a three-dimensional graphic model. Thus, it can enhance children's emotions while providing fun and excitement for children. Hereinafter, the user terminal 100 will be described in detail with reference to FIGS.

8 is a configuration block diagram of a user terminal according to an embodiment of the present invention.

The user terminal 100 of the present invention is described as an example of a smart phone as an example of a smart phone but it may be a desktop PC, a tablet PC, a slate PC, A notebook computer, a digital broadcasting terminal, a PDA (Personal Digital Assistants), a PMP (Portable Multimedia Player), a navigation, a digital camera 110, and the like. Of course, it is needless to say that the terminal to which the present invention is applicable is not limited to the above-described type, but may include both terminals capable of communicating with external devices.

For reference, a smart phone is an intelligent terminal that adds a computer support function such as internet communication and information search to a mobile phone, and can install an application desired by the user. The smart phone combines the advantages of a mobile phone and a personal digital assistant (PDA), integrating data communication functions such as schedule management, fax transmission and reception, and Internet access to mobile phone functions. In addition to being able to connect directly to the Internet using wireless Internet, it can be accessed in various ways using various browsing programs. In addition, unlike existing mobile phones, you can install, add or delete hundreds of various applications (applications) as you want, and you can create your own applications. Can be implemented.

The user terminal 100 of the present invention downloads and installs a flat-screen stereoscopic application (tentative name) from an application store server such as Google Market or Apple Store. The user terminal 100 drives the provided plan drawing stereoscopic application to capture a 2D sketch picture in a marker printed on the 2D flat surface 10 to generate a captured image and extract a 3D virtual image matched with the captured image , And creates a color added 3D virtual image by recording the chapter image in the color texture information in the 3D virtual image.

8, the user terminal 100 includes a 3D virtual image database 120, a 2D sketch image database 130, a camera 110, a captured image generating unit 150, a 3D virtual image acquiring unit 130, A color addition 3D virtual image generating unit 170, an image synthesizing unit 180, and a planar drawing stereoscopic application executing unit 190. The 3D virtual image creating unit 170 may be a 3D virtual image creating unit, In addition, it may further include a position detection sensor 140.

The 3D virtual image database 120 is a database in which a plurality of 3D virtual images are stored together with their unique numbers. Such a 3D virtual image may correspond to a three-dimensional graphic model of a three-dimensional character such as a dog, a dinosaur, and a person, and may be a variety of three-dimensional virtual images other than characters. The 3D virtual image may be registered and stored in the 3D virtual image database 120 provided in the user terminal 100. In another embodiment, the 3D virtual image may be separately stored in the 3D virtual image registration server 200 to be described later.

The 2D sketch picture database 130 is a database (DB) in which images when 3D virtual images are viewed from the front are stored as 2D sketch pictures together with respective unique numbers. Dimensional stereoscopic models of 3D virtual images are stored as 2D sketch images. For reference, an example of a storage table of the 2D sketch figure database 130 is shown in [Table 1] below.


Unique number

2D sketch illustration

Related 3D virtual images

Ox10ab

Dog 2D sketch illustration

Dog 3D Virtual Imagery

Ox10ac

Dinosaur 2D sketch illustration

Dinosaur 3D Virtual Image

Ox10ad

Car 2D sketch illustration

Automobile 3D virtual image
...
...
... ...
...
... ...
...
...

The 3D virtual image database 120 and the 2D sketch image database 130 may include a hard disk drive, an SSD drive, a flash memory, a CF card, It is a module that can input and output information such as SD card (Secure Digital Card), SM card (Smart Media Card), MMC card (Multi-Media Card) or Memory Stick May be provided in the apparatus of FIG.

The camera 110 photographs the 2D flat surface 10 at the upper front side. 2D plane sheet 10 and provides it to the captured image generation unit 150 as an image file. For reference, the camera 110 may include a lens assembly, a filter, a photoelectric conversion module, and an analog / digital conversion module. The lens assembly includes a zoom lens, a focus lens, and a compensation lens. The focus distance of the lens can be shifted under the control of the focus motor MF. The filters include an optical low pass filter and an optical low pass filter. And may include an Infra-Red Cut Filter. An optical low-pass filter removes the high-frequency component of optical noise, while the Infra-Red cut filter blocks the infrared component of incident light. The photoelectric conversion module may include an image pickup device such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal-Oxide-Semiconductor). The photoelectric conversion module converts light from the optical system (OPS) into an electrical analog signal. The analog / digital conversion module may include a CDS-ADC (Correlation Double Sampler and Analog-to-Digital Converter) device. An analog / digital conversion module (not shown) processes an analog signal from the photoelectric conversion module OEC, removes the high-frequency noise, adjusts the amplitude, and converts the analog signal into a digital signal.

The captured image generating unit 150 captures a sketch image in the marker of the 2D flat sheet 10 to generate a captured image. That is, the entire inside of the marker is captured and generated as a captured image. The captured image has color information. If nothing is painted, it will have white information, and if it is painted, it will have information about the color painted.

The 3D virtual image acquisition unit 160 determines whether there is a 2D sketch picture having the same shape as the captured image. As a result of the determination, if there is a 2D sketch picture having the same shape as the captured image, a 3D virtual image matching with the 2D sketch picture is acquired from the 3D virtual image database 120. For example, if the captured image is a puppy, and there is a puppy 2D sketch picture having the same shape as the puppy, a puppy 3D virtual image matching with the puppy 2D sketch picture is acquired from the 3D virtual image database 120.

The color-added 3D virtual image generating unit 170 records the color of the target image in the color texture information in the provided 3D virtual image to generate a 3D virtual image (hereinafter referred to as a color-added 3D virtual image) . In detail, in the 3D virtual image, which is a 3D graphic model, there is data, which is a material representing the color and texture of the 3D graphic model, and the texture data contains texture data. The surface texture data usually have image extensions (jpg, png, psd, etc.), and this surface texture data is captured through the camera 110 and put in the captured image.

9 is an exemplary photograph of a function of UV in a known 3D production tool, and the UV function enables to control how texture data is embedded in the 3D graphic model. As shown in the photograph, the 3D graphic model is made up of one surface. On the right side of FIG. 9, the surfaces laid out in the 3D coordinates can be flattened to determine which surface is placed on the surface texture.

In the present invention, the 3D graphic model is expanded in a plane using the 2D plane sheet 10, which is an off-line print, as the surface texture data, and the surface is arranged (head, body, In the first place. This structure is called mapping and can be regarded as real-time UV mapping technology. Thus, the present invention is called off-line via the camera 110.

The image combining unit 180 synthesizes the color-added 3D virtual image with the background image photographed by the camera 110, and displays the synthesized 3D virtual image on the touch screen panel. At this time, a color-added 3D virtual image is synthesized and displayed based on the marker. When the size, direction, and position information on which the color added 3D virtual image is to be drawn are set on the basis of the marker, the color added 3D virtual image is superimposed on the marker of the real time background image of the camera 110 according to the set reference . That is, the image synthesizer 180 determines the size, direction, and position of the color-added 3D virtual image on the basis of the marker recognized in real time by the camera 110 and displays the determined size.

For example, the center point of the 3D virtual image can be placed in the center of the square of the marker. In addition, the inside of the square of the marker can be divided into quadrants, and the center point of the 3D virtual image can be arranged at the center point of the first quadrant area inside the square of the marker. Similarly, the center point of the quadrant The center point of the 3D virtual image can be arranged at the center point of the third quadrant area inside the square of the marker, and the center point of the 3D virtual image can be arranged at the center point of the quadrant area of the marker in the 3D virtual The center point of the image can be arranged.

Therefore, since the 3D virtual image with color added is displayed based on the coordinate value of the marker by recognizing the 2D plane sheet 10 on which the marker is drawn, when the 2D plane sheet 10 is moved, the color added 3D virtual image also moves based on the marker So that a color added 3D virtual image appears to be integrated with the 2D flat surface 10.

For reference, a color added 3D virtual image is displayed on a display window of the user terminal 100, which may be implemented as a touch screen panel. The touch screen panel is a display window provided on the front side of the user terminal 100 to display a work screen by providing a touch screen screen capable of simultaneously performing input and display. And displays a graphical user interface (GUI) for communication with the user. When the touch panel is implemented as a touch screen panel, the keyboard of the keyboard, which is the input unit, overlaps the work screen in a graphic form. The position and transparency of the keyboard layout input window can be adjusted by the user. The touch screen panel is not only a screen display means but also an input means for sensing a touch by touch means such as a touch pen or a finger. The touch screen panel may be realized by a touch panel of any one of a resistance film type, a capacitive type, an infrared type, and an ultrasonic type.

The stereoscopic application application unit 190 provided in the user terminal 100 includes a captured image generation unit 150, a 3D virtual image acquisition unit 160, a color added 3D virtual image generation unit 170, 180) to perform the solid-state drive. That is, the planar-pattern stereoscopic application execution unit 190 captures a 2D sketch picture in a marker printed on the 2D plane sheet 10 to generate a captured image, extracts a 3D virtual image matched with the captured image, And a clip image is recorded in the color texture information in the image to generate and display a color added 3D virtual image.

In addition, the color-added 3D virtual image displayed on the user terminal 100 can stereoscopically view the color-added 3D virtual image according to the position coordinate change of the user terminal 100. For example, when the position of the user terminal 100 for photographing the 2D flat surface 10 is not changed, a color added 3D virtual image is displayed as shown in FIG. 10, and as shown in FIG. 11, When the position of the color-added 3D virtual image 100 is changed, the rear part of the color-added 3D virtual image can be displayed as much as the positional change, so that it can be displayed stereoscopically.

In addition, the color added 3D virtual image displayed on the user terminal 100 can be displayed with a perspective according to the position coordinate change of the user terminal 100. [ To this end, the user terminal 100 may include a three-dimensional position sensing sensor 140 for grasping the three-dimensional movement coordinates of the user terminal 100. The three-dimensional positioning sensor 140 may be implemented as a sensor capable of grasping three-dimensional coordinates such as a gyro sensor so that the user terminal 100 can sense movement of three-dimensional coordinates on the X, Y, and Z axes have. Accordingly, the image combining unit 180 can display the color added 3D virtual image with a perspective change according to the three-dimensional moving coordinates of the user terminal 100. [ For example, if the center point of the color added 3D virtual image is displayed in agreement with the center of the marker, if the user terminal 100 moves away from the center of the marker in the future, It is possible to reflect the 3D virtual image with the added color.

7 and 8, an example using the 3D virtual image database 120 stored in the user terminal 100 has been described. The present invention is not limited to this. As shown in FIG. 13, a 3D virtual image stored in a separate 3D virtual image registration server 200 connected to the wired / wireless communication network 300 may be received and implemented.

The wired / wireless communication network 300 transmits the unique number of the captured image captured from the user terminal 100 to the virtual image registration server, and transmits the 3D virtual image allocated to the unique number of the captured image from the virtual image registration server to the user terminal 100 To a wireless network. When the wired / wireless communication network 300 is implemented as a wireless communication network, a wireless mobile communication network including a base transceiver station (BTS), a mobile switching center (MSC), and a home location register (HLR) So that data communication can be performed. For reference, a mobile radio communication network includes, in addition to a base station (BTS), a mobile switching center (MSC), and a home location register (HLR), an access gateway (Access Gateway) Data Serving Node), and the like. In addition, when the wired / wireless communication network 300 is implemented as a wired communication network, it can be implemented as a network communication network, and data communication can be performed according to an Internet protocol such as TCP / IP (Transmission Control Protocol / Internet Protocol).

The 3D virtual image registration server 200 allocates 3D virtual images for each 2D sketch image, registers them, stores them, and provides the 3D virtual images to the user terminal 100. When there is a 2D sketch picture having the same shape as the captured image, the user terminal 100 transmits the unique number of the 2D sketch picture to the 3D virtual image registration server 200, Is supplied from the 3D virtual image registration server 200 and utilized for synthesizing the color added 3D virtual image. Hereinafter, the 3D virtual image registration server 200 will be described in detail with reference to FIG.

FIG. 14 is a configuration block diagram of a 3D virtual image registration server according to an embodiment of the present invention. The 3D virtual image registration server 200 may include a server communication unit 230, a 3D virtual image storage unit 210, and a 3D option image storage unit 220.

The server communication unit 230 supports hardware and software protocols for communication with the user terminal 100. The server communication unit 230 can perform data communication according to an Internet protocol such as TCP / IP (Transmission Control Protocol / Internet Protocol).

The 3D virtual image storage unit 210 is a memory in which a 3D virtual image is allocated and registered for each unique number of each 2D sketch image. Like 3D virtual image database 120 stored in the user terminal 100, 3D virtual images, which are three-dimensional graphic models matched to 2D sketch pictures of various characters such as dogs, dinosaurs, and automobiles, are stored. For reference, the 2D sketch drawing of the 3D option image may be displayed on the 2D plane sheet 10 or may not be displayed.

The 3D option image database is a memory in which a 3D option image whose display appearance is changed by a user operation is registered and stored for each unique number of each 2D sketch image. For example, in the case of a 2D sketch drawing of a dog, a 3D option image of a soccer ball is allocated and stored. In the case of a 2D sketch drawing of a car, a 3D option image of a traffic light may be allocated and stored. Also, as shown in FIGS. 10 to 12, a 3D option image of a tree can be allocated and stored. This 3D option image is used for realizing the augmented reality as a 3D graphic model like 3D virtual image.

The user terminal 100 is provided with a 3D option image assigned to a 2D sketch drawing when it is provided with a 3D virtual image matched with a 2D sketch drawing. The user adds a color optional 3D virtual image and a 3D option image to the camera 110 And displayed on the background image. For example, when a 3D virtual image of a dog is provided, a 3D option image of a soccer ball allotted together is received together, and a 3D option image of the soccer ball is displayed together with a 3D virtual image of the dog displayed overlaid on the background image . Likewise, when a 3D virtual image of an automobile is provided, a 3D option image of a traffic light assigned together can be received together, and a 3D optional image of a traffic light can be displayed together with a 3D virtual image of a car displayed overlaid on the background image.

The user terminal 100 can move the 3D virtual image or change the 3D option image when the user touches any of the 3D virtual image and the 3D option image displayed on the touch screen.

For example, when the user touches the 3D virtual image of the dog displayed on the touch screen of the user terminal 100, the 3D virtual image of the dog is displayed as an image to be played back or barked and displayed as audio .

In addition, when the user touches the 3D option image of the soccer ball near the 3D virtual image of the dog displayed on the touch screen of the user terminal 100, the 3D virtual image of the dog runs on the 3D option image of the soccer ball, I can kick the ball and get rid of it. In the case of a 3D option image to be eaten with a cake other than a soccer ball, it is possible to eat away the 3D optional image of the cake.

The embodiments of the present invention described above are selected and presented in order to assist those of ordinary skill in the art from among various possible examples. The technical idea of the present invention is not necessarily limited to or limited to these embodiments And various changes, modifications, and equivalents may be made without departing from the spirit and scope of the present invention.

10: 2D flat surface
100: User terminal
110: camera
120: 3D virtual image database
130: 2D sketch picture database
140: Positioning sensor
150: Capture image generation unit
160: 3D virtual image acquisition unit
170: Color added 3D virtual image generating unit
180:
200: 3D virtual image registration server

Claims (7)

A 2D plane having a sketch drawing, a marker surrounding the 2D sketch drawing printed thereon, and receiving a color overlay on the 2D sketch drawing from a user; And
A 3D virtual image matching the captured image is extracted by capturing a 2D sketch picture in the marker printed on the 2D plane sheet to record the captured image in the color texture information in the 3D virtual image, A user terminal for generating and displaying a color added 3D virtual image;
, ≪ / RTI &
The user terminal comprises:
A 3D virtual image database storing a plurality of 3D virtual images together with a unique number;
A 2D sketch picture database storing images of the 3D virtual images viewed from the front together with respective unique numbers as 2D sketch pictures;
A camera for photographing the 2D flat paper;
A capture image generation unit for capturing a sketch picture in the marker of the 2D plane sheet to generate a captured image;
A 3D virtual image acquisition unit for acquiring a 3D virtual image matching the 2D sketch image from the 3D virtual image database when there is a 2D sketch image having the same shape as the captured image;
A color added 3D virtual image generation unit for generating a color added 3D virtual image by recording the captured image in color texture information in a provided 3D virtual image;
An image synthesizer for synthesizing and displaying the color added 3D virtual image on a background image photographed by the camera; And
A 3D virtual image acquiring unit, a color-added 3D virtual image generating unit, and an image synthesizing unit to perform stereoscopic driving;
And an image processing unit for processing the stereoscopic image.
delete The image synthesizing apparatus according to claim 1,
Wherein the size, direction, and position of the color-added 3D virtual image are determined based on the marker recognized in real time by the camera, and displayed.
The method according to claim 1,
The user terminal includes a three-dimensional location sensor for detecting three-dimensional moving coordinates of the user terminal,
Wherein the image synthesizer displays a perspective change with respect to the color added 3D virtual image according to the 3D movement coordinates of the user terminal.
The apparatus as claimed in claim 1,
And a 3D virtual image registration server that registers and registers a 3D virtual image for each 2D sketch image,
Wherein the user terminal transmits a unique number of the 2D sketch figure to the 3D virtual image registration server when there is a 2D sketch figure having the same shape as the captured image and outputs the 3D virtual image matched with the 2D sketch figure to the 3D Wherein the virtual image registration server receives from the virtual image registration server.
The 3D virtual image registration server according to claim 5,
A 3D virtual image storage unit for allocating and registering a 3D virtual image for each unique number of each 2D sketch picture; And
And a 3D option image storage unit in which a 3D option image whose display appearance is changed by a user operation is registered for each unique number of each 2D sketch image,
The user terminal is provided with a 3D option image allocated to a 2D sketch drawing when the 3D virtual image matching with the 2D sketch drawing is provided, and the color added 3D virtual image and the 3D option image are captured by the camera And the background image is synthesized and displayed.
The method of claim 5,
Wherein the 3D virtual image and the 3D option image are changed when the user touches any of the 3D virtual image and the 3D option image displayed on the touch screen.
KR1020150065969A 2015-05-12 2015-05-12 Apparatus for stereogram of ground plan KR101576538B1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
KR1020150065969A KR101576538B1 (en) 2015-05-12 2015-05-12 Apparatus for stereogram of ground plan
PCT/KR2016/000534 WO2016182167A1 (en) 2015-05-12 2016-01-19 Device for implementing three-dimensionalization of planar design

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
KR1020150065969A KR101576538B1 (en) 2015-05-12 2015-05-12 Apparatus for stereogram of ground plan

Publications (1)

Publication Number Publication Date
KR101576538B1 true KR101576538B1 (en) 2015-12-21

Family

ID=55083864

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020150065969A KR101576538B1 (en) 2015-05-12 2015-05-12 Apparatus for stereogram of ground plan

Country Status (2)

Country Link
KR (1) KR101576538B1 (en)
WO (1) WO2016182167A1 (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101725194B1 (en) 2016-03-08 2017-04-12 주식회사 아이아라 Method for identifying terminal
KR101751178B1 (en) * 2016-03-22 2017-06-28 (주)디지엔터테인먼트 Sketch Service Offering System and Offering Methodh thereof
KR20200046381A (en) * 2018-10-24 2020-05-07 디에스글로벌 (주) Method and system for providing augmented reality contents based on location information and time information
KR20210034468A (en) * 2019-09-20 2021-03-30 (주)셀빅 Activity character creating method in virtual environment based on sketch character
KR102264803B1 (en) * 2019-12-12 2021-06-15 주식회사 옐로테일 Method for generating character animation through extracting a character from an image and device using thereof
US11182968B2 (en) 2017-08-16 2021-11-23 Samsung Electronics Co., Ltd. Electronic device and control method thereof

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116958332B (en) * 2023-09-20 2023-12-22 南京竹影数字科技有限公司 Method and system for mapping 3D model in real time of paper drawing based on image recognition

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20110136026A (en) * 2010-06-14 2011-12-21 주식회사 비즈모델라인 System for optimizing a augmented reality data
US20120136950A1 (en) 2010-11-16 2012-05-31 Tibco Software Inc. Compression of message information transmitted over a network
KR101470757B1 (en) * 2013-09-30 2014-12-08 세종대학교산학협력단 Method and apparatus for providing augmented reality service

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101248736B1 (en) * 2010-02-05 2013-03-28 에스케이플래닛 주식회사 Augmented reality book station based augmented reality system and method, augmented reality processing apparatus for realizing the same

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20110136026A (en) * 2010-06-14 2011-12-21 주식회사 비즈모델라인 System for optimizing a augmented reality data
US20120136950A1 (en) 2010-11-16 2012-05-31 Tibco Software Inc. Compression of message information transmitted over a network
KR101470757B1 (en) * 2013-09-30 2014-12-08 세종대학교산학협력단 Method and apparatus for providing augmented reality service

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101725194B1 (en) 2016-03-08 2017-04-12 주식회사 아이아라 Method for identifying terminal
KR101751178B1 (en) * 2016-03-22 2017-06-28 (주)디지엔터테인먼트 Sketch Service Offering System and Offering Methodh thereof
US11182968B2 (en) 2017-08-16 2021-11-23 Samsung Electronics Co., Ltd. Electronic device and control method thereof
KR20200046381A (en) * 2018-10-24 2020-05-07 디에스글로벌 (주) Method and system for providing augmented reality contents based on location information and time information
KR102199686B1 (en) 2018-10-24 2021-01-07 디에스글로벌(주) Method and system for providing augmented reality contents based on location information and time information
KR20210034468A (en) * 2019-09-20 2021-03-30 (주)셀빅 Activity character creating method in virtual environment based on sketch character
KR102343562B1 (en) * 2019-09-20 2021-12-27 (주)셀빅 Activity character creating method in virtual environment based on sketch character
KR102264803B1 (en) * 2019-12-12 2021-06-15 주식회사 옐로테일 Method for generating character animation through extracting a character from an image and device using thereof

Also Published As

Publication number Publication date
WO2016182167A1 (en) 2016-11-17

Similar Documents

Publication Publication Date Title
KR101576538B1 (en) Apparatus for stereogram of ground plan
US11880541B2 (en) Systems and methods of generating augmented reality (AR) objects
CN109584295B (en) Method, device and system for automatically labeling target object in image
CN105323252A (en) Method and system for realizing interaction based on augmented reality technology and terminal
Pucihar et al. Exploring the evolution of mobile augmented reality for future entertainment systems
JP2003533817A (en) Apparatus and method for pointing a target by image processing without performing three-dimensional modeling
CN106934777A (en) Scan image acquisition methods and device
Hoberman et al. Immersive training games for smartphone-based head mounted displays
CN111833457A (en) Image processing method, apparatus and storage medium
CN106814963A (en) A kind of human-computer interaction system and method based on 3D sensor location technologies
CN111860252A (en) Image processing method, apparatus and storage medium
US11663788B2 (en) Multi-feed context enhanced imaging
KR20180032059A (en) 3-Dimensional Contents Providing System, Method and Computer Readable Recoding Medium
WO2023155532A1 (en) Pose detection method, apparatus, electronic device, and storage medium
US20130076909A1 (en) System and method for editing electronic content using a handheld device
CN113470190A (en) Scene display method and device, equipment, vehicle and computer readable storage medium
KR101685505B1 (en) Method for generating 3d image by user's participation and system and method for edutainments service using the same
CN111640190A (en) AR effect presentation method and apparatus, electronic device and storage medium
CN111258413A (en) Control method and device of virtual object
Ortman et al. Guidelines for user interactions in mobile augmented reality
Kim et al. Design and implementation of immersive media system based on dynamic projection mapping and gesture recognition
KR101643569B1 (en) Method of displaying video file and experience learning using this
CN107580157A (en) Image processing apparatus, image processing method and program
Kim et al. Vision-based all-in-one solution for augmented reality and its storytelling applications
US20190325244A1 (en) System and method to enable creative playing on a computing device

Legal Events

Date Code Title Description
E701 Decision to grant or registration of patent right
GRNT Written decision to grant
FPAY Annual fee payment

Payment date: 20181203

Year of fee payment: 4

FPAY Annual fee payment

Payment date: 20191121

Year of fee payment: 5