KR101013751B1 - Server for processing of virtualization and system for providing augmented reality using dynamic contents delivery - Google Patents

Abstract

The present invention extracts the posture information of the mobile communication terminal to the input image of the camera, through which the virtualization processing server and augmented reality providing system using a virtualization processing server and DCD content to provide augmented reality service by synthesizing a location-based virtual image to a real image to provide.

The virtualization processing server compares the reference region selection unit for selecting a reference region from the first image input from the mobile communication terminal, the second image input according to the change in the camera position of the first image and the mobile communication terminal, and compares the second image. A posture information extracting unit which extracts posture information of the mobile communication terminal by measuring the shift value; And generating location-based mashup information using location information of the mobile communication terminal and contents corresponding to the location information, and including the mashup information in the virtual object to which the movement displacement value is applied according to the posture information. It includes a synthesis information generation unit to generate.

Augmented Reality, AR (Augmented Reality), Virtual Reality, DCD (Dynamic Contents Delivery)

Description

Server for processing of virtualization and system for providing augmented reality using dynamic contents delivery}

The present invention extracts the posture information of the mobile communication terminal to the input image of the camera, and through this to a virtualization processing server and augmented reality providing system using a virtual processing server that provides augmented reality service by synthesizing a location-based virtual image to a real image It is about.

Augmented Reality (AR) is a visualization technology that synthesizes a computer-generated image, ie, virtual computer graphics, on a real image and provides the same to a user. This technique is distinguished from Virtual Reality (VR), which provides the user with only computer-generated images in that the information provided to the user is based on real-world images.

In general, augmented reality provides more realism than virtual reality technology because the reality of computer graphics is less than the actual image. In addition, by providing spatial information (spatial information) to the image of the real world to the user, it is possible to quickly transfer the on-site situation information.

Augmented reality technology is a method of calculating the position and direction of the camera by recognizing a marker of a specific pattern in the image acquired using the camera, and the position and direction of the camera using a GPS (Global Positioning System) and a gyro sensor. A technique for determining and recognizing a marker of an image acquired by determining a target or applying a projective geometry is used.

However, the tracking failure due to the occlusion between the camera and the marker by the body or the object and the rapid movement of the marker may occur. When using an inertial sensor such as a gyroscope and an accelerometer, the data update rate High tracking) is effective when tracking fast moving objects, but when moving slowly or there is no change in speed, the signal-to-noise ratio (SNR) from the sensor is low. Therefore, an error accumulates as time passes.

The present invention can extract the posture information such as the position and the direction of the camera using only the input image without using the conventional camera sensor and marker, to solve the error due to the malfunction and error of the sensor, and the cover phenomenon due to the marker and An object of the present invention is to provide an augmented reality providing system using a virtualization processing server and DCD content, which can prevent tracking failures.

In addition, the present invention generates the synthesis parameter information including the information on the virtual image to be synthesized to the real image in the virtualization processing server, the mobile communication terminal using the synthesis parameter information to synthesize the composite image synthesized with the virtual image to the real image It is an object of the present invention to provide augmented reality providing system using a virtualization processing server and DCD content, which can reduce the computational throughput of a mobile communication terminal and provide more effective augmented reality service.

In addition, the present invention by generating a virtual image to be synthesized to the actual image based on the location information of the mobile communication terminal and the location-based Dynamic Content Delivery (DDC) content corresponding to the location information, map information, news, weather, exchange rate To provide augmented reality providing system using a virtualization processing server and DCD contents that can provide augmented reality service upgraded by providing location-based information providing service with dynamic additional information that should be reflected in real time such as securities and securities. Has its purpose.

 In addition, the present invention is to extract the feature point in the measurement of the movement displacement of the first image received from the mobile communication terminal and the second image received according to the camera movement and to measure the movement displacement based on the reference region based on this, even with the input image only An object of the present invention is to provide an augmented reality providing system using a virtualization processing server and DCD content, which can efficiently extract the attitude information.

In order to achieve the above object, the virtualization processing server according to an embodiment of the present invention, the reference region selection unit for selecting a reference region from the first image input from the mobile communication terminal, and the camera position change of the first image and the mobile communication terminal A posture information extracting unit which compares the inputted second image and extracts posture information of the mobile communication terminal by measuring a shift value of the second image; And generating location-based mashup information using location information of the mobile communication terminal and contents corresponding to the location information, and including the mashup information in the virtual object to which the movement displacement value is applied according to the posture information. It includes a synthesis information generation unit to generate.

The content is characterized as being Dynamic Contents Delivery (DDC) content conforming to the Open Mobile Alliance (OMA) standard.

The reference region selecting unit extracts a feature point in units of blocks from the first image, selects the block having the largest number of feature points as the reference region, and references the reference image to the second image based on the reference region of the first image. The method determines whether the region exists and, when the reference region exists in the second image, measures the shifted displacement value of the second image to extract the posture information.

The posture information extracting unit reselects a reference region by extracting feature points in units of blocks from the second image when the reference region does not exist in the second image, and based on the reference region of the reselected second image. The posture information may be extracted.

In an augmented reality providing system using DCD content according to another embodiment of the present invention, a posture information is measured using a mobile communication terminal for generating a composite image obtained by combining a real image and a virtual image, and an input image received from the mobile communication terminal. A virtualization processing server for generating a virtual image to be synthesized with a real image based on DCD content corresponding to posture information and position information and position information of a mobile communication terminal; And a DCD relay server for requesting DCD content to a DCD content server of a DCD Dynamic Contents Delivery Contents Provider (CPD) according to the location information and transmitting the received DCD content to the virtualization processing server.

The mobile communication terminal includes a positioning unit for measuring the position information of the mobile communication terminal based on the Global Positioning System (GPS) satellite positioning service and transmitting it to the virtualization processing server.

The virtualization processing server compares the first image received from the mobile communication terminal and the second image input according to the change in the camera position of the mobile communication terminal, and measures the shifted displacement value of the second image. A posture information extraction unit for extracting posture information; And generate mashup information by using the location information of the mobile communication terminal and the DCD content corresponding to the location information, and include the mashup information in a virtual object to which the movement displacement value is applied according to the attitude information to synthesize the actual image. Synthesis information generation unit for generating the virtual image.

The DCD relay server includes a DCD interface unit that provides a mapping function between the DCD content server and a DCD content channel.

The DCD relay server may further include a content collector which receives the DCD content by requesting the DCD content server according to the location information through the DCD content channel to the DCD content server.

The DCD relay server includes a content format changer for changing the format of the content according to the DCD standard of the OMA.

According to the augmented reality providing system using the virtualization processing server and DCD content of the present invention has the following effects.

First, the present invention can extract the posture information such as the position and direction of the camera using only the input image without using the conventional camera sensor and marker, to solve the error due to the malfunction and error of the sensor, obscured by the marker There is an advantage that can prevent the phenomenon and tracking failure.

Secondly, the virtualization processing server generates synthesis parameter information including information on the virtual image to be synthesized with the real image, and the mobile communication terminal is configured to generate the synthesized image synthesized with the virtual image using the synthesis parameter information. By doing so, there is an advantage of reducing the computational throughput of the mobile communication terminal, which has been a problem, and providing an augmented reality service more effectively.

Third, map information, news, weather, exchange rate, securities, etc. are generated by generating a virtual image to be synthesized with the real image based on the location information of the mobile communication terminal and the location-based Dynamic Contents Delivery (DDC) content corresponding to the location information. The location-based information service is provided along with the dynamic additional information to be reflected in real time, which provides an upgraded augmented reality service.

 Fourth, in the movement displacement measurement of the first image received from the mobile communication terminal and the second image received according to the camera movement, feature point extraction and reference displacement based movement displacement measurement are performed, thereby making it possible to perform efficient posture information using only the input image. There is also an advantage that extraction can take place.

Advantages and features of the present invention, and methods for achieving them will be apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various forms, and only the embodiments of the present invention make the disclosure of the present invention complete and the general knowledge in the technical field to which the present invention belongs. It is provided to fully convey the scope of the invention to those skilled in the art, and the present invention is defined only by the scope of the claims. Like reference numerals refer to like elements throughout.

1 is a conceptual diagram for providing augmented reality according to an embodiment of the present invention.

As shown in FIG. 1, the mobile communication terminal 100 performs captured image transmission, data information management, and 3D image synthesis, and the virtualization processing server 200 performs image processing, posture information extraction, and synthesis parameter information. You will be creating.

In this case, the mobile communication terminal 100 includes a mobile terminal, a UMPC (Ultra Mobile PC), a PDA, and a terminal capable of embedding or mounting a camera, such as a digital camera. In addition, a wireless LAN, WCDMA, Wibro, HSDPA, Blue tooth, etc. may be used as a communication method between the mobile communication terminal 100 and the virtualization processing server 200.

In an embodiment of the present invention, the mobile communication terminal 100 generates a synthesized image obtained by synthesizing a real image and a virtual image.

In addition, the virtualization processing server 200 measures posture information by using the input image received from the mobile communication terminal 100 and actually uses the posture information and contents according to the location information and the location information of the mobile communication terminal 100. Create a virtual image to be composited into the image. Here, the virtual image is included in the synthesis parameter information and transmitted to the mobile communication terminal 100, and the mobile communication terminal 100 synthesizes the virtual image into the real image through the synthesis parameter information.

Hereinafter, the detailed principle of the present invention will be described in more detail together with the description of the components of each device.

2 is a block diagram of an image synthesizing apparatus according to an embodiment of the present invention.

As shown in FIG. 2, the image synthesizing apparatus 101 includes an image processor 110, a first transceiver 120, a positioning unit 130, an image synthesizer 140, an image output unit 150, and an image processor. 1 includes the information storage unit 160, the image synthesizing apparatus 101 is included in the mobile communication terminal 100 is implemented. Hereinafter, each component will be described in more detail.

The image processor 110 performs image processing on the image photographed by the camera of the mobile communication terminal 100, and captures the captured image and compresses the image.

The first transceiver 120 performs data transmission and reception with the virtualization processing server 200. For example, the first transceiver 120 virtualizes the compressed image of the image processor 110 and the location information (direction, angle, motion vector, etc.) of the mobile communication terminal 100 measured by the positioning unit 130 to be described later. It transmits to the processing server 200 or receives a virtual image including the synthesis parameter information from the virtualization processing server 200. Here, the composition parameter information includes information for synthesizing the location-based virtual object in the real image.

The positioning unit 130 measures location information of the mobile communication terminal 100 based on a satellite positioning service such as GPS. The latitude and longitude value of the GPS is used for interworking mashup information based on user location, which will be described later.

The image synthesizer 140 generates a synthesized image obtained by synthesizing the virtual image with the real image. That is, the image synthesizing unit 140 generates a synthetic image obtained by synthesizing the virtual image with the real image using the synthesis parameter information received from the virtualization processing server 200. At this time, two-dimensional or three-dimensional object rendering is performed.

The image output unit 150 outputs the synthesized image obtained by synthesizing the virtual object to the real image on the screen.

The first information storage unit 160 is an image captured by the camera, the location information of the mobile communication terminal 100, the IP (Internet Protocol) information of the virtualization processing server 200, the image synthesis algorithm, the synthesis parameter information Stores various data for synthesis.

3 is a block diagram of a synthesis parameter extraction apparatus according to an embodiment of the present invention.

The synthesis parameter extractor 201 includes a second transceiver 210, an image converter 220, a reference region selector 230, a posture information extractor 240, a synthesis information generator 250, and a second receiver. The information storage unit 260 is included, and the synthesis parameter extractor 201 is included in the virtualization processing server 200 and implemented.

The second transceiver 210 transmits / receives data with the mobile communication terminal 100, and receives the image and location information of the mobile communication terminal 100 from the first transceiver 120, for example. The synthesis parameter information is transmitted to 100.

Also, in another embodiment, the second transceiver 210 may receive DCD content corresponding to a DCD content channel while communicating with a dynamic content delivery server (DCD) 300 to be described later. Here, the DCD content conforms to the Open Mobile Alliance (OMA) standard, which will be described later in detail with reference to FIGS. 10 and 11.

The image converter 220 receives the input image, decompresses it, converts the input image in the form of RGB (Red, Green, Blue) into the input image in the form of HLS (Hue, Lightness, Saturation), and a reference region to be described later. Through the selector 230, accurate and stable feature point extraction is performed.

The reference region selector 230 performs feature point extraction on the input image and selects the block having the largest number of feature points as the reference region. In this case, a global feature point extraction method may be used, and a feature point extraction method using a corner point having the corner point of the image may be performed. In the case of using a corner point, if a corner point is taken from the input image corner and the distance between each point is set as a ratio, it may be determined that an image having the same ratio as that ratio is likely to be the same image.

In addition, high reliability of feature tracking can be expected when the brightness of pixels such as the edge of a building or the edge of a window frame changes sharply or the texture is clear due to a distinctive pattern. In the feature tracking, open software such as Lucas-Kanade algorithm and ARToolKit for tracking feature points using gradients of image brightness values can be used.

In addition, as described above, the block is divided vertically and horizontally in the input image to extract feature points in units of blocks, and a block having the most feature points may be selected as a reference region.

In addition, the reference region selector 230 may generate a reference region for each orientation, and the orientation region for each orientation may detect only the first azimuth as the first movement vector of the user in the mobile communication terminal 100 environment having only GPS. In order to determine the first azimuth angle to which the user is directed, using only the camera image mounted in the mobile communication terminal 100, since the bearing due to the movement of the user's camera cannot be measured at the same position in the stationary state without the change of the user's GPS. This is because it is possible to synthesize different virtual three-dimensional objects according to the shooting surface of the camera by extracting the angle viewed by the user by the GPS motion vector of the camera.

The posture information extractor 240 extracts posture information using the input image. That is, the posture information extractor 240 may include an image of the input image (hereinafter, referred to as a first image) in which the reference region is selected and an image of an input image (hereinafter, referred to as a second image) according to a camera position change of the mobile communication terminal. In this case, it is determined whether the reference region of the first image exists in the second image, and if the reference region exists in the second image, the displacement moved from the reference region of the first image is measured. Information (horizontal and vertical distances, azimuths, etc.) can be extracted. In this case, the existence of the reference region may be determined based on whether the corresponding origin is included in the second image based on the center of the reference region.

When the reference region does not exist in the second image, the posture information extractor 240 recognizes the new orientation and gives a new azimuth ID, and detects and stores the reference region again through feature point extraction. In this case, relative movement directions of the camera such as up, down, left, right, etc. are estimated based on the reference region of the last input image frame (second image).

Here, the new azimuth ID is a field-of-view (FOV) of the input image that can be viewed by the camera, and thus can be defined as an ID for managing a plurality of image planes displayed according to the angle of view.

In addition, when the reference region exists in the second image, the posture information extractor 240 extracts the posture information by measuring the displacement moved from the reference region of the first image, and uses the same to calculate the object composition displacement to be applied to the object. Calculate Here, the object composition displacement refers to the movement displacement of the virtual object to be synthesized in the real image, and refer to FIG. 7.

The composite information generation unit 250 generates mashup information using the location information of the mobile communication terminal 100 and the contents according to the location information, and also applies the displacement value (object compound displacement value) according to the posture information. The mashup information is included in the virtual image to be synthesized in the real image. The synthesis information generation unit 250 includes the virtual image in the synthesis parameter information and transmits the virtual image to the virtualization processing server 200.

Here, the mashup information is location-based information obtained by using location information to extract content related to the region where the image is captured. The content includes map information, traffic information, real estate information, local information, restaurants, accommodations, gas stations, banks, institutions, etc. Information may be included and may be provided as additional information along with various virtual objects.

The virtual object may be a two-dimensional or three-dimensional image, a video medium, an auditory medium such as music or sound effects, or a tactile medium.

In addition, when the additional information related to the area where the image is captured is extracted through the location information, the additional information may be changed in real time, and thus, it is necessary to generate more accurate mashup information. For example, information such as map information, news, weather, etc. may be changed in real time, and when the map information is changed, the real time map information is generated so that the changed map information in real time is matched with the region where the image is captured and generated as mashup information. Must be delivered to the virtualization processing server 200.

Therefore, the content may be DCD content in another embodiment, and real-time DCD content corresponding to location information may be transmitted from the DCD relay server 300 to be used for generating mashup information. This will be described in more detail with reference to FIGS. 10 and 11.

The second information storage unit 260 may include posture information, an image processing algorithm, an image received from the mobile communication terminal 100, a lookup table indicating a calibrated average angle according to the pixel moving distance, a virtual object, location information, content information, and the like. Stores various information required for generating synthesis parameter information.

Here, in another embodiment, the second information storage unit 260 may include subscriber information of the mobile communication terminal 100, and the subscriber information of the mobile communication terminal 100 may include a registered DCD content channel, terminal information, and user ID. It includes information such as.

Therefore, when the virtualization processing server 200 includes the subscriber information of the mobile communication terminal 100 in the location information and transmits it to the DCD relay server 300, the DCD relay server 300 to be described later is registered by the user through the corresponding information. The DCD content corresponding to the content channel is searched and provided to the virtualization processing server 200 through a data push method, and the virtualization processing server 200 generates mashup information including the corresponding DCD content.

Figure 4 is a flow chart of a method for providing augmented reality according to an embodiment of the present invention.

First, the camera of the mobile communication terminal 100 is initialized, and when the user captures an image by using the camera, the image is captured, and the compression is performed on the corresponding image and transmitted to the virtualization processing server 200 (S11 to S14). . In this case, the position information (bearing, angle, initial movement vector, etc.) of the mobile communication terminal 100 measured based on a satellite positioning service such as GPS may be transmitted together.

Next, the virtualization processing server 200 performs decompression and image processing on the received input image (S15). Here, the virtualization processing server 200 may perform an image processing process of converting an RGB input image into an HLS input image.

The virtualization processing server 200 generates synthesis parameter information including the virtual image using the input image, the location information, and the content corresponding to the location information (S16).

In this case, a detailed process of generating the synthesis parameter information will be described in detail with reference to FIG. 5.

Next, the virtualization processing server 200 transmits the generated synthesis parameter information to the mobile communication terminal 100 (S17).

Then, the mobile communication terminal 100 receives the synthesis parameter information and generates a composite image synthesized with the virtual image to the real image through the object rendering (S18 ~ S19).

Thereafter, the generated synthesized and augmented image is output on the screen (S20).

5 illustrates a detailed process of generating synthesis parameter information according to an embodiment of the present invention. 6 illustrates an example of selecting a reference region by extracting feature points, and FIG. 7 is a conceptual diagram of measuring movement displacement of a virtual object when a camera moves.

Referring to FIG. 5, a detailed process of generating the synthesis parameter will be described. First, the reference region selecting unit 230 receiving the input image performs feature point extraction on the input image and selects the reference region through the input region (S51).

In this case, as shown in FIG. 6, when the input image is in the RGB form, the image converter 220 converts the input image into the HLS form, and the reference region selector 230 performs per-block feature point extraction on the input image. The block having the largest number of feature points among the corresponding blocks is selected as the reference area 40.

Next, posture information extraction is performed on frames of the input image (first image) including the (initial) reference region and the next input image (second image) according to the movement of the camera. In operation S52, the controller determines whether the reference region of the first image exists in the second image.

In this case, when the reference region does not exist in the second image, the posture information extractor 240 recognizes the new orientation as a new orientation (S53), and reselects the reference region by extracting the feature point from the second image. It goes through the process (S54).

Meanwhile, when the reference area exists in the second image, the posture information extractor 240 extracts the posture information of the mobile communication terminal by measuring the shifted displacement (that is, object synthesis displacement) of the second image (S55). Here, the object composition displacement refers to the displacement value of the virtual object to be synthesized in the real image.

Specifically, as shown in FIG. 7, the displacement of the virtual object may be measured based on the reference area.

More specifically, (a) of FIG. 7 is an original object, (b) is an object to which ∠Op is applied, (c) is an object to which ∠Ot is applied, and (d) is a conceptual view of an object to which ΔS is applied. As shown. Where ∠Op is the horizontal rotation angle of the object, ∠Ot is the vertical rotation angle of the object, and ΔS is the variation ratio (A1 / A2) with respect to the original size of the object (A1) and the size (A2) after camera movement. The object composition displacement values ∠Op,, Ot, and ΔS are applied to rotation and size conversion of the virtual object according to the distance and time at which the camera photographs the object.

In addition, the lookup table of Table 1 representing a calibrated average angle according to horizontal and vertical pixel moving distances around the reference area may be used to apply to the virtual object.

(Table 1)

Input: Horizontal pixel travel Output: Calibrated Average Angle 0 0 30 5 60 15 90 30 120 50 … Create new azimuth after 75 ~ range Input: vertical pixel travel Output: Calibrated Average Angle 0 0 -20 -3 -40 -9 -60 -18 -80 -30 -100 or more -45 ~ Exceeded upper and lower limits when out of range

In Table 1, when the horizontal or vertical pixel movement distance is input as the input value, the calibrated average angle is calculated as an output value. Therefore, by applying a rotation transformation (up, down, left, right) to the 3D virtual object with the output value of the lookup table, it is possible to generate a 3D virtual object image according to the viewing field of the camera. At this time, the pixel shift distance input values not shown in Table 1 are determined by the interpolation method.

Next, the synthesis information generation unit 250 generates mashup information by using the location information received from the mobile communication terminal 100 and the content corresponding thereto (S56). Here, the mashup information may include map information and various additional information related to the region where the input image is captured, that is, geographic information of the camera imaging point, surrounding information such as height, type, and number of floors of the facility.

At this time, in another embodiment, the synthesis information generator 250 may generate mashup information using location-based DCD content (for example, real-time map information and traffic information) provided from the DCD relay server 300. This will be described later with reference to FIGS. 10 and 11.

Then, the synthesis information generation unit 250 includes the mashup information in the virtual object to which the object composition displacement value is applied, generates a virtual image to be synthesized into the real image, and includes it in the synthesis parameter information (S57). Here, the synthesis parameter information includes the virtual image including the virtual object and the location-based mashup information, and the parameter information for synthesizing the virtual image into the real image, so that the mobile communication terminal 100 then synthesizes the parameter information. Through it, you can easily create a composite image.

On the other hand, the composite image generated by the mashup information generation and the virtual object is described with reference to Figures 8 and 9 to be described later.

8 illustrates an example of a mashup information generation process according to an embodiment of the present invention. FIG. 9 illustrates an output screen of a composite image in which a virtual object is synthesized with a real image through the synthesis parameter information according to an embodiment of the present invention.

As shown in (a) of FIG. 8, the virtualization processing server 200 may obtain map information as location information received from the mobile communication terminal 100 and contents corresponding thereto. In this case, the map information may include a map image included in the location-based DCD content received from the DCD (Dynamic Contents Delivery) server to be described later.

Then, as shown in (b) of FIG. 8, the virtualization processing server 200 extracts a key marker from the map information and matches the location of the user using the location information, and then (c) of FIG. As shown in Fig. 1, rotation conversion by azimuth is performed, and a key color for image synthesis is inserted as shown in (d) of FIG.

Thereafter, the location-based mashup information generated as described above is generated as a virtual image along with various other peripheral information, and the virtual image is included in the synthesis parameter information and transmitted to the mobile communication terminal 100.

Then, as shown in FIG. 9, the mobile communication terminal 100 synthesizes the virtual image to the real image through the synthesis parameter information and displays the synthesized image on the screen of the mobile communication terminal 100 as an augmented composite image.

Accordingly, the composite image in which the virtual image is augmented with the actual image is provided on the screen of the mobile communication terminal 100. At this time, the user interface 800 is displayed to perform a command according to the user's input.

As such, the augmented reality service in which the virtual object is augmented with only the input image may be provided.

10 is a conceptual diagram for providing DCD content according to another embodiment of the present invention.

The Open Mobile Alliance (OMA), a standardization organization, is pushing for standardization of Dynamic Content Delivery (DCD) in order to efficiently deliver various contents to users in a mobile environment.

In another embodiment of the present invention, the DCD (Dynamic Contents Delivery) server 300 is a relay server between the DCD content server 400 and the virtualization processing server 200 of the DCD CP (Dynamic Contents Delivery Contents Provider). By configuring, for example, in the case of information that needs to be updated in real time, such as map information, the DCD relay server 300 receives the real-time information from the DCD content server 400 that provides the map image and various data information related thereto, and then virtualizes it again. By transmitting to the server 200, the location-based mashup information can be generated as the latest information and provided to the user's mobile communication terminal 100.

For example, the DCD relay server 300 is virtualized by receiving map information, news, weather, and various other information in real time by interworking with the DCD content server 400 (s) providing content corresponding to a content channel to which the user subscribes. It may be transmitted to the processing server 200. In this case, the DCD relay server 300 has a platform structure according to the Open Mobile Alliance (OMA) standard.

Hereinafter, the components of the DCD relay server 300 will be described in more detail with reference to FIG. 11.

11 is a block diagram of a DCD relay server according to another embodiment of the present invention.

The DCD relay server 300 includes a DCD interface 310, a content collector 320, a content format changer 330, a content manager 340, and a content delivery unit 350. Hereinafter, each component of the DCD relay server 300 will be described.

The DCD interface unit 310 provides a channel mapping function for the DCD content channel between the DCD content server 400 and the DCD relay server 300, so that the content collection unit 320 to be described below uses the corresponding DCD content channel. It is possible to receive DCD content according to each channel from the server 400.

In this case, the DCD interface 310 may provide a channel mapping function using subscriber information (registered DCD content channel, terminal information, ID, etc.) and location information, and move through channel mapping with the DCD content server 400. The DCD content according to the location information may be searched for from the DCD content channel to which the subscriber of the communication terminal 100 subscribes (registers) to receive the corresponding DCD content.

When the content collecting unit 320 receives a data packet including location information and subscriber information from the virtualization processing server 200, the content collecting unit 320 receives the DCD content (for example, real-time map information) according to the location information through the DCD content channel of the corresponding subscriber. Request to and receive DCD content server 400.

Here, the content collection is collected according to the content characteristics according to the DCD content channel and the DCD content server 400 of the DCD CP corresponding to the channel. Here, the DCD content is location-based DCD content including, for example, real-time map information and other additional information.

At this time, in another embodiment, the DCD relay server 300 may directly transmit DCD contents to the mobile communication terminal 100 so that information such as news and weather may be output on the screen.

Meanwhile, if it is determined that the DCD content transmission from the DCD content server 400 has failed through the DCD content channel, the content collector 320 may request the DCD content server 400 to retransmit the DCD content. When the transmission of the DCD content to the virtualization processing server 200 fails, the DCD content may be retransmitted.

The content format changing unit 330 changes the content format of the content provided from the DCD content server 400 according to the OMA (Dynamic Contents Delivery) standard of the Open Mobile Alliance (OMA). In this case, when the content transmitted from the DCD content server 400 is DCD content according to the corresponding standard, the component 330 may be omitted.

 The content manager 340 stores and manages information on the DCD content channel of the subscriber, DCD content information, and information on the DCD content server 400 of the DCD CP.

For example, when a user requests registration of N DCD content channels through the mobile communication terminal 100, the content manager 340 registers the DCD content channels according to the user's request. In this case, the DCD content channel subscription (registration) of the mobile communication terminal 100 may be quickly performed with only one request in package form for N channels providing content.

Therefore, when the virtualization processing server 200 transmits a data packet including location information and subscriber information (registered DCD content channel, terminal information, ID, etc.), the DCD relay server 300 receives the corresponding subscriber to receive the data packet. The DCD content channel is searched for and received from the DCD content server 400 through the corresponding channel. In this case, channel mapping is performed through the DCD interface 310 as described above.

In addition, the content manager 340 manages the content according to the validity period and the update period for the content received from the DCD CP.

In addition, the content manager 340 registers, deletes, and corrects its stored data. For example, the content manager 340 may modify information on the subscriber information when the information is changed, or change information on the registered DCD content channel at the request of the user or when the information expires. To perform.

In addition, the content manager 340 communicates with the DCD content server 400 in real time, monitors whether the DCD content is upgraded or not, and downloads the latest information on the DCD content in real time. It may be stored in the management unit 340.

The content delivery unit 360 transmits the DCD content received through the DCD content channel to the virtualization processing server 200. Here, the content delivery unit 360 may transmit the DCD content to the virtualization processing server 200 by various transmission methods (for example, On demand, Scheduled, Prefetched, etc.) using a push or pull method. In another embodiment, the present invention may be directly transmitted to the mobile communication terminal 100.

The above-described DCD relay server 300 and the virtualization processing server 200 may be configured as one integrated server (not shown), in this case the mobile communication terminal 100 and the integrated server and DCD content server 400 of the DCD CP Data transmission and reception between the two) can be provided augmented reality services through the provision of DCD content. Accordingly, the DCD relay server 300 may include the components of the virtualization processing server 200 of FIG. 3 described above and vice versa.

In addition, the DCD relay server 300 may directly transmit DCD content to the mobile communication terminal 100. In this case, the mobile communication terminal 100 may be equipped with a DCD module (not shown) to receive the corresponding DCD content, and A composite image including DCD content is generated in the image. For reference, the user may subscribe to the DCD content channel through the DCD module of the mobile communication terminal 100 to receive DCD content through the DCD content channel.

In addition, the first information storage unit 160 of FIG. 2, the second information storage unit 260 of FIG. 3, and the content storage unit 320 of FIG. 11 include a cache, a read only memory (ROM), and a PROM (Programmable). Non-volatile memory devices, such as ROM, Eraseable Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), and Flash memory, or volatile memory devices such as random access memory (RAM), or hard disk drives A hard disk (HDD), or a storage medium such as a CD-ROM, but is not limited thereto.

In addition, each component illustrated in FIGS. 2, 3, and 11 may be configured as a kind of 'module'. The 'module' refers to a hardware component such as software or a Field Programmable Gate Array (FPGA) or an Application Specific Integrated Circuit (ASIC), and the module plays certain roles. However, modules are not meant to be limited to software or hardware. A module may be configured to reside on an addressable storage medium and may be configured to execute one or more processors. The functionality provided by the components and modules may be combined into a smaller number of components and modules or further separated into additional components and modules.

Although embodiments of the present invention have been described above with reference to the accompanying drawings, those skilled in the art to which the present invention pertains may implement the present invention in other specific forms without changing the technical spirit or essential features thereof. I can understand that. Therefore, it should be understood that the embodiments described above are exemplary in all respects and not restrictive.

1 is a conceptual diagram for providing augmented reality according to an embodiment of the present invention.

2 is a block diagram of an image synthesizing apparatus according to an embodiment of the present invention.

Figure 3 is a block diagram of the synthesis parameter extraction apparatus according to an embodiment of the present invention.

Figure 4 is a flow chart of augmented reality providing method according to an embodiment of the present invention.

5 is a detailed process of generating synthesis parameter information according to an embodiment of the present invention.

6 is an example of selection of a reference region through feature point extraction.

7 is a conceptual diagram of measuring movement displacement of a virtual object during camera movement.

8 is an example of a mashup information generation process according to an embodiment of the present invention.

9 is an output screen of a composite image in which a virtual object is synthesized with a real image through the synthesis parameter information according to an embodiment of the present invention.

10 is a conceptual diagram for providing DCD content according to another embodiment of the present invention.

11 is a block diagram of a DCD relay server according to another embodiment of the present invention.

<Explanation of symbols on main parts of the drawings>

100: mobile communication terminal

110: image processor 120: first transceiver

130: positioning unit 140: image synthesis unit

150: image output unit 160: first information storage unit

200: virtualization processing server

210: second transceiver 220: image converter

230: reference region selection unit 240: posture information extraction unit

250: Synthetic information generation unit 260: Second information storage unit

300: DCD relay server

310: DCD interface unit 320: content collection unit

330: content format change unit 340: content management unit

350: content delivery unit

400: DCD Content Server

Claims (10)

A reference region selecting unit which selects a reference region from the first image input from the mobile communication terminal;       A posture information extraction unit for comparing the first image with a second image input according to a change in the camera position of the mobile communication terminal and extracting posture information of the mobile communication terminal by measuring a shift value of the second image;       Generate location-based mashup information by using the location information of the mobile communication terminal and the contents corresponding to the location information, and include the mashup information in a virtual object to which the movement displacement value is applied according to the posture information to synthesize the actual image. Virtualization processing server comprising a composite information generating unit for generating a virtual image to be made.       The virtualization processing server of claim 1, wherein the content is Dynamic Contents Delivery (DDC) content conforming to an Open Mobile Alliance (OMA) standard.       The display apparatus of claim 1, wherein the reference region selector extracts feature points from the first image in units of blocks, selects the block having the largest number of feature points as the reference region, and selects the reference region based on the reference region of the first image. And determining whether the reference region exists in a second image, and extracting the attitude information by measuring a shifted displacement value of the second image when the reference region exists in the second image. Process Server.       The method of claim 3, wherein the posture information extracting unit reselects the reference region by extracting feature points in units of blocks from the second image when the reference region does not exist in the second image, and reselects the second image. Virtualization processing server, characterized in that the extraction of the attitude information is made based on the reference region of the.        A mobile communication terminal for generating a composite image obtained by synthesizing a real image and a virtual image;       After measuring the shifted displacement according to the position change of the first image, which is a reference received from the mobile communication terminal, and the second image, which is an image image captured by the camera of the mobile communication terminal, from the reference region of the first image, The attitude information calculated by reflecting the displaced value by reflecting the horizontal and vertical distances and the azimuth angle is extracted, and the extracted attitude information, the location information of the mobile communication terminal, and the DCD content corresponding to the location information are extracted. A virtualization processing server for generating the virtual image to be synthesized with the real image based on the image;       Using the DCD content configured to include the DCD relay server for requesting the DCD content to the DCD content server of the DCD Dynamic Contents Delivery Contents Provider (CP) according to the location information and transmits the received DCD content to the virtualization processing server. Augmented reality providing system.       [6] The DCD of claim 5, wherein the mobile communication terminal further comprises a positioning unit for measuring the location information of the mobile communication terminal based on a GPS positioning service and transmitting the location information to the virtualization processing server. Augmented reality providing system using the content.       The method of claim 5, wherein the virtualization server compares the first image received from the mobile communication terminal with the input second image according to a change in the camera position of the mobile communication terminal and measures the shifted displacement value of the second image. A posture information extraction unit for extracting posture information of the mobile communication terminal;       Generating mashup information using location information of the mobile communication terminal and DCD content corresponding to the location information, and including the mashup information in a virtual object to which the movement displacement value is applied according to the attitude information to synthesize the actual image; Augmented reality providing system using the DCD content, characterized in that it further comprises a synthetic information generating unit for generating a virtual image.       The system of claim 5, wherein the DCD relay server further includes a DCD interface unit that provides a mapping function between the DCD content server and the DCD content channel.       The DCD relay server of claim 8, wherein the DCD relay server further comprises a content collector which receives the DCD content by requesting the DCD content according to the location information through the DCD content channel to the DCD content server. Augmented reality providing system.       10. The system of claim 5, wherein the DCD relay server further comprises a content format changer for changing the format of the content according to the DCD standard of the OMA. .

Images