KR101954926B1 - Method of providing enlarged region of image photographed by dron with high image quality and augmented reality sightseeing system therefor - Google Patents

Abstract

The present invention relates to a method for providing an enlarged region of an image photographed by a drone with a high quality image and an augmented reality sightseeing system therefor that is capable of allowing the enlarged region of the image photographed by a user′s drone to be provided with high resolution. According to the present invention, the augmented reality sightseeing system includes: a drone for photographing an image while flying and converting an original image into an image having lower resolution than the original image; a user interface unit for receiving at least one of command, selection and information of a user; a dome projector for displaying the image received from the drone as 360 degree image through a dome screen and transmitting coordinate information on a specific area if the specific area to be enlarged in the 360 degree image is selected through the user interface unit; and an experience control unit for requesting the drone to transmit the image corresponding to the coordinate information on the specific area if the coordinate information on the specific area is received from the dome projector, receiving the image corresponding to the coordinate information on the specific area extracted from the original image from the drone, and overlapping the image corresponding to the coordinate information on the specific area with the 360 degree image to display the enlarged image with high resolution onto the dome projector.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for providing an enlarged region of an image photographed by a drone at a high image quality and an augmented reality sightseeing system for the same,

The present invention relates to a high-resolution image providing method, and more particularly, to a method for providing an enlarged region of an image photographed by a drone at a high image quality and an augmented reality sightseeing system therefor.

With the development of information and communication technologies, currently available mobile devices are configured to perform various functions as well as a simple call function. Examples of such various functions include web browsing, chatting, watching TV, navigating, playing games, shooting pictures or videos through cameras, audio storage, playing music files through a speaker system, and displaying images or video have.

To this end, mobile device manufacturers are working hard to develop hardware and software to provide more complex and diverse functions.

In addition, recently introduced smart phones are equipped with a variety of complex application software such as mobile banking, mobile coupons, transportation cards, games, and security.

In addition, currently available smart phones are equipped with a high-performance processor and a high-resolution touch screen screen, thereby enabling use of various multimedia functions and applications.

Particularly, in recent years, interest in wearable devices that can be worn on the body has been increasing, and the demands of users have been diversified accordingly.

The wearable device may be a portable information device that provides a variety of computing technologies and sensing technologies while providing comfort and activity in terms of ergonomics as a computing device wearable on the wearer's body.

For example, in recent years, Samsung Electronics introduced a wristwatch-type Galaxy Gear, which provides not only a clock function but also a communication function like a mobile communication terminal.

The Drone has a pronounced "low-pitched sound". Here, a variety of meanings are derived and are used to mean, for example, "a bass" or "a bass produced by a musical instrument". In recent years, however, it has been mainly used to refer to a small unmanned aerial vehicle (UAV).

Recently, a dron has been equipped with a function to keep the balance of the gas using a plurality of small propellers (rotors) mounted and to photograph a surrounding situation by mounting a camera or the like.

Conventional drones have been mainly used for military purposes such as unmanned reconnaissance, combat support, and recently, they have also been used for commercial purposes such as delivery of goods.

However, until now, the use of the drone has been focused on simple image acquisition, and the user has provided a very limited variety of experiences that can be encountered through the drones, so that the utilization of the unmanned aerial vehicle is not maximized.

Therefore, it is required to develop a system capable of maximizing the user's flight experience by using an unmanned aerial vehicle.

Accordingly, a conventional flight experience system and method using an unmanned aerial vehicle are disclosed in Japanese Patent No. 10-1569454.

However, in the conventional experience system using the unmanned aerial vehicle, the experience control device could only receive images with a resolution of up to 1080p through a video modem. Therefore, when the user enlarges the image of 1080p resolution, it is difficult to recognize the terrain or the object because the resolution of the image is low. Accordingly, there is a demand for a method of viewing an enlarged region of an image photographed by a unmanned aerial vehicle at high resolution.

The present invention has been made in order to solve the above-mentioned problems, and it is an object of the present invention to provide an image processing apparatus and method, in which when a user desires to enlarge a specific area of an image photographed by a dron, A method for providing an enlarged region of an image at a high image quality, and an augmented reality sightseeing system therefor.

In order to achieve the above object, according to an embodiment of the present invention, the augmented reality sightseeing system using the drones according to the present invention can capture an image during flight, convert the original image into an image having a resolution lower than that of the original image A dron for transferring; A user interface device for receiving at least one of a user's command, selection, and information; A dome screen for displaying the received image from the drones in a 360-degree image, and a user selecting a specific area to be enlarged in the 360-degree image through the user interface device, Projector; And a control unit for requesting an image corresponding to coordinate information of the specific area with the drones when receiving the coordinate information of the specific area from the dome projector and displaying the image corresponding to the coordinate information of the specific area extracted from the original image And an experience control unit for controlling the display unit to display an enlarged image of a high resolution through the dome projector by overlapping an image corresponding to coordinate information of the received specific area with the 360 degree image.

Preferably, the drones divide the image into a first image and a second image sequentially, and the experience control device re-synthesizes the first image and the second image and displays the result through the dome projector .

Preferably, when the user desires to enlarge the specific area by a factor of two, the drone reduces the area ratio of the image cut out from the original image to 1/2 and transmits the reduced area ratio to the experience control device. If the user desires to enlarge the specific area by a factor of four, the drones reduce the area ratio of the image cut out from the original image to 1/3 and transmit it to the experience control device. And the image cut out from the original image is directly transferred to the experience control device.

Preferably, the user interface device is a mouse.

Preferably, the dome projector sets an enlarged area of the 360-degree image as the user clicks the wheel of the mouse.

Preferably, the dome projector adjusts magnification of the enlarged area of the 360-degree image as the user scrolls the wheel of the mouse.

Preferably, the dome projector divides an image photographed by the drones into left and right images, warps the separated left and right images according to curvature characteristics of the dome screen, Components are displayed orthogonally to each other on the dome screen.

According to another embodiment of the present invention, a method of providing an enlarged region of an image photographed by a drone in accordance with the present invention at a high image quality can be realized by converting a source image photographed by a dron during flight into an image having a resolution lower than that of the original image To a dome projector; Displaying the image received from the dome projector as a 360-degree image through a dome screen; When the user selects a specific area to be enlarged in the 360-degree image through the user interface device, the dome projector transmits coordinate information of a specific area to the experience control device; Wherein when the experience control apparatus receives coordinate information of the specific region from the dome projector, the experience control apparatus requests an image corresponding to the coordinate information of the specific region with the dragon and outputs coordinate information of the specific region extracted from the original image Receiving a corresponding image; And providing the high resolution enlarged image to the dome projector by overlapping the image corresponding to the coordinate information of the specific area received by the experience control device with the 360 degree image.

Preferably, when the user desires to enlarge the specific area by a factor of two, the drone reduces the area ratio of the image cut out from the original image to 1/2 and transmits the reduced area ratio to the experience control device. If the user desires to enlarge the specific area by a factor of four, the drones reduce the area ratio of the image cut out from the original image to 1/3 and transmit it to the experience control device. And the image cut out from the original image is directly transferred to the experience control device.

Preferably, in the step of displaying with the 360-degree image, the dome projector separates the image photographed by the drones into left and right images, and separates the separated left and right images according to curvature characteristics of the dome screen. And polarized light components between the wobbled left and right images are displayed on the dome screen, respectively.

As described above, according to the present invention, when a user wants to enlarge a specific area of a 1080p resolution image taken by a drones on a dome screen, the enlarged image is provided in high image quality, The terrain and the object can be easily recognized.

1 is a block diagram of an augmented reality sightseeing system using a drone according to an embodiment of the present invention;
FIG. 2 is a block diagram showing a schematic configuration of a drones according to an embodiment of the present invention;
FIG. 3 is a flowchart illustrating an operation of the augmented reality sightseeing system using the drone according to the embodiment of the present invention,
4 is a diagram illustrating an example of a user gesture used in the augmented reality sightseeing system using a drone according to an embodiment of the present invention;
5 is a signal flow diagram illustrating a method of providing an enlarged region of an image photographed by a drone at a high image quality according to an embodiment of the present invention.
6 is a diagram for explaining a method of setting a magnified area by a user in the embodiment of the present invention,
7 is a diagram for explaining a user's enlargement operation and area operation using a mouse in the embodiment of the present invention, and Fig.
FIGS. 8 to 10 are diagrams for explaining a method of controlling the drone by combining a movement of a user and a button input.

It is noted that the technical terms used herein are used only to describe specific embodiments and are not intended to limit the invention. It is also to be understood that the technical terms used herein are to be interpreted in a sense generally understood by a person skilled in the art to which the present invention belongs, Should not be construed to mean, or be interpreted in an excessively reduced sense. Further, when a technical term used herein is an erroneous technical term that does not accurately express the spirit of the present invention, it should be understood that technical terms that can be understood by a person skilled in the art are replaced. In addition, the general terms used in the present invention should be interpreted according to a predefined or prior context, and should not be construed as being excessively reduced.

Also, the singular forms "as used herein include plural referents unless the context clearly dictates otherwise. In the present application, the term " comprising " or " comprising " or the like should not be construed as necessarily including the various elements or steps described in the specification, Or may be further comprised of additional components or steps.

Further, the suffix " module " and " part " for components used in the present specification are given or mixed in consideration of ease of specification, and do not have their own meaning or role.

Furthermore, terms including ordinals such as first, second, etc. used in this specification can be used to describe various elements, but the elements should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component.

In the augmented reality sightseeing system using the drone according to the embodiment of the present invention, the dron converts the original image of 4K resolution into the image of 1080p resolution in consideration of the transmission capacity of the image modem of the experience control device, i.e., Lt; / RTI > In the following, " 360 degree image " means the converted 1080p resolution image.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings, wherein like reference numerals refer to like or similar elements throughout the several views, and redundant description thereof will be omitted.

In the following description, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail. It is to be noted that the accompanying drawings are only for the purpose of facilitating understanding of the present invention, and should not be construed as limiting the scope of the present invention with reference to the accompanying drawings.

1 is a configuration diagram of an augmented reality sightseeing system using a drone according to an embodiment of the present invention.

1, the augmented reality sightseeing system using a dron according to the present invention is an augmented reality sightseeing system using a dron connected to at least one dron 300 through a network 200. The augmented reality sightseeing system includes a curved display device 110 A user interface device 120, a dome projector 130, an experience control device 140, and a storage device 150, and the like.

The curved display device 110 displays images and flight information received from the drones 300.

The user interface device 120 recognizes the gesture of the user and remotely controls the drones 300 through the network 200. For this purpose, the user interface device 120 is a device for receiving at least one of a command, selection, and information of a user. In particular, the user interface device 120 recognizes and outputs a gesture of a user as a camera, And may include a plurality of input keys and function keys for setting.

For example, the user interface device 120 may be a notebook, a tablet PC, a joystick, a key pad, a dome switch, a touch pad (static / static), a touch screen A touch screen, a jog wheel, a jog switch, a jog shuttle, a mouse, a stylus pen, a touch pen, and the like.

As a modification of the user interface device 120, the user interface device 120 may include a main control device for displaying an image photographed by the drones 300 on the screen, recognizing and outputting a gesture of the user, A hold button for controlling movement and stop of the drones 300 in order to control the drones 300 more finely in cooperation with the apparatus, a synchronization button for synchronizing the movement of the user with the movement of the drones 300, An ascending button for commanding the descent of the drones 300, and a descending button for instructing the descent of the drones 300. [ Here, a scale factor is applied between the movement of the user and the movement of the drones 300, so that a large movement of the drones 300 can be reflected even to a user's small movement. The scale factor is set by double-clicking the hold button, and the set value can be displayed on the screen of the main control unit. Of course, since the acceleration and deceleration limits of the drone 300 are set, rapid start-up operations can be prevented in advance.

The main control unit also includes an Auto Scale Factor function for automatically setting a scale factor by referring to a terrain image combined with image, GPS, Digital Terrain Elevation Data (DTED), etc. .

In addition, the main steering device may display a warning alarm (or mini-map) on the screen when the user approaches a compartment such as a wall in a limited indoor space.

The main steering device is a device for measuring the speed, direction, gravity, and acceleration of a moving object and includes a wearable display device and a smart phone having an IMU (Inertial Measurement Unit) sensor such as an accelerometer, a speedometer, a geomachine, and an altimeter The auxiliary steering device may be a joystick, a key pad, a dome switch, a touch pad (static / static), a touch screen, a jog wheel, a jog switch, A jog shuttle, a mouse, a stylus pen, a touch pen, and the like. Here, the threshold value is set for each sensor of the main control unit, so that the drone 300 is stopped for a slow speed lower than a set threshold value, and a low-pass filter (LPF) ) Can be applied to prevent the drill operation of the drone 300.

The dome projector 130 displays images and flight information received from the drones 300 to the user through a dome screen. The user can designate a specific area of the 360-degree image displayed on the dome screen using the user interface device 120, for example, a mouse, and enlarge the designated specific area. Accordingly, the dome projector 130 can transmit coordinate information (for example, latitude, longitude, magnification, enlargement area size, etc.) of a specific area to be enlarged by the user to the experience control device 140.

Also, the dome projector 130 separates the image photographed by the drones 300 into left and right images, warps the separated left and right images according to the curvature characteristics of the dome screen, By displaying the polarized light components orthogonally to each other on the dome screen, a stereoscopic image can be provided to the user.

The experience control device 140 receives the image and flight information from the drones 300 and stores the received image and flight information in the storage device 150 or transmits the received image and flight information to the curved display device 110 and the dome projector 130, And displays it through the curved display device 110 or the dome projector 130. [ The experience control device 140 transmits control information corresponding to the gesture input through the user interface device 120 to the drones 300.

Meanwhile, when receiving coordinate information of a specific area from the dome projector 130, the experience control device 140 according to the present invention requests the image corresponding to the coordinate information of the specific area to the drones 300, The image corresponding to the coordinate information of the received specific region is overlapped with the 360-degree image to display a high-resolution enlarged image through the dome projector 130 can do.

The experience control device 140 sequentially receives the first image and the second image divided by the drones 300, re-synthesizes the divided first and second images, And can be displayed through the display device 110 or the dome projector 130.

For this purpose, the experience control device 140 may include a wired / wireless communication unit 141, an image processing unit 142, a drone control unit 143, and the like.

The wired and wireless communication unit 141 receives the image and the flight information from the drone 300 and transmits the received image and the flight information to the curved display device 110 or the dome projector 130. The wired / wireless communication unit 141 may transmit control information received from the user interface device 120 to the drones 300.

The wired / wireless communication unit 141 may include a wired / wireless communication module. Herein, the wireless Internet technology may be a wireless LAN (WLAN), a Wi-Fi, a wireless broadband (Wibro), a World Interoperability for Microwave Access (HSDPA), IEEE 802.16, Long Term Evolution (LTE), Wireless Mobile Broadband Service (WMBS), and the like. The Short Range Communication Technology may include Bluetooth, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, and the like. The wired communication technology may include USB (Universal Serial Bus) communication, and the like.

The image processing unit 142 converts the image received from the drones 300 through the wired / wireless communication unit 141 into an image suitable for each of the curved display device 110 and the dome projector 130.

In addition, the image processing unit 142 can generate a high-resolution enlarged image by overlapping the 360-degree image with the image corresponding to the coordinate information of the specific area received from the drones 300. In addition, the image processing unit 142 may re-synthesize the first image and the second image sequentially received from the drones 300.

The drones control unit 143 transmits control information corresponding to the gesture inputted through the user interface device 120 to the drones 300. [

In addition, the drones control unit 143 may have a number of functions including a manual operation mode, an automatic takeoff and landing function, an automatic route point flight, a manual stability enhancement flight, and an emergency return. The manual operation mode is a mode that is operated according to the supervisory command of the manager. The mode can be operated at a limited speed and a tilt angle, and urgent action is possible in an unexpected situation. The automatic takeoff and landing function can be automatic landing and landing with only the controller switch or GCS (Geographic Coordinate System) screen button without manual operation. In the automatic route point flight, it is possible to plan various scenarios by inputting a plurality of route points, to provide the same tourist scenarios by editing and storing the route, and to reduce the work intensity of the manager and to manage the operation smoothly. Manual Stability Enhancement The flight can differentiate between the flight performance limitations of tourists and managers (speed and attitude limitations). The emergency return is a function of automatically returning to a safe position based on the judgment of the dron controller 143 or the operator's judgment in an emergency mode such as a sudden malfunction or a battery low voltage situation.

In addition, the experience control device 140 can display tourist information or shopping information at a location corresponding to the image together with the image.

In addition, the experience control device 140 controls the drones 300 so that the flight range for each course stored in the storage device 150 does not deviate from the flight range of the selected course based on the set map information and the location information of the drones 300 can do. Then, the experience control device 140 can guide the sightseeing information in textures or voices when the local information is displayed.

The storage device 150 stores route information for each course of a sightseeing spot, map information of a tourist spot, tourist information for each spot of a tourist spot, dron information, dron control information corresponding to a gesture, image information received from the drones 300, do.

To this end, the storage device 150 includes a route information DB 151 for storing route information for each course of sightseeing spots, a map information DB 152 for storing map information of tourist spots, tourist information A shopping information DB 155 for storing information of a shopping mall selling special products or souvenirs, a shopping mall DB 155 for storing information of shopping malls selling special products or souvenirs, An image information DB 156 for storing images, a flight information DB 157 for storing images and flight information received from the drone 300, and a drones control information DB 158 for storing dron control information corresponding to gestures, And the like. Here, when the image of the drones 300 is stored in the memory card of the 360-degree omnidirectional camera, the image information DB 156 storing the image received from the drones 300 may be omitted.

The control server 400 monitors the flight so that the drones 300 are not in a dangerous area and generates control information when the drones 300 are abnormally operated according to the user's operation and transmits them to the drones 300. Thus, the drones 300 may control the flight based on the control information of the control server 400. [ At this time, the control server 400 can receive input information from an operation expert of the drones 300, and can generate control information for operating the drones 300 based on the input information.

FIG. 2 is a block diagram showing a schematic configuration of a drones according to an embodiment of the present invention.

2, the drone 300 includes a 360 degree omnidirectional camera 310, a wireless communication unit 320, a control unit 330, an activation unit 340, a position measurement unit 350, and an orientation measurement unit 360, And the like.

The control unit 330 receives the control information from the experience control device 140 through the wireless communication unit 320 and controls the activation unit 340 that controls the movement of the drones 300 according to the control information, Can fly. At this time, the wireless communication unit 320 may support a communication method according to the network 200. [

The control unit 330 may transmit the image photographed through the 360 degree omnidirectional camera 310 to the experience control device 140 through the wireless communication unit 320 when the drones 300 are flying. At this time, the 360 degree omnidirectional camera 310 is simple because it can capture images of all directions with one camera, and angle adjustment is not necessary.

Meanwhile, when the experience control device 140 requests an image corresponding to coordinate information of a specific region, the controller 330 extracts an image corresponding to the coordinate information of the specific region from the original image , And transmits the image corresponding to the coordinate information of the extracted specific area to the experience control device 140 through the wireless communication unit 320. [

In addition, the control unit 330 divides the photographed image into a first image and a second image using the 360 degree omnidirectional camera 310, sequentially experiences the first image and the second image through the wireless communication unit 320, To the control device 140. FIG.

The control unit 330 includes a gyro sensor, an acceleration sensor, and the like. The control unit 330 determines attitude information based on the sensed values measured through the attitude measuring unit 360 that measures the attitude of the drones 300 according to the control information And transmits it to the experience control device 140 through the silent communication unit 320. [

Further, the controller 330 generates position information on the current position of the drones 300 based on the measured position values measured by the position measuring unit 350 including a Global Positioning System (GPS) unit, 320 to the experience control device 140.

3 is a flowchart illustrating an operation of an augmented reality sightseeing system using a drone according to an embodiment of the present invention, and Fig. 4 is a view illustrating an example of a user gesture used in a augmented reality sightseeing system using a drone according to an embodiment of the present invention .

Referring to FIG. 3, the user selects one of the tour courses he wishes to fly using the user interface device 120. Then, the user interface device 120 transmits information on the selected tour course to the experience control device 140 (S310). These scenic courses are pre-set in a safe area with a lot of scenery and lots of attractions.

The experience control device 140 remotely controls the drones 300 through the network 200 according to previously stored route information corresponding to the selected tour course (S320). The drones 300 are controlled by the drones control unit 143 of the experience control device 140 so that the images and the flight information photographed through the 360 degree omnidirectional camera 310 are controlled by the experience control To the device (140).

The experience control device 140 receives the image and the flight information from the dragon 300 using the 360 degree omnidirectional camera 310 and displays the received image and the flight information through the curved display device 110 S330). The experience control device 140 can store video and flight information in the video information DB 156 and the flight information DB 157, respectively, if necessary.

For example, the curved-surface display device 110 may include a 360-degree 360-degree omnidirectional camera 310 of the drones 300, such as attitude, altitude, heading, latitude, longitude, Additional flight information can be displayed.

In addition, the curved display device 110 processes the original image photographed using the 360-degree omnidirectional camera 310, and outputs a field angle area, converts the MJPG streamer real time image into an MJPG format image It can also be output via a smart device with built-in Ethernet web browser.

In addition, it is possible to provide a separate space for the curved display device 110 so that the user can feel the feeling of proper sightseeing. For example, in the case of an image for Southeast Asia tourism, the atmosphere can be produced around the curved display device 110 by model palm trees, benches, and the like.

In addition, the experience control device 140 can display a hyperlink of a homepage that can expose local information of the region photographed by the drones 300 or purchase special products of the area on the screen of the curved display device 110 . When the user selects a desired hyperlink, the image processing unit 142 of the experience control device 140 refers to the shopping information DB 155 and connects the shop to a shopping mall selling special products or souvenirs, Provide tourist information explaining the area.

In step S330, the experience control device 140 transmits the image and the flight information photographed by the 360 degree omnidirectional camera 310 from the drones 300 to the dome projector 130 as well as the curved display device 110 Can be displayed. The user can designate a specific area of the 360-degree image displayed on the dome screen using the user interface device 120, for example, a mouse, and enlarge the designated specific area. Accordingly, the dome projector 130 can transmit coordinate information of a specific area to be enlarged by the user to the experience control device 140.

The experience control device 140 requests the image corresponding to the coordinate information of the specific region to the drones 300 when receiving the coordinate information of the specific region from the dome projector 130, The image corresponding to the coordinate information of the specific region is received and the image corresponding to the coordinate information of the received specific region is overlapped with the 360 degree image so that a high resolution enlarged image can be displayed through the dome projector 130. [

In addition, the experience control device 140 sequentially receives the first image and the second image, which are divided by the drones 300, and re-synthesizes the received first and second images, It can be displayed through the curved display device 110 or the dome projector 130.

In this state, or before step S320 and step S330, the user can move the drones 300 to his / her desired position according to need. To this end, when the user takes a gesture, the user interface device 120 recognizes the gesture using the built-in camera, and transmits the information about the recognized gesture to the experience control device 140. As described above, the experience control device 140 receives the user gesture through the user interface device 120 (S340).

Thereafter, the experience control device 140 controls the drones 300 through the network 200 remotely according to the control information according to the user gesture (S350). An example of the gesture is shown in Fig.

Referring to FIG. 4, the gesture includes a gesture for controlling the 360-degree omnidirectional camera 310 and a gesture for controlling the dragon 300, which is a flying object.

In the gesture for controlling the 360 degree omnidirectional camera 310, there is a gesture for enlarging, reducing, rotating, and selecting the camera image. Such a gesture is possible using both fingers. For example, when the two fingers are separated from each other, the camera image is enlarged, and when both fingers are closer, the camera image is reduced. When one hand is shaken to the left and right, the line of sight of the 360 degree omnidirectional camera 310 is rotated.

Gestures that manipulate the flying body drone (300) include gestures such as forward / backward, left / right, up / down, and stop. For example, if one hand holds a fist and the other hand is raised, the drones (300) move forward, one hand holds a fist, and the other hand shakes downward.

When one hand holds the fist and the other hand shakes to the left, the drone (300) turns to the left, the one hand holds the fist, and the other hand shakes to the right.

When the thumb of both hands is raised, the drones 300 are lifted, and when both hands are lowered, the drones 300 are lowered.

Further, when the both hands are raised, the drones 300 are stopped.

These various gestures can be added, deleted, and modified as needed.

5 is a signal flow diagram illustrating a method of providing an enlarged region of an image photographed by a drone at a high image quality according to an embodiment of the present invention.

Referring to FIG. 5, the dome projector 130 determines whether there is a selection input for a specific area to be enlarged in a 360-degree image displayed on the dome screen from the user through the user interface device 120 (S510) The dome projector 130 calculates an enlargement point of a specific area to be enlarged by the user in operation S520, (E.g., latitude and longitude) of a specific area to be enlarged by the user to the experience control device 140 (S530).

When receiving the coordinate information of the specific area from the dome projector 130, the experience control device 140 requests the drones 300 for the image corresponding to the coordinate information of the specific area (S540).

The drone 300 extracts an image corresponding to the coordinate information of the specific area from the original image in response to the video request of the experience control device 140 at step S550 and displays the image corresponding to the coordinate information of the extracted specific area in the experience control device 140 (S560). That is, the drone 300 cuts out the image corresponding to the coordinate information of the specific region in the original image of 4k resolution and transmits it to the experience control device 140. For example, when the user desires to enlarge a specific area by 2 times, the drone 300 reduces the area ratio of the image cut out from the original image to 1/2 and transmits it to the experience control device 140, The drone 300 transmits the area ratio of the image cut out from the original image to the experience control device 140 as it is.

The experience control device 140 generates an enlarged image of high resolution by overlapping the image corresponding to the coordinate information of the specific area received from the drones 300 with the 360 degrees image in step S570, (Step S580).

Finally, the dome projector 130 displays an enlarged image of high resolution received from the experience control device 140 (S590).

6 is a diagram for explaining a method of setting a magnified area by a user in an embodiment of the present invention.

Referring to FIG. 6, the user can adjust the enlarged area by clicking the wheel of the mouse. For example, if the user clicks the wheel of the mouse once, the first stage region 610 is set. When the user clicks the wheel of the mouse twice, a two-step area 620 is set which is larger than the first-step area 610. When the user clicks the wheel of the mouse three times, a three-step area 630 that is larger than the two-step area 620 is set. When the user clicks the wheel of the mouse four times, the first stage region 610 is set again. As described above, the dome projector 130 according to the embodiment of the present invention has a loop of area setting according to the click of the wheel of the user's mouse. Although the embodiment of the present invention has been described by merely dividing into three stages for convenience of explanation, the present invention is not limited thereto, and the loop of area setting may include at least two stages.

FIG. 7 is a diagram for explaining an enlarging operation and a region manipulation of a user using a mouse in the embodiment of the present invention. FIG.

Referring to FIG. 7A, the user can adjust the magnification of the enlarged region by scrolling the wheel of the mouse from a minimum of 1 times to a maximum of 4 times. When the user intends to enlarge the enlarged area by one time, the dragon 300 transmits the image cut out from the original image directly to the experience control device 140. When the user intends to enlarge the enlarged area twice, If the user desires to enlarge the enlarged area by three times, the drones 300 can reduce the area of the image cut out from the original image, If the user wants to enlarge the enlarged area by 4 times, the drones 300 reduce the area ratio of the image cut out from the original image to 1/4 To the experience control device (140). In other words, a description is made as to when a 4K-level camera original image is transmitted through a wired / wireless communication unit having a limited resource and is finally displayed to a user as a second image through a dome projector or the like. At this time, when the user desires to enlarge the second image, actually, the enlarged portion of the second image can be seen more clearly, but the portion that is not enlarged can not be seen, Is reduced by that much.

Referring to FIG. 7 (b), the user can designate an area in three steps by clicking the mouse wheel. For example, when the user clicks the wheel of the mouse once, the area designation 1 corresponding to the first-stage area 610 is designated. When the user clicks the wheel of the mouse twice, When the user clicks the wheel of the mouse 3 times, the area designation 3 corresponding to the 3-step area 630 larger than the 2-step area 620 is designated .

FIGS. 8 to 10 are diagrams for explaining a method of controlling the drone by combining a movement of a user and a button input.

When the user depresses the hold button while holding down the hold button, depresses the hold button three times in succession, and depresses the down button, the drones 300 are lowered. Thereafter, the drones 300 are stopped by pressing the hold button once when the user stops all operations.

When the user holds the hold button and releases the hold button halfway, the hold button is pressed twice continuously, and when the raise button is pressed, the drones 300 are raised. Thereafter, the drones 300 are stopped by pressing the hold button once when the user stops all operations.

When the user tilts the posture of the auxiliary control device while holding the synchronization button, the attitude of the drones 300 and the posture of the assistant control device are synchronized with each other based on the front screen of the drones 300 to move forward, backward, , And a right turn.

More specifically, referring to FIG. 8, if the user is in a standby state without pressing the hold button, the drones 300 perform hovering (stop flight) (S810).

Subsequently, when the user moves without pressing the hold button, the drones 300 maintain the hovering state (S820)

Subsequently, when the user presses the hold button while moving, the drones 300 move (S830).

Subsequently, when the user releases his or her hand from the hold button while moving, the drones 300 continue to move (S840).

Subsequently, when the user stops in a state in which the hold button is not pressed, the drones 300 continue to move (S850). At this time, the user is freely movable, and the movement of the user does not affect the movement of the drones 300.

Finally, when the user presses and releases the hold button in the stopped state, the drone 300 stops because the user's last state is stopped (S860).

Referring to FIG. 9, when the user advances while holding the hold button, the drones 300 advance in the direction of the user's eyes (S910).

When the user holds the hold button and moves to the right, the drones 300 move to the right of the user's sight line (S920).

When the user moves backward while holding the hold button, the drones 300 move backward in the direction of the user's sight (S930).

Referring to FIG. 10, when the user tilts the gas of the auxiliary control apparatus while pushing the synchronization button, the drones 300 advance in the direction of the user's sight (S1010).

When the user tilts the gas of the auxiliary control device to the right while pressing the synchronization button, the drones 300 move to the right of the user's sight line (S1020).

When the user tilts the gas of the auxiliary control device to the left while pressing the synchronization button, the drones 300 move to the left in the direction of the user's eyes (S1030).

The above-described method can be implemented by various means. For example, embodiments of the present invention may be implemented by hardware, firmware, software, or a combination thereof.

In the case of hardware implementation, the method according to embodiments of the present invention may be implemented in one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs) , FPGAs (Field Programmable Gate Arrays), processors, controllers, microcontrollers, and microprocessors.

In the case of an implementation by firmware or software, the method according to embodiments of the present invention may be implemented in the form of a module, a procedure or a function for performing the functions or operations described above. The software code can be stored in a memory unit and driven by the processor. The memory unit may be located inside or outside the processor, and may exchange data with the processor by various well-known means.

As described above, in the embodiment of the present invention, it is not necessary to adjust the angle of the camera in the dron by applying a 360 degree omnidirectional camera capable of shooting the 360 degree image to the dron, so that the weight of the dron can be reduced.

Further, in the embodiment of the present invention, tourists can steer the drones with simple gestures.

In addition, in the embodiment of the present invention, tourists can receive images taken while using the drones.

Further, in the embodiment of the present invention, a tourist can view a high-quality image through the curved display device.

In addition, in the embodiment of the present invention, a plurality of tourists can see a sightseeing area where a person can not go in real time through a dome projector.

Further, in the embodiment of the present invention, the area to be enlarged on the screen of the dome projector can be viewed with high quality.

In addition, in the embodiment of the present invention, it is possible to turn around the place where the individual wants to see in real time using the goggles.

In addition, in the embodiment of the present invention, it is also possible to provide the recorded image and the flight information together so that the extended service (for example, state notification, special product shopping, terrain information description, voice guidance, And screen text display, etc.).

The embodiments disclosed herein have been described with reference to the accompanying drawings. Thus, the embodiments shown in the drawings are not to be construed as limiting, and those skilled in the art will understand that the present invention can be combined with each other, and when combined, some of the components may be omitted.

Here, terms and words used in the present specification and claims should not be construed as limited to ordinary or dictionary meanings, but should be construed as meaning and concept consistent with the technical idea disclosed in the present specification.

Therefore, the embodiments described in the present specification and the drawings are only exemplary embodiments of the present invention, and are not intended to represent all of the technical ideas disclosed in the present specification, so that various equivalents It should be understood that water and variations may be present.

110: Surface display device 120: User interface device
130: Dome projector 140: Experience control device
141: wired / wireless communication unit 142:
143: Drone control unit 150: Storage device
151: Path information DB 152: Map information DB
153: sightseeing DB 154: drone information DB
155 shopping information DB 156 video information DB
157: flight information DB 158: drone control information DB
200: Network 300: Drones
400: control server

Claims (10)

A dron for capturing an image in flight, converting the original image into an image having a resolution lower than that of the original image, and transmitting the image;
A user interface device for receiving at least one of a user's command, selection, and information;
A dome screen for displaying the received image from the drones in a 360-degree image, and a user selecting a specific area to be enlarged in the 360-degree image through the user interface device, Projector; And
Wherein when the coordinate information of the specific area is received from the dome projector, an image corresponding to coordinate information of the specific area is requested by the dragon, and a video corresponding to the coordinate information of the specific area extracted from the original image An experience control device which controls the dome projector to display an enlarged image of high resolution by overlapping an image corresponding to coordinate information of the received specific area with the 360 degree image;
, ≪ / RTI &
Wherein the user interface device is a mouse,
Wherein the dome projector sets an enlarged region of the 360-degree image as the user clicks the wheel of the mouse. The method according to claim 1,
The drone divides an image into a first image and a second image, sequentially transmits the divided image,
Wherein the experience control apparatus re-synthesizes the first image and the second image and displays the re-synthesized image through the dome projector. The method according to claim 1,
When the user desires to enlarge the specific area by 2 times, the drone reduces the area ratio of the image cut out from the original image to 1/2 and transmits the result to the experience control device,
When the user desires to enlarge the specific area by 3 times, the drone reduces the area ratio of the image cut out from the original image to 1/3 and transmits the reduced ratio to the experience control device,
Wherein when the user desires to enlarge the specific region by 4 times, the drone transmits the image cut out from the original image to the experience control apparatus as it is. delete delete The method according to claim 1,
Wherein the dome projector adjusts the magnification of the enlarged area of the 360-degree image as the user scrolls the wheel of the mouse. The method according to claim 1,
The dome projector separates the image photographed by the drones into left and right images, warps the separated left and right images according to the curvature characteristics of the dome screen, And displays them on the dome screen, respectively. Converting the original image photographed by the drones in flight into an image having a lower resolution than the original image and transmitting the converted image to the dome projector;
Displaying the image received from the dome projector as a 360-degree image through a dome screen;
When the user selects a specific area to be enlarged in the 360-degree image through the user interface device, the dome projector transmits coordinate information of the specific area to the experience control device;
Wherein when the experience control apparatus receives coordinate information of the specific region from the dome projector, the experience control apparatus requests an image corresponding to the coordinate information of the specific region with the dragon and outputs coordinate information of the specific region extracted from the original image Receiving a corresponding image; And
Providing an enlarged image of a high resolution to the dome projector by overlapping an image corresponding to coordinate information of a specific area received by the experience control device with the 360 degree image;
And providing the enlarged area of the image photographed by the drones at a high image quality. 9. The method of claim 8,
When the user desires to enlarge the specific area by 2 times, the drone reduces the area ratio of the image cut out from the original image to 1/2 and transmits the result to the experience control device,
When the user desires to enlarge the specific area by 3 times, the drone reduces the area ratio of the image cut out from the original image to 1/3 and transmits the reduced ratio to the experience control device,
Wherein when the user desires to enlarge the specific area by 4 times, the dragon transmits the image cut out from the original image to the experience control device as it is, and a method for providing the enlarged area of the image photographed by the dron with high quality . 9. The method of claim 8,
The dome projector separates the image photographed by the drones into left and right images, warps the separated left and right images according to the curvature characteristics of the dome screen, And displaying the dome image on the dome screen, respectively.

Images