KR102196683B1 - Device and method for photographing 3d tour - Google Patents

Abstract

The present invention relates to a three-dimensional tour photographing device which can reduce an error of positioning data, and a method thereof. The three-dimensional tour photographing device comprises: a camera module photographing an around view image for a specific point; a position tracking module detecting a change in position or direction of the camera module; a tilt unit changing a position or angle between the camera module and the position tracking module; and a control module generating a three-dimensional tour including the plurality of around view images and position information based on data received from the camera module and the position tracking module.

Description

Device and method for photographing 3d tour}

The present invention relates to a 3D tour photographing apparatus and method. Specifically, the present invention relates to a 3D tour photographing apparatus and method capable of simultaneous indoor location tracking and 360-degree spatial photographing.

The best way to record a 3D space is to take a 360-degree view of the space and store it as a 360-degree image, and create a 3D tour by connecting 360-degree images for each location.

At this time, a 360 degree camera is used to photograph a 360 degree space. In addition, in order to create a 3D tour by connecting 360-degree images, it is necessary to find out a point and a photographing path of the 360-degree image to be captured.

In general, in the case of an outdoor space, GPS (Global Positioning System) information may be used to track a photographing path and point. However, since GPS information cannot be used in an indoor space, it is difficult to track a photographing point and a route in real time.

Accordingly, for indoor location tracking, a moving location and path can be estimated in real time using SLAM (Simultaneous Localization and Mapping) technology. When shooting indoor spaces, the technology of tracking the shooting path and tracking the shooting point can be usefully used in various industries.

However, in order to use such SLAM, special equipment such as a tracking camera is required. In other words, in order to use the SLAM technology, there is an inconvenience of having to prepare a separate location measurement device when shooting a 360-degree space.

In addition, since a position measuring device is provided separately from the image capturing device for generating a 3D tour, image capturing and position measurement are separated, and it is difficult for a user to control it at the same time.

In addition, when collecting data for creating a 3D tour, since image capture and location measurement are separated, there is a problem that an error occurs in the location measurement data.

An object of the present invention is to provide a 3D tour photographing apparatus and method capable of reducing an error in position measurement data by simultaneously capturing a 360-degree image and tracking an indoor space.

In addition, another object of the present invention is to provide a 3D tour photographing apparatus and method capable of increasing the accuracy of position measurement by changing a position and angle between a 360-degree image capturing module and a position tracking module using a tilt unit.

In addition, another object of the present invention is to provide a notification when a location measurement error occurs due to lack of feature points of data captured by the location tracking module so that the user can change the location of the location tracking module. It is to provide a tour filming apparatus and method.

The objects of the present invention are not limited to the above-mentioned objects, and other objects and advantages of the present invention that are not mentioned can be understood by the following description, and will be more clearly understood by examples of the present invention. In addition, it will be easily understood that the objects and advantages of the present invention can be realized by the means shown in the claims and combinations thereof.

In order to achieve the above technical problem, a 3D tour photographing apparatus according to an embodiment of the present invention includes a camera module for capturing an around-view image for a specific point, a position tracking module for detecting a change in position or direction of the camera module, A tilt unit for changing the position or angle between the camera module and the location tracking module, and a control module for generating a 3D tour including a plurality of around view images and location information based on data received from the camera module and the location tracking module Includes.

In addition, a notification unit for providing a notification to the user may be further included, and the control module may activate the notification unit when the change in position or direction is not detected.

In addition, when the number of feature points detected by the location tracking module is lower than the reference value, the control module provides a notification notifying that the location or angle of the location tracking module is adjusted through the notification unit, or through an application installed in the user terminal. Notifications can be provided.

In addition, the control module performs the 3D tour using the plurality of around-view images received from the camera module and the location information calculated based on the location change or direction change received from the location tracking module. However, the 3D tour may include a movement path of the camera module and photographing location information of the around view image mapped on the movement path.

In addition, the location tracking module may include a sensor unit including a plurality of sensors spaced apart from each other, and a change measurement unit that measures a change in position or direction of the camera module based on data measured by the sensor unit. .

In addition, the sensor unit may include at least one of a LiDAR sensor, an RGB sensor, an infrared sensor, an ultraviolet sensor, an ultrasonic sensor, a gyro sensor, and an acceleration sensor.

In addition, the control module may derive a movement path of the camera module using data sensed by the location tracking module and current location information.

The tilt unit may further include an angle adjusting unit for changing a position or angle between the camera module and the position tracking module based on a control signal from the control module.

In addition, the camera module may include a plurality of cameras, and may generate one around view image by connecting images captured by the plurality of cameras.

In addition, the camera module includes a camera unit for photographing one side and a rotating unit for rotating the camera unit around a reference axis, and connects a plurality of images continuously captured by the camera unit to generate one around view image. Can be generated.

On the other hand, the 3D tour photographing method performed in the 3D tour photographing apparatus according to an embodiment of the present invention includes a camera module that photographs an around view image for a specific point, and a location tracking that detects a change in position or direction of the camera module. Initializing a module, calculating current location information and a movement path based on data received from the location tracking module, and mapping a photographed image received from the camera module to the calculated movement path to generate a 3D tour And displaying the generated 3D tour on a device associated with the 3D tour photographing device.

In addition, calculating the location information and the movement path may include changing the location or angle of the location tracking module when the location change or direction change is not detected by the location tracking module or the number of detected feature points is lower than a reference value. It may further include generating a notification informing that.

The apparatus and method for photographing a 3D tour according to the present invention can reduce an error in measurement data and improve convenience of shooting a 3D tour by simultaneously capturing a 360-degree image and tracking an indoor space.

In addition, the 3D tour photographing apparatus and method according to the present invention may improve the accuracy of position measurement by changing the photographing position and angle between the 360-degree image photographing module and the location tracking module. In addition, when the 3D tour is not photographed, the camera module and the location tracking module can be stacked and stored, thereby improving the portability of the 3D tour photographing device.

In addition, the 3D tour photographing apparatus according to the present invention provides a notification to change the location of the location tracking module when accurate location measurement is difficult due to insufficient feature points of the data captured by the location tracking module, thereby creating a 3D tour. It is possible to reduce the number of repetitive shots and improve the data reliability of 3D tours.

The effects of the present invention are not limited to the above-described effects, and those skilled in the art of the present invention can easily derive various effects of the present invention from the configuration of the present invention.

1 is a diagram schematically showing a 3D tour photographing system according to an embodiment of the present invention.
2 is a view showing a 3D tour photographing apparatus according to an embodiment of the present invention.
3 is a block diagram illustrating the components of a 3D tour photographing apparatus according to an embodiment of the present invention.
4 is a block diagram illustrating an example of components of a 3D tour photographing apparatus according to an embodiment of the present invention.
5 is a flowchart illustrating a method of operating a 3D tour photographing apparatus according to an embodiment of the present invention.
6 is a flowchart illustrating a notification providing algorithm of a 3D tour photographing apparatus according to an embodiment of the present invention.
7 and 8 are diagrams for explaining a moving path and a photographing interface derived from a 3D tour photographing apparatus according to an embodiment of the present invention.

Advantages and features of the present invention, and a method of achieving them will become apparent with reference to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but will be implemented in various forms different from each other, and only these embodiments make the disclosure of the present invention complete, and common knowledge in the technical field to which the present invention pertains. It is provided to completely inform the scope of the invention to those who have it, and the invention is only defined by the scope of the claims. The same reference numerals refer to the same components throughout the specification.

The terms used in the present specification are for describing exemplary embodiments and are not intended to limit the present invention. In this specification, the singular form also includes the plural form unless specifically stated in the phrase. As used in the specification, "comprises" and/or "comprising" refers to the presence of one or more other components, steps, actions and/or elements, and/or elements, steps, actions and/or elements mentioned. Or does not exclude additions.

Unless otherwise defined, all terms (including technical and scientific terms) used in the present specification may be used as meanings that can be commonly understood by those of ordinary skill in the art to which the present invention belongs. In addition, terms defined in a commonly used dictionary are not interpreted ideally or excessively unless explicitly defined specifically.

When shooting indoor spaces, the technology of tracking the shooting path and tracking the shooting point can be usefully used in various industries. Construction, interior, facility management, factories, etc., in terms of industries mainly performed in indoor spaces where GPS information cannot be received, can obtain useful space management data if the location of the indoor space can be tracked and photographed together.

In the present invention, the location and path of the 3D tour photographing apparatus may be estimated by using a SLAM (Simultaneous Localization and Mapping) technology for indoor location tracking.

In addition, in the present invention, the '3D tour' may be composed of a combination of images (eg, panoramic images) stored by shooting 360 degrees from a plurality of points. In this case, the '3D tour' may include location information and direction information in which a plurality of images are captured.

Here, each location information may be mapped on a floor plan and provided to a user. In addition, location information and direction information derived while moving for shooting a 3D tour are used to derive a movement path, and the derived movement path may be displayed on a user terminal.

In this case, the image captured at 360 degrees and the location information and direction information in which the image was captured may be obtained through the 3D tour photographing apparatus and method of the present invention to be described later.

Hereinafter, with reference to the drawings, a 3D tour photographing apparatus and a method of operating the same according to an embodiment of the present invention will be described in detail.

1 is a diagram schematically showing a 3D tour photographing system according to an embodiment of the present invention.

Referring to FIG. 1, a 3D tour photographing system according to an embodiment of the present invention may include a 3D tour photographing apparatus 100, a user terminal 200, and a 3D tour management server 300.

In an embodiment of the present invention, the user terminal 200 may mean a mobile electronic device that can have a unique identification number, such as a mobile phone, a smart phone, and a wearable device (eg, a watch-type terminal). Here, it goes without saying that the operating system (OS) of the user terminal 200 is not limited to a specific operating system (eg, an iOS or Android operating system).

In addition, the user terminal 200 is an input unit that receives a user's input, a display unit that displays visual information, a communication unit that transmits and receives signals to and from the outside, and processes data, controls each unit inside the user terminal 200, and controls the units. It may include a control unit that controls the transmission / reception of data between. Hereinafter, what the control unit performs in the user terminal 200 according to a user's command is collectively referred to as what the user terminal 200 performs.

In an embodiment of the present invention, the user terminal 200 may receive an image, location information, and direction information captured from the 3D tour photographing apparatus 100 through an installed application.

In addition, the user terminal 200 may display a notification received from the 3D tour photographing apparatus 100. Here, the user terminal 200 may provide the user with a notification received through a pre-installed application.

In an embodiment of the present invention, the user terminal 200 is at least one of the 3D tour photographing apparatus 100 and the 3D tour management server 300 on a mobile communication network established according to a technology standard or communication method for mobile communication through a communication unit. It can transmit and receive one radio signal. In an embodiment of the present invention, the user terminal 200 may transmit a signal input by the user to the 3D tour photographing apparatus 100, and from the 3D tour photographing apparatus 100 through an application installed in the user terminal 200. You can receive 3D tours and notifications.

In an embodiment of the present invention, the 3D tour photographing apparatus 100 may capture an image for a specific point and then collect feature points and location information for the captured image. Subsequently, the 3D tour photographing apparatus 100 may generate a 3D tour using the collected data. Data collected by the 3D tour photographing apparatus 100 may be transmitted and received to the user terminal 200 and the 3D tour management server 300.

When the 3D tour photographing apparatus 100 lacks feature points of the captured image, the 3D tour photographing apparatus 100 communicates with the camera module 110 through the notification unit 150 or the user terminal 200 provided in the corresponding apparatus. A notification notifying that the position or angle between the location tracking modules 120 is adjusted may be provided.

In an embodiment of the present invention, the 3D tour management server 300 may manage and store data received from the 3D tour photographing apparatus 100, and the stored data is stored in the 3D tour photographing apparatus 100 and the user terminal 200. Can be shared. Here, the 3D tour management server 300 is a computer system that provides information or services to clients through a communication network, and may mean a computer (sever program) or a device (device).

In this case, the 3D tour management server 300 may be directly operated or managed by a specific company or individual or entrusted to the outside, and may be operated by the same subject. In addition, it goes without saying that the functions performed by the 3D tour management server 300 may be performed separately from a plurality of servers.

Hereinafter, a 3D tour photographing apparatus and method according to an embodiment of the present invention will be described in detail.

2 is a view showing a 3D tour photographing apparatus according to an embodiment of the present invention.

Referring to FIG. 2, a 3D tour photographing apparatus 100 according to an embodiment of the present invention includes a camera module 110, a location tracking module 120, and a tilt unit 130.

The camera module 110 is a device capable of photographing the surroundings at 360 degrees. The camera module 110 may store an image photographing the surroundings at a specific angle at a specific point. In this case, the captured image may be stored in an around view format (eg, a single image in which a plurality of images are merged or an image taken while rotating at a specific angle or more based on a specific point). At this time, the format of the captured image may be variously modified and implemented.

In an embodiment of the present invention, the camera module 110 may include a plurality of cameras capable of photographing the front and the rear. In this case, a plurality of images captured by a plurality of cameras may be converted into a single image using a stitching technique.

Specifically, the camera module 110 may photograph the surroundings at 360 degrees using the camera lens CL. The camera module 110 may include a plurality of camera lenses CL, and the plurality of camera lenses CL may be used for 360 degree photography by being disposed in various positions and directions. In this case, the camera module 110 may use a wide-angle lens, a fisheye lens, or the like, and various lens groups may be used for 360 degree photography.

Meanwhile, in another embodiment of the present invention, although not clearly illustrated in the drawings, the camera module 110 may include a single camera unit capable of photographing only one side and a rotating unit rotating 360 degrees with respect to a reference axis. In this case, the camera module 110 may rotate 360 degrees around a reference axis at a specific point and photograph the surroundings. In this case, images continuously photographed by the camera module 110 may be connected through a stitching technique and converted into one image.

The location tracking module 120 measures location information of the 3D tour photographing apparatus 100 and a change thereof. The location tracking module 120 may create a space map and estimate a current location using a SLAM (Simultaneous localization and mapping) technology.

Specifically, SLAM refers to a technology that creates a map of the unknown environment while the robot moves around the unknown environment and recognizes its own location without external help with only the sensors attached to the robot. SLAM can estimate the location of a mobile robot based on feature points or landmarks of several physical spaces.

To this end, the location tracking module 120 may include sensors such as LiDAR, a camera (eg, RGB, infrared, ultraviolet), an ultrasonic sensor, a gyro sensor, an acceleration sensor, and the like. In this case, the location tracking module 120 may measure the location coordinates and the location change of the 3D tour photographing apparatus 100 using a plurality of sensors.

The location tracking module 120 may detect a change in location by extracting feature points from images coming through a plurality of lenses. For example, the location tracking module 120 may receive an image through a first lens L1 located on one side and a second lens L2 located on the other side, and extract feature points from a plurality of received images. Subsequently, the position tracking module 120 may determine a change in position and a change in direction by extracting the moving direction and the moving speed of the feature point. In this case, the plurality of lenses in the location tracking module 120 may be located in various positions and directions.

The tilt unit 130 may adjust the position and direction of the location tracking module 120 based on the camera module 110. For example, the tilt unit 130 may have a structure capable of rotating the position of the position tracking module 120 in the vertical direction or left and right directions. That is, by using the tilt unit 130, the user can adjust the position and angle of the location tracking module 120 based on the camera module 110.

Meanwhile, in the 3D tour photographing apparatus 100, adjusting the location of the location tracking module 120 in a direction in which a large number of feature points of an image can be extracted is most important in reducing the error of location information.

Accordingly, when the feature point of the currently captured image is smaller than the reference value, the 3D tour photographing apparatus 100 may provide an alarm notifying the user to change the location of the location tracking module 120. Such an alarm may be provided in the form of light or sound through the notification unit 150 in FIG. 4. In addition, the alarm may be provided to the user through an application installed in the user terminal 200.

Additionally, the tilt unit 130 may change the position so that the camera module 110 and the position tracking module 120 overlap each other. That is, the camera module 110 and the location tracking module 120 may be aligned in the same direction. Accordingly, the user's portability of the 3D tour photographing apparatus 100 may be improved, and inconvenience in that the user must separately possess a camera photographing apparatus and a location tracking apparatus may be reduced.

Hereinafter, a peripheral device that exchanges data with the 3D tour photographing apparatus 100 will be described.

3 is a block diagram illustrating the components of a 3D tour photographing apparatus according to an embodiment of the present invention.

Referring to FIG. 3, in the embodiment of the present invention, the 3D tour photographing apparatus 100 may further include a control module 140 that controls the operation of the camera module 110 and the location tracking module 120.

The control module 140 is connected to the camera module 110 and the location tracking module 120 by wired or wirelessly, and may receive an image captured by the camera module 110 and location information measured by the location tracking module 120. .

Here, the photographed image may be an image of an around view format photographed at 360 degrees at a specific point. In addition, the captured image may be a plurality of images captured within the same project. In this case, the control module 140 merges the plurality of images based on the location information received from the location tracking module 120 to achieve 360 degrees. It can be created with one image representing all

In this case, the location information received by the control module 140 may include a coordinate value of a specific coordinate system and time information of the corresponding coordinate value. That is, the location information may also include location change information that can grasp the movement of coordinate values over time.

In addition, the control module 140 may include a wireless communication module. Using the wireless communication module, the control module 140 may wirelessly exchange data with the user terminal 200 and the 3D tour management server 300.

For example, the 3D tour photographing apparatus 100 may perform wireless communication using a wireless communication module such as a WiFi module or a Bluetooth module. At this time, in order to minimize communication interference, a communication protocol (for example, Bluetooth) between the 3D tour photographing device 100 and the user terminal 200, and between the 3D tour photographing device 100 and the 3D tour management server 300 Communication protocols (for example, Wi-Fi) may be different.

Meanwhile, the user may control the operation of the 3D tour photographing apparatus 100 through the user terminal 200.

For example, the user can control the operation of the camera module 110 of the 3D tour photographing apparatus 100 through an interface provided by an application preinstalled in the user terminal 200. That is, the user remotely controls the camera module 110 through the application of the user terminal 200, and obtains a 360-degree image of each photographing point through a photographing command.

In addition, the 3D tour photographing apparatus 100 may acquire location information using the location tracking module 120 while the camera module 110 performs a photographing operation.

At this time, when the feature point of the image captured by the location tracking module 120 is lower than the reference value, the control module 140 sends a notification to adjust the angle and position of the location tracking module 120 to the notification unit 150 or the user terminal. It can be provided to (200).

The user may change the location of the location tracking module 120 based on the alarm displayed on the user terminal 200. In this case, the user may adjust the photographing position and direction of the location tracking module 120 by using the tilt unit 130 of the 3D tour photographing apparatus 100.

In addition, as described above, the tilt unit 130 may freely change the photographing direction of the location tracking module 120 vertically, horizontally, and vertically, and may be configured with a joint structure capable of adjusting the photographing angle. In this case, the tilt unit 130 may adopt various structures capable of changing the photographing direction of the location tracking module 120.

Subsequently, when the feature point of the image captured by the location tracking module 120 is higher than the reference value according to the changed location of the location tracking module 120, the control module 140 notifies (for example, location setting is completed) A notification that it has been done) may be additionally provided to the notification unit 150 or the user terminal 200. Through this, the user can obtain more accurate location information for image capture.

The 3D tour management server 300 may receive a 360-degree image and location information captured by the 3D tour photographing apparatus 100, and generate a 3D tour based on the received plurality of images and location information. The generated 3D tour is stored and managed in the storage of the 3D tour management server 300, and the stored 3D tour may be shared and used by the 3D tour photographing apparatus 100 or the user terminal 200 when necessary.

Hereinafter, a specific example of a 3D tour photographing apparatus according to an embodiment of the present invention will be described in detail.

4 is a block diagram illustrating an example of components of a 3D tour photographing apparatus according to an embodiment of the present invention.

4, the 3D tour photographing apparatus 100 includes a camera module 110, a location tracking module 120, a tilt unit 130, a control module 140, a notification unit 150, and a communication unit 160. Can include.

First, the camera module 110 may include a plurality of camera units 111. In this case, the camera unit 111 may capture images of different positions and directions through a plurality of cameras located at various positions and directions.

In addition, the camera module 110 may include a single camera unit 111 and a rotating unit 113. At this time, the rotation unit 113 rotates the camera unit 111 about the reference axis of the 3D tour photographing apparatus 100 so that the camera unit 111 can capture a 360-degree image.

The location tracking module 120 may include a sensor unit 121 and a change measurement unit 123. The sensor unit 121 may include a plurality of sensors spaced apart from each other, and the sensors may measure various images, location information, and direction information.

Here, the sensors provided in the sensor unit 121 may include a LiDAR sensor, an RGB sensor, an infrared sensor, an ultraviolet sensor, an ultrasonic sensor, a gyro sensor, and an acceleration sensor.

For example, a gyro sensor is a sensor that can measure how much the angular velocity of a rotating object, that is, the angle of a rotating object, has changed every second. When a user photographs with the 3D tour photographing apparatus 100, the photographed image may include information on a rotation angle by a gyro sensor. The 3D tour photographing apparatus 100 may generate a 3D tour based on rotation angle information in the captured image.

The change measurement unit 123 may receive data measured by the sensor unit 121. The change measuring unit 123 may analyze the received data to measure a change in position or direction of the camera module 110.

The control module 140 may receive an around view image and location information from the camera module 110 and the location tracking module 120. The location information may include a change in position or direction of the camera module 110.

Subsequently, the control module 140 may derive a movement path of the camera module 110 based on the current location information and data such as location change and direction change received from the location tracking module 120.

In addition, the control module 140 may generate a 3D tour based on the received around view image, a moving path of the camera module 110, and photographing location information and data of the image.

Meanwhile, the control module 140 may provide a notification to the user when a feature point of an image detected by the location tracking module 120 is smaller than a reference value or when a change in position and a change in direction are not detected.

In this case, the notification may include a notification notifying that the position or angle of the camera module 110 and the location tracking module 120 is adjusted through the notification unit 150 or an application installed in the user terminal. The notification may appear in the form of light or sound, or may be provided through a separate message.

Subsequently, when the user changes the location of the location tracking module 120 based on the notification, the control module 140 may determine whether a feature point of the image recognized by the location tracking module 120 is higher than a reference value. .

If the feature point is higher than the reference value, the control module 140 may display a notification that the location change recognition is properly performed through the notification unit 150 or an application installed in the user terminal 200.

In an embodiment of the present invention, the tilt unit 130 includes an angle adjustment unit 131 and may be located between the camera module 110 and the location tracking module 120. At this time, the angle adjustment unit 131 may be configured with a small motor, a hydraulic structure, or the like.

In addition, the angle adjustment unit 131 may control an operation based on a control signal received from the control module 140. If the feature point of the image captured by the location tracking module 120 is lower than the reference, the control module 140 transmits a control signal to the tilt unit 130 to request adjustment of the position and angle of the location tracking module 120 Can send. Based on the control signal, the angle adjustment unit 131 may change the position and direction of the position tracking module 120.

Subsequently, when the feature point of the image captured by the location tracking module 120 is higher than the reference point at the changed position, the control module 140 may transmit a control signal for stopping the operation of the angle adjustment unit 131.

The notification unit 150 may provide a notification notifying that the location of the location tracking module 120 is changed to a user. The notification unit 150 may include a light emitting device or a sound generating device, and may provide a corresponding notification to a user using light or sound.

The communication unit 160 may transmit and receive data to and from the user terminal 200 through a communication module (eg, Wi-Fi or Bluetooth) using a wireless communication protocol. In addition, the communication unit 160 may transmit a control signal and a notification received from the control module 140 to the user terminal 200.

5 is a flowchart illustrating a method of operating a 3D tour photographing apparatus according to an embodiment of the present invention. 6 is a flowchart illustrating a notification providing algorithm of the 3D tour photographing apparatus according to an embodiment of the present invention.

Referring to FIG. 5, the 3D tour photographing apparatus 100 receives a spatial photographing request from the user terminal 200 (S110). In this case, the user may command the 3D tour photographing apparatus 100 to perform spatial photographing by using an application previously installed in the user terminal 200.

Subsequently, the 3D tour photographing apparatus 100 initializes the camera module 110 and the location tracking module 120 (S120). Through this, the 3D tour photographing apparatus 100 may prepare space photographing and location tracking.

Subsequently, the 3D tour photographing apparatus 100 derives the current location information and a movement path based on the data received from the location tracking module 120 (S130).

Specifically, referring to FIG. 6, the 3D tour photographing apparatus 100 determines whether a change in position or a change in direction is detected through the location tracking module 120 of the 3D tour photographing apparatus 100 (S131). In this case, the location tracking module 120 may measure a change in position and direction of the 3D tour photographing apparatus 100 using an ultrasonic sensor, a gyro sensor, an acceleration sensor, or the like.

Subsequently, when a change in position or direction of the 3D tour photographing apparatus 100 is detected, the 3D tour photographing apparatus 100 determines whether the number of feature points detected by the location tracking module 120 is smaller than a reference value (S133). ).

If a change in position or direction is not detected by the location tracking module 120, or the number of detected feature points is smaller than a reference value, the 3D tour photographing apparatus 100 activates the notification unit 150 (S135).

At this time, the control module 140 may notify the user to adjust the position and angle of the location tracking module 120 by sending a control signal to the notification unit 150. In addition, it is possible to notify the user to adjust the position and angle of the location tracking module 120 through an application installed in the user terminal 200. The user recognizes the displayed notification and changes the position or angle between the camera module 110 and the location tracking module 120 (S137).

Subsequently, when the location and angle of the location tracking module 120 is reset, the 3D tour photographing apparatus 100 repeats steps S131 to S137.

Meanwhile, when a change in position or a change in direction is detected by the location tracking module 120 and the number of detected feature points is higher than the reference value, the notification unit 150 in the 3D tour photographing apparatus 100 is deactivated (S136).

Subsequently, referring again to FIG. 5, the 3D tour photographing apparatus 100 maps the photographed image received from the camera module 110 to the derived movement path (S140). That is, a location where an image is photographed is displayed among the moving paths of the 3D tour photographing apparatus 100, and the photographed image is connected to the corresponding location.

Subsequently, the 3D tour photographing apparatus 100 provides the extracted movement path and an image corresponding thereto to the user terminal 200. That is, the movement path received from the 3D tour photographing apparatus 100 may be displayed on the screen of the user terminal 200 in a plan view. The user may select a location in which the image was captured among the movement paths, and in this case, the user terminal 200 may display a 3D image of the location.

Subsequently, the 3D tour photographing apparatus 100 generates a 3D tour based on the data measured and derived from the camera module 110 and the location tracking module 120 after shooting is completed, and the 3D tour management server 300 (ie , Cloud server) and share with the user's terminal (S150).

Hereinafter, an example of a moving path and a 3D tour photographing interface derived from a 3D tour photographing process according to an embodiment of the present invention will be described.

7 and 8 are views for explaining a moving path and a photographing interface derived from a 3D tour photographing apparatus according to an embodiment of the present invention.

Referring to FIG. 7, FIG. 7 shows a spatial photographing point and a movement path MP and a current position CP of the 3D tour photographing apparatus 100 during a photographing process.

The 3D tour photographing apparatus 100 may set the first spatial photographing point as a starting point SP.

Subsequently, the 3D tour photographing apparatus 100 derives the movement path through the change in position and direction measured by the location tracking module 120, and the second spatial photographing point (P2) on the derived movement path, and the third spatial photographing The around view image photographed at the point P3 and the fourth spatial photographing point P4 may be mapped onto a corresponding movement path MP.

At this time, the 3D tour photographing apparatus 100 uses SLAM technology to derive the position change and current position coordinates of the 3D tour photographing apparatus 100 without a GPS signal, and derive an accurate movement path (MP) based on this. have.

Referring to FIG. 8, the location tracking module 120 included in the 3D tour photographing apparatus 100 of the present invention derives each feature point included in a screen to be photographed.

Specifically, the 3D tour photographing apparatus 100 may display a plan view GP in which a moving path MP is displayed on one side of the screen, and an image captured in real time on the other side of the screen in a moving image format.

For example, a plan view GP may be displayed in the first area D1 of the divided screen, and the video screen TP to be photographed may be displayed in the second area D2. A point at which the around view image is captured may be displayed on the plan view GP.

The location tracking module 120 extracts the location where the around-view image was captured, and displays the location on the plan view GP as a first location P1 and a second location P2, and represents the current location. (CP) can be expressed.

The user can start capturing the around view image by clicking the capturing button B1, and can stop or end the capturing by clicking the capturing button B1 again during shooting.

A plurality of feature points (SP) are distributed in the captured image, and the location tracking module 120 extracts a change in position or direction from the distributed feature points (SP), and uses the current location information to capture a 3D tour photographing device 100 ) Of the movement path (MP) can be calculated.

Specifically, the location tracking module 120 extracts the moving speed and displacement of the derived feature points. Subsequently, the location tracking module 120 may calculate current location information for the moved location of the 3D tour photographing apparatus 100 based on the movement speed and displacement of the extracted feature points. Subsequently, the control module 140 may calculate a movement path MP of the 3D tour photographing apparatus 100 by using data sensed by the location tracking module 120 and current location information.

At this time, when the feature point of the image captured by the location tracking module 120 is smaller than the reference value, the control module 140 provides a notification notifying that the angle and position of the location tracking module 120 is adjusted to the user terminal 200. I can.

The user may change the location of the location tracking module 120 based on the alarm displayed on the user terminal 200. In this case, the user may adjust the photographing direction of the location tracking module 120 by using the tilt unit 130 of the 3D tour photographing apparatus 100.

Subsequently, when the feature point of the image captured by the location tracking module 120 is higher than the reference value according to the changed position of the location tracking module 120, the control module 140 additionally notifies the user terminal 200. Can provide. Through this, the user can obtain more accurate location information for image capture.

In summary, the 3D tour photographing apparatus and method according to the present invention can reduce an error in measurement data and improve the convenience of photographing a 3D tour by simultaneously capturing a 360-degree image and tracking an indoor space.

In addition, the 3D tour photographing apparatus according to the present invention may improve the accuracy of position measurement by using a tilt unit capable of changing the shooting angle of the 360-degree image capturing module and the position tracking module, and the camera module and the position tracking module As it can be stored in layers, the portability of the 3D tour photographing device can be improved.

In addition, in the 3D tour photographing apparatus and method according to the present invention, when an error occurs in the location measurement due to insufficient feature points of the data captured by the location tracking module, a notification is provided to change the location of the location tracking module. , It is possible to reduce the number of repetitive shooting during 3D tour production and improve the data reliability of the 3D tour.

As described above, although the present invention has been described by the limited embodiments and drawings, the present invention is not limited to the above embodiments, which is various modifications and variations from these descriptions to those of ordinary skill in the field to which the present invention belongs. Transformation is possible. Accordingly, the idea of the present invention should be grasped only by the scope of the claims set forth below, and all equivalent or equivalent modifications thereof will be said to belong to the scope of the idea of the present invention.

100: 3D tour photographing device 110: camera module
120: position tracking module 130: tilt unit
200: user terminal 300: 3D tour management server

Claims (12)

A camera module for capturing an around view image for a specific point;
A location tracking module that detects a change in position or direction of the camera module;
A tilt unit for changing an angle between the camera module and the location tracking module; And
A control module for generating a 3D tour including a plurality of around view images and location information based on the data received from the camera module and the location tracking module,
The tilt part,
It is composed of a joint structure capable of adjusting the shooting angle of the location tracking module by changing the location of the location tracking module in the vertical direction or left and right directions with respect to the camera module,
Based on the control signal received from the control module, comprising an angle adjustment unit for adjusting the angle between the camera module and the position tracking module
3D tour shooting device.
The method of claim 1,
Further comprising a notification unit for providing a notification to the user,
The control module, when the position change or the direction change is not detected, to activate the notification unit
3D tour shooting device.
The method of claim 2,
When the number of feature points detected by the location tracking module is lower than a reference value, the control module provides a notification notifying that the location or angle of the location tracking module is adjusted through the notification unit, or the notification through an application installed in the user terminal To provide
3D tour shooting device.
The method of claim 1,
The control module,
The 3D tour is generated using the plurality of around-view images received from the camera module and the location information calculated based on the location change or direction change received from the location tracking module,
The 3D tour includes a movement path of the camera module and photographing location information of the around view image mapped on the movement path.
3D tour shooting device.
The method of claim 1,
The location tracking module,
A sensor unit including a plurality of sensors spaced apart from each other,
Comprising a change measuring unit for measuring the position change or direction change of the camera module based on the data measured by the sensor unit
3D tour shooting device.
The method of claim 5,
The sensor unit includes at least one of a LiDAR sensor, an RGB sensor, an infrared sensor, an ultraviolet sensor, an ultrasonic sensor, a gyro sensor, and an acceleration sensor.
3D tour shooting device.
The method of claim 5,
The control module is configured to derive a movement path of the camera module using data detected by the location tracking module and current location information.
3D tour shooting device.
The method of claim 1,
The tilt unit is disposed between the camera module and the position tracking module, and includes a motor or hydraulic structure whose operation is controlled by the control module.
3D tour shooting device.
The method of claim 1,
The camera module,
Including a plurality of cameras, by connecting the images photographed by the plurality of cameras to generate the one around view image
3D tour shooting device.
The method of claim 1,
The camera module,
A camera unit that photographs one side,
Including a rotation unit for rotating the camera unit about the reference axis,
Connecting a plurality of images continuously photographed by the camera unit to generate the one around view image
3D tour shooting device.
delete delete

Images