KR20220053983A - Apparatus and method for providing guidance information for visual space shooting - Google Patents

Abstract

Disclosed are an apparatus and method for providing guidance information for space image photographing to enable even beginners to easily take space images. According to one embodiment of the present invention, a method for visual space photographing of a space image photographing apparatus comprises the following steps: setting a photographing environment and photographing direction of a camera for photographing a space image; generating an overlay image from an image captured by the camera; providing overlay guidance information generated on the basis of the overlay image to the camera; and determining whether optical tracking of the camera is possible and providing the camera with optical guidance information generated in consideration of the moving position of a subject captured in the image according to a determination result of whether the optical tracking is possible.

Description

Apparatus and method for providing guidance information for spatial imaging {APPARATUS AND METHOD FOR PROVIDING GUIDANCE INFORMATION FOR VISUAL SPACE SHOOTING}

The present invention relates to a spatial image photographing technology for point cloud-based three-dimensional restoration, and more particularly, to a technique for generating and providing guidance information for efficient point cloud generation and spatial image photographing.

In general, the conventional spatial imaging technology takes a lot of shooting time due to overshooting in a way that the photographer shoots as many images as possible or performs post-processing processing depending on the senses, and the point cloud creation time becomes long due to the number of unnecessary images, etc. There is a problem in that resources are wasted and it is difficult to guarantee the shooting quality. In addition, there is a problem in that the photographing quality is deteriorated, such as a deviation in the photographed image quality occurs depending on the skill of the photographer.

To improve this, it is possible to reduce unnecessary shooting images and shooting time by providing shooting guidance information suitable for point cloud generation, and to maintain optimal image quality by minimizing deviations in shooting skills. You need a point cloud shooting method that can respond.

Meanwhile, Korean Patent Application Laid-Open No. 10-2018-0087947 “Modeling method and modeling device using three-dimensional point cloud” extracts at least one region from an image captured by a camera, and extracts two-dimensional feature points from the image. Receive the camera pose information tracked based on the data, estimate first depth information of the image based on the pose information of at least one region and the camera, and model the map corresponding to the image based on the first depth information Disclosed is an apparatus and method for generating a three-dimensional point cloud for

In order to solve the problems of the prior art described above, the present invention provides guidance information necessary for spatial image shooting, such as camera posture and image quality, for spatial imaging in real time in real time, so that a user has a spatial image having a suitable image quality at an optimal location. An object of the present invention is to provide an apparatus and method capable of photographing.

A method for providing guidance information for spatial image shooting according to an embodiment of the present invention for achieving the above object is a method for providing guidance information for spatial image shooting of a guidance information providing device for spatial image shooting, spatial image shooting setting a shooting environment and shooting direction of the camera for Generating an overlay image from the image taken by the camera, providing overlay guidance information generated based on the overlay image to the camera, determining whether the camera is optically traceable, and determining whether the optical trace is possible and providing, to the camera, optical guidance information generated in consideration of the moving position of the subject photographed in the image according to the result.

In addition, the apparatus for providing guidance information for spatial image shooting according to an embodiment of the present invention for achieving the above object includes: a shooting setting unit for setting a shooting environment and shooting direction of a camera for spatial image shooting; and generating an overlay image from the image taken by the camera, providing overlay guidance information generated based on the overlay image to the camera, determining whether the camera is optically traceable, and determining whether the optical trace is possible and a photographing guide unit for providing the optical guide information generated in consideration of the moving position of the subject photographed in the image according to the result to the camera.

The present invention is implemented to be directly or interlocked with a camera shooting device such as a mobile phone app, DSLR, PC, etc., and provides guidance information so that even beginners can easily shoot spatial images without relying on the senses of experts or people. results can be obtained.

In addition, according to the present invention, it is possible to reduce post-processing cost due to overshooting, reduce cost and time for re-shooting due to insufficient shooting, and increase spatial image content due to easy spatial image shooting.

1 is an operation flowchart illustrating a method for providing guidance information for spatial image shooting according to an embodiment of the present invention.
2 is a diagram illustrating a process of providing an overlay image for providing guidance information according to an embodiment of the present invention.
3 is a block diagram illustrating an apparatus for providing guidance information for spatial image shooting according to an embodiment of the present invention.
4 is a diagram illustrating a computer system according to an embodiment of the present invention.

The present invention will be described in detail with reference to the accompanying drawings as follows. Here, repeated descriptions, well-known functions that may unnecessarily obscure the gist of the present invention, and detailed descriptions of configurations will be omitted. The embodiments of the present invention are provided in order to more completely explain the present invention to those of ordinary skill in the art. Accordingly, the shapes and sizes of elements in the drawings may be exaggerated for clearer description.

Throughout the specification, when a part "includes" a certain element, it means that other elements may be further included, rather than excluding other elements, unless otherwise stated.

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the accompanying drawings.

1 is an operation flowchart illustrating a method for providing guidance information for spatial image shooting according to an embodiment of the present invention.

In the method for providing guidance information for spatial image shooting according to an embodiment of the present invention, in order to shoot with image quality and quantity of images suitable for optimal conditions for point cloud generation, the ability of a photographer is provided by providing guidance information for spatial image shooting. Space shooting is possible with little deviation.

Referring to FIG. 1 , in the method for providing guidance information for spatial image shooting according to an embodiment of the present invention, a shooting environment may be set first ( S101 ).

That is, in step S101, a photographing environment of a camera for photographing a spatial image may be set.

In this case, in step S101, since the photographing method varies according to the characteristics of the spatial photographing target, initial environment settings such as a threshold value suitable for the photographing method and environment may be performed.

In this case, in step S101, a shooting target may be set to either one of a case of shooting a wide space or a case of shooting an object object, and a shooting method may be set by dividing into a panorama mode and an object mode.

For example, point cloud photography has a feature in that a point cloud is created by extracting a correlation based on the overlapped contents of images taken by continuously shooting with a considerable overlap.

Accordingly, in step S101, after determining a photographing method in the panoramic mode or the object mode, an appropriate overlapping degree may be set according to the target environment.

In this case, step S101 may define an optical tracking target area setting, an optical tracking target image scale, an optical tracking sensitivity, an effective range, an automatic shooting timer, and the like through prior understanding of the imaging target.

In addition, the method for providing guidance information for spatial image shooting according to an embodiment of the present invention may set the shooting direction (S102).

That is, in step S102, the shooting direction of the camera for spatial image shooting may be set.

At this time, in step S102, when shooting starts, it is necessary to determine the direction in which to shoot continuously based on the first captured image. In the manual mode that the user selects, gyroscope information such as a mobile phone is used, or movement between images It can be provided in an automatic mode that automatically determines the direction by a method such as optical tracking that calculates .

In addition, the method for providing guidance information for spatial image shooting according to an embodiment of the present invention may provide overlay guidance information (S103).

That is, in step S103, an overlay image may be generated from an image captured by the camera, and overlay guidance information based on the overlay image may be provided to the camera.

At this time, in step S103, in order to easily position the camera posture to be taken consecutively in accordance with the direction of movement after the first shooting, the previous shot image is moved from the corresponding position by the overlapping degree, and then projected on the camera viewfinder as an overlay transparently. It is possible to generate overlay guidance information including the induced result.

At this time, in step S103, if the overlay guidance information is provided to the camera, the camera can be moved and adjusted according to the information provided by the projected overlay guidance image to position the camera more quickly and accurately, thereby enabling rapid shooting. .

The existing shooting method has no choice but to take pictures depending on the senses such as the photographer's sight, so there is a problem that the overlapping degree is not constant, so there is a problem that the picture is taken excessively or insufficiently. Some of the problems that can cause image quality degradation can be improved.

In addition, the method for providing guidance information for spatial image shooting according to an embodiment of the present invention may determine whether the optical tracking of the camera is possible (S104).

That is, in step S104, when the camera includes an optical tracking function, optical guidance information is provided to the camera (S105), and when the camera does not include an optical tracking function, a focus check may be performed (S107) .

That is, in step S105, it is determined whether the camera is capable of optical tracking, and, according to the determination result of whether the optical tracking is possible, the optical guidance information generated in consideration of the movement position of the subject photographed in the image is provided to the camera can do.

In this case, step S105 may provide optical guidance information for using real-time optical tracking together with providing overlay guidance information to maintain a more accurate camera posture position and image quality.

The continuous shooting method has a characteristic in that the same subject is moved within a certain range between adjacent images, so that the overlapping pictures are taken.

In this case, in step S105, the movement position of the subject characterized by the two images captured by the camera may be calculated in real time by a preset optical tracking algorithm based on the characteristics.

At this time, step S105 calculates the moving position in real time by further considering the preset target overlapping degree, camera posture, etc., and optical guidance information including information (arrows, etc.) for correcting the horizontal and vertical positions. can create

In this case, step S105 provides optical guidance information to the camera so that the camera can perform shooting at an optimal position.

In this case, in step S105, the optimal camera position such as horizontal and vertical of the camera may be calculated by comparing it with the transparent overlay image extracted from the overlay guidance information in order to know the relative position of the camera to be photographed.

In this case, in step S105, it is possible to easily move to a nearby location using the overlay guidance information generation technique, and finely adjust the camera posture according to the optical tracking result.

At this time, in step S105, the same feature point of optical tracking is directly connected with a line segment so that the user can easily adjust the posture, and then displayed to the user in real time to increase readability of self-position errors, and movement guidance information such as arrows It can also help you to easily complete the camera posture adjustment direction by continuously providing

In this case, in step S105, not only manual shooting when the camera is positioned at an optimal position, but also automatic shooting may be provided if the camera is within an effective value.

In this case, in step S105, the same object tracking between the image and the image may be performed using an optical tracking algorithm, and the optical tracking algorithm may be modified according to the characteristics of spatial imaging.

At this time, in step S105, in the case of panoramic mode spatial shooting, the target optical tracking target object moves horizontally based on the center of the camera in many cases, and object mode spatial shooting taking while rotating around the target subject is performed at the center of the camera In order to increase the effect of the optical tracking algorithm considering that the amount of change is very small, the user can adjust the optical tracking target suitable for the shooting method as an area of interest, or select it automatically by comparing the amount of change versus the overlap.

At this time, in step S105, in the case of object mode spatial shooting, when the region of interest is centered on the target object, the amount of change is very small and a significant difference from the desired overlap occurs. By setting , you can shoot while maintaining the target overlap and the number of captured images.

In addition, the method for providing guidance information for spatial image shooting according to an embodiment of the present invention may perform a color difference test and a focus test (S106 and S107).

That is, in steps S106 and S107, if the geometrical position of the camera is selected as described above, the user can easily take pictures with the same color through color difference inspection and focus inspection to secure image quality. , if the image taken just before is insufficient due to exposure changes such as sunlight during shooting, the image taken just before is automatically deleted and the user can be automatically guided to retake the shot immediately.

At this time, step S106 is performed in odd-numbered areas (masks) such as 3x3, 9x9, etc. with the same feature point matched as a result of optical tracking in consideration of the method of comparing the pixels of the entire overlapping area between images and the performance of devices. It can be calculated by summing the corresponding pixel differences.

In this case, in step S106, since the overlapping region is two or more due to the characteristics of the continuous image, the color difference test may be performed by comparing the pixel difference of at least three images.

In this case, step S106 may determine that there is no color difference when a value within 3% is calculated using the pixel sum calculation of Equation 1 as an example of the color difference test.

[Equation 1]

In addition, in step S107, a focus test is performed to prevent the spread of image quality degradation due to a focus error, and only images verified through the focus test can be stored and transmitted for all images.

In this case, in step S107, after immediately deleting the photographed image in case of a quality problem, it is possible to provide re-photography inducing information for inducing re-photography to the user.

In addition, the method for providing guidance information for spatial image shooting according to an embodiment of the present invention may store, manage, and transmit images and additional information (S108).

That is, in step S108, the start and end of the captured image are divided into one scene for the convenience of managing the captured image, and in the case of repeated shooting of the same scene, it can be managed by dividing it into a take. there is.

In this case, in step S108, when the subject or space to be photographed is large, it is possible to easily divide the photographing by dividing it into several scenes.

In this case, in step S108, additional information regarding shooting, such as a unique value of a shooting camera, the number of captured images, and the names of the captured images, may be separately created, stored, and managed during the scene or take unit storage management.

For example, in step S108, when the image is transmitted in real time through HTTP or the like to a remote point cloud generating device with additional information at the same time as the image is taken, it is not possible to know the end of the continuous shooting image. Alternatively, additional information in a take unit may be transmitted.

Through this, the point cloud generating device can use the additional information to recognize the end of the input image and operate immediately, and cross-verify the content of the additional information for the relevance and integrity of the received images.

2 is a diagram illustrating a process of providing an overlay image for providing guidance information according to an embodiment of the present invention.

Referring to FIG. 2 , as an example, after selecting the target subject 202 , after checking the subject through the camera viewfinder 201 , shooting starts, and if the shooting direction is to the right as shown in FIG. 2 , the previous shot image 204 is displayed by the user It is projected as an overlay image as much as the specified overlapping degree. Accordingly, the user adjusts the position or posture of the camera according to the overlay image guidance information to match the subject 202 input to the actual camera and the overlay-projected image 203, and then takes the picture. If there is no overlay induced image, the camera position and posture are captured based on the subject 202 of the input camera image to match the human senses, so it is impossible to shoot a spatial image with a uniform overlap that is the target. . The present invention by providing the direction 205 in real time by utilizing the gyro information provided by a cell phone or DSLR camera when providing an overlay guidance image, or by providing the direction 205 when the user actually moves for automatic guidance information 204 and 205 such as optical tracking It's easier to accomplish what you're aiming for.

3 is a block diagram illustrating an apparatus for providing guidance information for spatial image shooting according to an embodiment of the present invention.

Referring to FIG. 3 , an apparatus for providing guidance information for spatial image shooting according to an embodiment of the present invention includes a shooting setting unit 310 , a shooting inducing unit 320 , and an image management unit 330 .

The photographing setting unit 310 may set a photographing environment and a photographing direction of a camera for photographing a spatial image.

At this time, the photographing setting unit 310 may perform initial environment settings, such as a threshold value suitable for the photographing method and environment, because the photographing method varies according to the characteristics of the spatial photographing target.

In this case, the photographing setting unit 310 may set a photographing target in either one of a case of photographing a large space and a case of photographing an object object, and may set a photographing method by dividing it into a panorama mode and an object mode.

For example, point cloud photography has a feature in that a point cloud is created by extracting a correlation based on the overlapped contents of images taken by continuously shooting with a considerable overlap.

Accordingly, the photographing setting unit 310 may set an appropriate overlapping degree according to the target environment after determining the photographing method of the panorama mode or the object mode.

In this case, the imaging setting unit 310 may define an optical tracking target area setting, an optical tracking target image scale, an optical tracking sensitivity, an effective range, an automatic shooting timer, and the like through prior understanding of the imaging target.

At this time, when shooting starts, the shooting setting unit 310 must determine the direction to shoot continuously based on the first captured image, using the manual mode that the user selects directly and gyroscope information such as a mobile phone, or using the image It can be provided in an automatic mode that automatically determines the direction by a method such as optical tracking that calculates the movement of the liver.

The photographing induction unit 320 may generate an overlay image from an image photographed by the camera, and provide overlay guidance information based on the overlay image to the camera.

At this time, in order to easily position the camera posture to be continuously photographed in accordance with the direction of movement after the first photographing, the photographing guide unit 320 moves the previous photographed image from the corresponding position by the degree of overlap and projects it on the camera viewfinder as an overlay transparently to generate overlay guidance information including the induced result.

At this time, if the photographing guide unit 320 provides the overlay guide information to the camera, if the camera is moved and adjusted according to the information given by the projected overlay guide image, it is possible to position the posture more quickly and accurately, so that rapid shooting is possible. Do.

The existing shooting method has no choice but to take pictures depending on the senses such as the photographer's sight, so there is a problem that the overlapping degree is not constant, so there is a problem that the picture is taken excessively or insufficiently. Some of the problems that can cause image quality degradation can be improved.

In addition, the photographing guide unit 320 may provide optical guidance information to the camera when the camera includes an optical tracking function, and perform a focus check when the camera does not include an optical tracking function.

That is, the photographing guide unit 320 determines whether the optical tracking of the camera is possible, and according to the determination result of whether the optical tracking is possible, the optical guide information generated in consideration of the moving position of the subject photographed in the image is transmitted to the camera. can provide

In this case, the photographing guide unit 320 may provide optical guidance information for using real-time optical tracking together with overlay guidance information to maintain a more accurate camera posture position and image quality.

The continuous shooting method has a characteristic in that the same subject is moved within a certain range between adjacent images, so that the overlapping pictures are taken.

At this time, the photographing guide unit 320 may calculate in real time the movement position of the subject between the two images photographed by the camera using a preset optical tracking algorithm based on the characteristics.

At this time, the photographing induction unit 320 calculates the moving position in real time by further considering the preset target overlap, the camera posture, etc., and optical guidance including information (arrows, etc.) for correcting the horizontal and vertical positions. information can be created.

At this time, the photographing guide unit 320 may provide the optical guide information to the camera so that the camera may photograph at an optimal position.

At this time, in order to know the relative position of the camera to be photographed, the photographing guide unit 320 may calculate the optimal camera position, such as horizontal and vertical, of the camera by comparing it with the transparent overlay image extracted from the overlay guide information.

In this case, the photographing guide unit 320 may easily move to a nearby location using the overlay guide information generation technique, and may finely adjust the camera posture according to the optical tracking result.

At this time, the photographing induction unit 320 directly connects the same feature points of optical tracking with line segments so that the user can easily adjust the posture, and then displays it to the user in real time to increase readability of self-position errors and induce movement of arrows, etc. It can also help you to easily complete camera posture adjustment directions by providing continuous information.

In this case, the photographing guide unit 320 may provide not only manual photographing when the camera is positioned at an optimal position, but also automatic photographing when the camera is within an effective value.

In this case, the photographing guide unit 320 may perform tracking of the same object between the image and the image by using the optical tracking algorithm, and may modify the optical tracking algorithm according to the characteristics of spatial photographing.

At this time, in the case of the panoramic mode spatial shooting, the shooting guide unit 320 often moves the target optical tracking target horizontally with respect to the center of the camera. Considering that the amount of change in the center is very small, in order to increase the effect of the optical tracking algorithm, the user can adjust the optical tracking target suitable for the shooting method as an area of interest, or automatically select it by comparing the amount of change versus the overlap.

At this time, in the case of object mode spatial shooting, when the area of interest is centered on the target object, the amount of change is very small and a significant difference from the desired overlap occurs. By setting the area, it is possible to shoot while maintaining the target overlap and the number of captured images.

Also, the photographing guide unit 320 may perform a color difference test and a focus test ( S106 and S107 ).

That is, if the photographing guide unit 320 has selected the geometrical position of the camera as described above, it provides a function for the user to easily photograph with the same color through color difference inspection and focus inspection to secure image quality, and sunlight during shooting If the image taken immediately before is less than the target value due to a change in exposure, etc., the image taken immediately before can be automatically deleted and the user can be automatically induced to take a picture again immediately.

At this time, the photographing induction unit 320 considers the method of comparing the pixels of the entire overlapping area between images and the performance of devices, and centered on the same feature point matched as a result of optical tracking, such as 3x3, 9x9, etc. odd-numbered areas (masks) It can be calculated by summing the pixel differences corresponding to .

At this time, since the overlapping region is two or more images due to the characteristics of the continuous image, the photographing guide unit 320 may compare the pixel difference of at least three images to perform a color difference test.

In this case, the photographing induction unit 320 may determine that there is no color difference when a value within 3% is calculated by using the pixel summation calculation of Equation 1 as an example of the color difference test.

In addition, the photographing guide unit 320 may perform a focus inspection to prevent the spread of image quality deterioration due to a focus error, and store and transmit all images verified through the focus inspection only.

In this case, the photographing guide unit 320 may immediately delete the photographed image in case of a quality problem, and then provide the user with re-photography induction information for inducing re-photography.

The image manager 330 may store, manage, and transmit the photographed image and additional information (S108).

That is, the image management unit 330 divides the beginning and the end of a photographed image into one scene for the convenience of managing the photographed image, and in the case of repeated photographing of the same scene, divides it into a take and manages it. can

In this case, the image management unit 330 may divide the image into several scenes and easily divide the photographing if the subject or space to be photographed is large.

In this case, the image management unit 330 may separately generate, store, and manage additional information related to shooting, such as a unique value of a shooting camera, the number of captured images, and the names of the captured images, when managing a scene or take unit of shooting.

For example, when the image management unit 330 transmits additional information and an image to a remote point cloud generating device in real time through HTTP or the like simultaneously with image capturing Additional information in units of new or take may be transmitted.

Through this, the point cloud generating device can use the additional information to recognize the end of the input image and operate immediately, and cross-verify the content of the additional information for the relevance and integrity of the received images.

4 is a diagram illustrating a computer system according to an embodiment of the present invention.

Referring to FIG. 4 , the apparatus for providing guidance information for spatial image capturing according to an embodiment of the present invention may be implemented in a computer system 1100 such as a computer-readable recording medium. As shown in FIG. 4 , the computer system 1100 includes one or more processors 1110 , a memory 1130 , a user interface input device 1140 , and a user interface output device 1150 that communicate with each other via a bus 1120 . and storage 1160 . In addition, the computer system 1100 may further include a network interface 1170 coupled to the network 1180 . The processor 1110 may be a central processing unit or a semiconductor device that executes processing instructions stored in the memory 1130 or the storage 1160 . The memory 1130 and the storage 1160 may be various types of volatile or non-volatile storage media. For example, the memory may include a ROM 1131 or a RAM 1132 .

An apparatus and method for providing guidance information for spatial image shooting according to an embodiment of the present invention provide an overlay induction that allows a user to easily shoot a continuous spatial image shooting position and posture when shooting a space for a point cloud-based 3D restoration Image information, detailed posture guidance information through real-time optical tracking, information that induces immediate re-shooting in case of quality error by performing color difference and focus inspection in real time, management of multiple continuous image captures and real-time image transmission by creating and storing additional information It is possible to provide guidance information for optimal spatial image shooting, such as triggering the time point cloud generation server to automatically operate.

As described above, in the apparatus and method for providing guidance information for spatial image shooting according to an embodiment of the present invention, the configuration and method of the embodiments described above are not limitedly applicable, but the embodiments are various All or part of each of the embodiments may be selectively combined so that modifications may be made.

310: shooting setting unit 320: shooting induction unit
330: image management unit
1100: computer system 1110: processor
1120: bus 1130: memory
1131: rom 1132: ram
1140: user interface input device
1150: user interface output device
1160: storage 1170: network interface
1180: network

Claims (1)

In the method of providing guidance information for spatial image shooting of an induction information providing device for spatial image shooting,
setting a shooting environment and a shooting direction of a camera for spatial image shooting;
generating an overlay image from the image captured by the camera, and providing overlay guidance information generated based on the overlay image to the camera; and
determining whether optical tracking of the camera is possible, and providing, to the camera, optical guidance information generated in consideration of a movement position of a subject photographed in the image according to a result of determining whether optical tracking is possible;
Guide information providing method for spatial image shooting, characterized in that it comprises a.

Images