KR20020096750A - Apparatus and method for eliminating hindrance as making panorama image - Google Patents

Abstract

PURPOSE: An apparatus and method is provided to allow panoramic images to be easily synthesized by automatically eliminating hindrances from the photographed image. CONSTITUTION: A hindrance eliminating apparatus comprises a storage unit(100) for storing original picture data having the same background, outputting the stored original picture data in accordance with the request signal input from an external source, eliminating hindrances, and storing the complete panoramic image data; a synthesis control unit(200) for outputting the request signal for the original picture data having the same background to the storage unit, receiving original picture data for two or more pictures having the same background from the storage unit, eliminating hindrances except the fixed background by using the hindrance eliminating algorithm and background selecting algorithm, eliminating hindrances of original picture data obtained from photographing performed at a predetermined angle while rotating at 360 degrees, creating a single panoramic image by synthesizing each picture where hindrances are eliminated, and outputting the completed panoramic image data to the storage unit; a display unit(300) for displaying process of panoramic image creation; and a key input unit(400) for outputting an original picture data output key signal and a hindrance eliminating key signal to the synthesis control unit.

Description

Obstacle removal device and method for producing panorama image {Apparatus and method for eliminating hindrance as making panorama image}

The present invention relates to an obstacle removing device and a method for removing a panoramic image.

More specifically, when creating a panoramic image using a separate authoring tool, when the camera is rotated 360 degrees in a specific place at regular time intervals, an obstacle (a moving object, not a fixed background, For example, the present invention relates to an obstacle removing device and a method for removing a panorama image to automatically remove a car, a person, etc.).

In general, the purpose of using a panoramic image is to reproduce a field of view in any direction seen from the point where the photographing was taken, using a single panoramic image generated through photographing and compositing, thereby virtually all spaces indoors and out. To let them experience.

The panorama image is used for tourist attractions, organizations such as corporations / schools, real estate, and the like, and when the road image such as an intersection is used, it can be applied as a navigation system for checking a destination and a waypoint.

In order to produce such a panoramic image, the camera is first fixed to a predetermined shooting position (for example, at the center of an intersection in the case of a road), and then rotated 360 degrees to capture the surrounding background at a predetermined angle (for example, 30 degrees). do.

After taking a photo, the authoring tool for compositing a photo is used to create a complete panoramic image by combining the surrounding backgrounds of photographs taken at a certain angle into one, and linking the generated panoramic image to the Internet or using a car navigation system (Car navigation system). Navigation system to make it available to users.

However, when the panorama image is produced by the conventional method as described above, the subject is photographed by a moving object such as a person or a car, not a fixed background, because a person, a car, etc. are inevitably taken together depending on the situation at the time of surrounding background shooting. When the synthesized pictures were synthesized, the surrounding background did not coincide with each other, and there was a problem that a clean image could not be obtained due to obstacles such as moving objects.

In addition, in order to obtain a clear panorama image, a picture should be taken when there are no obstacles such as a person or a car, but this also has a problem that it is almost impossible to take a picture while the obstacle is removed.

An object of the present invention is to solve the above-mentioned problems, and after storing a series of two or more consecutive pictures at a predetermined time interval at a certain angle while rotating the camera 360 degrees in a specific place in a computer, a panoramic image through photo synthesis When creating a panoramic image using the authoring tool that creates the image, an obstacle removing device and a method for removing the obstacle during the production of the panoramic image to automatically remove the obstacles (moving objects, not fixed backgrounds) inside the photographed picture. There is.

1 is a block diagram schematically showing the configuration of an obstacle removing apparatus when producing a panoramic image according to the present invention;

2 is a view for explaining an obstacle removal algorithm applied in the present invention,

3 is a view for explaining a background region selection algorithm applied in the present invention;

4 is a flowchart showing an embodiment of a method for removing an obstacle when producing a panoramic image according to the present invention;

5 is a view for explaining a processing procedure of an embodiment of the method for removing an obstacle when producing a panoramic image according to the present invention of FIG. 4;

6 is a flowchart illustrating another embodiment of a method for removing an obstacle when producing a panoramic image according to the present invention.

Explanation of symbols on the main parts of the drawings

100: storage means

200: synthesis control means

300: display means

400: key input means

Obstacle removal apparatus for producing a panoramic image according to the present invention for achieving this object, by dividing the original photo data input from the camera by the original photo data having the same background photographed two or more at a predetermined time interval for each predetermined angle Storage means for storing and outputting original photo data stored based on an external original photo data request signal, and storing one panoramic image data completed by removing obstacles input from the outside; After outputting the original picture data request signal having the same background to the storage means according to the synthesized original picture data output key signal inputted by the picture synthesis performer, and receiving two or more original picture data having the same background from the storage means. Eliminate obstacles by leaving only the fixed background in the original photo data using the pre-stored obstacle elimination algorithm and the background area selection algorithm.In the same way, remove obstacles in the original photo data taken at every angle while rotating 360 degrees in the same place. Synthesis control means for synthesizing each photo after the obstacle is removed to generate a panoramic image, and outputting the completed panoramic image data to a storage means; Display the original picture data under the control of the composition control means, display of the obstacle elimination process, display of the result of the photo removal result with the obstacle removed, etc. Display means; And a key input means for outputting an original photograph data output key signal to be synthesized, which is input by a photographer, to produce a panoramic image, and an obstacle elimination key signal for synthesizing the picture to the composition control means.

In addition, according to one embodiment of the method for removing obstacles in the production of a panoramic image according to the present invention, (1) an original photo having the same background on the original photograph data photographed two times at a predetermined time interval at a specific place through a camera; Storing the data by dividing the data; (2) extracting pixels larger than a predetermined boundary value by comparing characteristic values of respective pixels of two original photograph data photographed at a predetermined time interval at a predetermined angle; (3) generating a boundary line of each region by converting the connection state of the extracted pixels into region information; (4) copying a portion other than the boundary line of each generated area (the portion where the two original photo data coincide) to the resultant image according to the obstacle removing process; (5) After creating a virtual line in the direction perpendicular to the boundary line of each region created through the above process (3), divide the virtual line into two parts based on the boundary line for the two original photo data. Selecting an area of original photographic data having a small difference in brightness between the two and copying the image to the result of the obstacle elimination processing; (6) determining whether or not all obstacle removing processes of all original photograph data having the same background photographed at a certain place at a predetermined time interval for each angle are finished; And (7) generating one completed panoramic image by using the plurality of obstacle removal processing result images generated for each predetermined angle according to the obstacle removing process.

In addition, another embodiment of the obstacle removing method when producing a panoramic image according to the present invention, (A) the original having the same background to the original photograph data photographed three or more at a predetermined time interval at a certain angle at a specific place through the camera Dividing and storing the picture data; (B) extracting pixels larger than a predetermined boundary value by comparing characteristic values of each pixel of three or more original photograph data photographed continuously at a specific time interval at a specific angle; (C) converting the connection state of the extracted pixels into region information to generate a boundary line of each region; (D) copying a portion other than the boundary line of each generated area (a portion where three or more original photo data coincide) to the resultant image according to the obstacle removing process; (E) comparing three or more original photo data for each region generated through the above process (C), selecting a region of data having the highest frequency for each region as a result of the comparison, and copying the image to the result of obstacle removal processing. Process; (F) determining whether or not all obstacle removing processes of all original photograph data having the same background photographed with three or more shots at certain angles in a specific place are finished; And (G) generating one completed panoramic image by using the plurality of obstacle removal processing result images generated for each predetermined angle according to the obstacle removing process.

Hereinafter, with reference to the accompanying drawings will be described in detail the obstacle removing device and method for producing a panoramic image of the present invention.

1 is a block diagram schematically showing the configuration of an obstacle removing apparatus when producing a panoramic image according to the present invention.

As shown, the storage means 100 is a photo having the same background to the original photo data input from the camera according to the operation of the photo-compositor (that is, the original photo data taken two or more at a predetermined time interval at a predetermined angle) And store the original picture data stored according to the original picture data request signal from the synthesis control means 200, which will be described later, and remove the obstacle inputted from the synthesis control means 200. Stores panoramic image data.

The synthesizing control means 200 stores the original photo data request signal having the same background continuously photographed at a predetermined angle in accordance with the synthesized original photo data output key signal input from the key input means 400 to the storage means 100. Outputs and receives two or more original photo data having the same background from the storage means 100, and then uses a previously stored obstacle removal algorithm and a background area selection algorithm according to the obstacle removal key signal input from the key input means 400. Remove the obstacle by leaving only the fixed background in the original photo data, and remove the obstacle from the original photo data taken at every angle while rotating 360 degrees in the same place in the same way. Creates a panoramic image and saves the completed panoramic image data And it stores the output in stage 100.

At this time, when removing the obstacle while leaving only the fixed background in the original photo data through the above-described composite control means 200, when using two original photo data, the composite control means 200, first two original photos By comparing the pixels of the data, we detect the area of the other part, create a virtual line in the direction perpendicular to the area boundary of the other part, and divide the virtual line into two parts based on the boundary for the two original photo data. An area of the original photographic data having a small difference in brightness between these two portions is selected as a fixed background (see Fig. 3).

In addition, when removing obstacles while leaving only a fixed background in the original photo data through the composite control means 200, when using three or more original photo data, the composite control means 200, each of the three or more original photographs After detecting the areas of different parts through pixel comparison of data, compare the original photo data for each area of different parts, and select the area of the data with the highest frequency for each area as the fixed background as a result of the comparison. . For example, when five original photo data are used, if three out of five regions of the five regions are identical, the three regions are regarded as a fixed background.

The display means 300 is a process (for example, display of original photo data, display of an obstacle removing process, and photo removal result data from which obstacles are removed) according to a panorama image currently in progress under the control of the composition control means 200. Display, etc.) on the screen for the photographer to confirm.

The key input unit 400 outputs the synthesized original picture data output key signal input by the picture synthesis performer to produce a panoramic image, an obstacle removing key signal when synthesizing the picture, and the like to the synthesis control means 200.

2 is a view for explaining an obstacle removal algorithm applied in the present invention, it will be described based on the two original picture data.

First, if there is a moving object in the original photo data A and B photographed at a predetermined time interval from the camera through the camera, the moving regions R2 and R3 are extracted through the pixel comparison of the two original photo data A and B. .

After the moving region is extracted through pixel comparison, the common portions (R1, portions other than the moving region) of the two original photo data A and B are copied as they are, and the moving region R2 on the left is the original photo data B. Copy from the corresponding area of, and the moving area (R3) on the right is removed from the corresponding area of the original picture data A to remove the obstacle (moving object).

FIG. 3 is a diagram for describing a background region selection algorithm applied in the present invention, and will be described based on regions R2 and R3 extracted according to the pixel comparison result of FIG. 2.

First, an imaginary line N is generated in a direction perpendicular to the boundary line L of each region R2 and R3, and then the intersection point of the boundary line L and the imaginary line N for each of the two original photograph data ( The difference between the brightness value of P) and the brightness value of the virtual line N excluding the intersection point P is calculated. The expression at this time is as follows.

At this time, E is the sum of the brightness value difference,

I (p) is the brightness value at point P,

n is the point excluding P on the imaginary line N.

After the operation as described above is finished, the area of the original photo data having a small difference can be selected to remove the obstacle (moving object).

Next, an embodiment of a method for removing obstacles in producing a panoramic image according to the present invention configured as described above will be described in detail with reference to FIGS. 4 to 6.

First, with reference to Figures 4 and 5 showing an embodiment of the method for removing obstacles in the production of a panorama image according to the present invention (using original photograph data taken two sheets at a predetermined time interval for each predetermined angle).

The photographer who wants to produce a panoramic image of a specific place first installs a tripod at the place to be photographed, attaches the camera parallel to the ground, and shoots by rotating the camera 360 degrees at 30 or 45 degree intervals. In this case, when taking a picture, two pictures are taken with a time difference for each viewing direction so that moving obstacles can be removed.

After photographing two sheets at a predetermined time interval at a specific place through the camera in this manner, the captured original photo data are classified into original photo data having the same background and stored in the storage means 100 (S400).

When the original picture data shot by the camera is stored in the storage means 100, the composite control means 200 receives two original picture data shot at a specific time interval at a specific angle as shown in FIG. 5A. In operation S410, pixels larger than a predetermined boundary value are extracted by comparing feature values of each original picture data. In this case, the image extracted through pixel comparison and the image binarized through a predetermined threshold value are shown in FIG. 5B.

Then, the connection state of the pixels extracted through the above-described process (S410) is converted into region information (that is, the region is extracted from the binarized image shown in FIG. 5B), and then a boundary line of each region is generated (S420). (See Figure 5c)

Thereafter, portions other than the boundary lines of the generated regions (the portions where the two original photo data coincide) are copied to the resultant image according to the obstacle removing process (S430).

In addition, as shown in Figure 3, after generating the virtual line (N) in a direction perpendicular to the boundary line (L) of each region generated through the above-described process (S420), the boundary line for the two original picture data Calculate the difference between the brightness value of the intersection point P of (L) and the virtual line N and the brightness value of the virtual line N excluding the intersection point P, and select an area of the original photo data with a small difference to The result of the removal process is copied to the image (S440). (See FIG. 5D.)

That is, the background portions of the two original photograph data shown in FIG. 5A are selected with respect to the boundary lines of the respective regions generated through the above-described process (S420), and the obstacles that are moving objects are removed, and the result is the result of the obstacle removing process. To copy.

As described above, after performing the obstacle removing process of two original photograph data photographed continuously at a specific angle, the composite control means 200 has all the same background photographed by two sheets at a certain time interval at a certain angle in a specific place. It is determined whether the obstacle removing process of the original picture data is finished (S450).

As a result of the determination, when the obstacle removing process of all original photograph data is completed, the synthesis control unit 200 generates one completed panoramic image by using the plurality of obstacle removing result images generated for each predetermined angle (S460).

Next, another embodiment of the method for removing an obstacle when producing a panoramic image according to the present invention will be described with reference to FIG. 6, which illustrates the use of original photograph data photographed at least three times at a predetermined time interval.

As described above, the photograph synthesis performer installs a tripod at the place to be photographed, attaches the camera to be parallel to the ground, and photographs while rotating 360 degrees at 30 or 45 degree intervals. In this case, when taking a picture, three or more pictures are taken with a time difference for each viewing direction so that moving obstacles can be removed.

After three or more shots are taken at a predetermined time interval at a specific place through the camera, the original photographed data are classified into original photograph data having the same background and stored in the storage means 100 (S600). .

When the original picture data shot by the camera is stored in the storage means 100, the composition control means 200 compares in advance the characteristic values of each pixel of the three or more original picture data shot at a specific angle at a predetermined time interval. Pixels larger than the specified boundary value are extracted (S610).

In addition, after converting the connection state of the pixels extracted through the above-described process (S610) into area information, a boundary line of each area is generated (S620).

Thereafter, portions other than the boundary lines of the generated areas (parts where the three original photo data coincide) are copied to the resultant image according to the obstacle removing process (S630).

In addition, three or more original photograph data are compared for each region generated through the above-described process (S620), and the region of the data having the highest frequency for each region is selected and copied to the obstacle elimination processing result image. (S640).

That is, when three original photo data among the five original photo data are the same as an example of performing an obstacle removing process using five original photo data, the composition control means 200 identifies an area belonging to the three original photo data. Select and copy the result of obstacle removal process to the image.

As described above, after the obstacle removing process of the original photograph data photographed three or more consecutively at a specific angle is performed, the composition control means 200 displays the same background photographed three or more at a predetermined time interval at a predetermined angle in a specific place. The branch determines whether the obstacle removing process of all original photo data is completed (S650).

As a result of the determination, when the obstacle removing process of all original photograph data is completed, the synthesis control unit 200 generates one completed panoramic image by using the plurality of obstacle removing result images generated for each predetermined angle (S660).

As described above, according to the obstacle removing apparatus and the method of producing the panoramic image of the present invention, the obstacle which is a moving object, not a fixed background, is automatically removed from the pictures taken two or more consecutively at a predetermined time interval while rotating 360 degrees. By doing so, moving images such as people, cars, and the like, which are inevitably taken with a fixed background, can be easily synthesized, and a clear panoramic image can be obtained without the influence of obstacles such as moving objects.

Herein, while the present invention has been described with reference to the preferred embodiments, those skilled in the art will variously modify the present invention without departing from the spirit and scope of the invention as set forth in the claims below. And can be changed.

Claims (5)

The original photo data input from the camera is classified and stored for each original photo data having the same background taken two or more at a predetermined time interval at a predetermined angle, and stored according to an external original photo data request signal. A storage means for outputting the first image data and removing the obstacle input from the outside; Output the original picture data request signal having the same background to the storage means according to the synthesized original picture data output key signal inputted by the picture synthesis performer, and input two or more original picture data having the same background from the storage means. After receiving the obstacle, remove the obstacle by leaving only the fixed background in the original photo data using the pre-stored obstacle elimination algorithm and the background area selection algorithm, and rotate the 360 degrees in the same place in the same way. Synthesis control means for synthesizing each picture from which the obstacle has been removed after the obstacle is removed, and outputting the completed panoramic image data to the storage means; Under the control of the composition control means, on the screen so that the photograph synthesis performer can check the progress of the current panorama image production, such as display of the original photo data, display of obstacle removal process, display of photo synthesis result data from which obstacles are removed, etc. Display means for displaying; And A panoramic image comprising a key input means for outputting an original photograph data output key signal to be synthesized by a photo synthesis operator to produce a panorama image, and a key input means for outputting an obstacle elimination key signal to the composition control means when synthesizing the photo; How to get rid of obstacles in manufacturing. The method of claim 1, wherein when the obstacle is removed while leaving only a fixed background in the original photo data through the composition control means, two original photo data are used. By comparing the pixel of each original picture data, the area of different parts can be detected. After creating a virtual line in a direction perpendicular to the area boundary line of another part, divide the virtual line into two parts based on the boundary line for the two original picture data, and use the small difference in brightness between the two parts. Obstacle removal method for producing a panorama image, characterized in that the area is selected as a fixed background. The method of claim 1, wherein when the obstacle is removed while leaving only the fixed background in the original photo data through the composition control means, three or more original photo data are used. By comparing the pixel of each original picture data, the area of different part is detected and then compare each original picture data by each area. As a result of the comparison, the method of removing obstacles in the production of a panorama image, wherein the area of the data having the highest frequency for each area is selected as a fixed background. (1) a process of dividing and storing the original photo data photographed two times at a predetermined time interval at a specific place through a camera for each original photo data having the same background; (2) extracting pixels larger than a predetermined boundary value by comparing characteristic values of each pixel of two original photograph data photographed at a predetermined angle at a predetermined time interval; (3) generating boundary lines of each region by converting the connection state of the extracted pixels into region information; (4) copying portions other than the boundary lines of each generated region (the portions where the two original photo data coincide) to the resultant image according to the obstacle removing process; (5) After generating a virtual line in a direction perpendicular to the boundary line of each region generated through the step (3), divide the virtual line into two parts based on the boundary line for the two original photo data. Selecting an area of the original photographic data having a small difference in brightness between parts and copying the image to the result of the obstacle elimination processing; (6) determining whether all obstacle removing processes of all original photographic data having the same background photographed at two predetermined intervals at certain angles at a specific place are finished; And And (7) generating a single panoramic image by using the plurality of obstacle removal processing result images generated at predetermined angles according to the obstacle removing process. (A) a process of dividing and storing three or more original photo data taken by a camera at a predetermined time interval in a specific place by original camera data having the same background; (B) extracting pixels larger than a predetermined boundary value by comparing the characteristic values of each pixel of the three or more original photograph data photographed continuously at a predetermined time interval at a specific angle; (C) generating a boundary line of each region by converting the connection state of the extracted pixels into region information; (D) copying a portion other than the boundary line of each region (where three or more original photo data coincide) to the resultant image according to the obstacle removing process; (E) comparing three or more original photo data for each region generated through the above process (C), selecting a region of data having the highest frequency for each region as a result of the comparison, and copying the image to the result of obstacle removal processing. process; (F) determining whether all obstacle removing processes of all original photographic data having the same background photographed three or more at a predetermined time interval at a predetermined angle are terminated at a specific place; And (G) Obstacle removal method in the production of a panoramic image comprising the step of generating a complete panoramic image by using a plurality of obstacle removal processing result image generated by a predetermined angle in accordance with the obstacle removal process.