KR102567379B1 - Realistic object creation system through 3d modeling automatic correction - Google Patents

Abstract

The present invention relates to a system for creating realistic objects through automatic 3D modeling correction implemented in a computing device including one or more processors and one or more memories storing instructions executable by the processors, in which objects are generated in multiple directions through a camera of the electronic device. When a video image corresponding to a captured video frame is acquired as a photographing object is captured, photographing angle information based on a sensed value of a gyro sensor mounted in the electronic device and the electronic device provide information about the photographing object from the electronic device. a photographing information receiving unit for receiving video image information obtained by photographing; When the reception of the photographing angle information and the video image information is completed, the received photographing angle information and the video image information are analyzed, and based on the analysis result, a virtual object that is a 3D stereoscopic model corresponding to the photographing object is created in 3D a virtual object analysis unit implemented in a virtual space and checking whether at least one defective pixel among pixels constituting the virtual object exists; and when it is confirmed that at least one defective pixel exists among the pixels constituting the virtual object due to the performance of the function of the virtual object analyzer, color code information and depth matched to each of pixels adjacent to the at least one defective pixel. A sensory object creation unit that checks value information and uses at least one of the identified color code information and depth value information to complete correction of the defective pixel to complete creation of a sensory object; Reality through 3D modeling automatic correction It is characterized in that it includes an object creation system. In addition to this, various embodiments understood through this document are possible.

Description

Realistic object creation system through 3D modeling automatic correction {REALISTIC OBJECT CREATION SYSTEM THROUGH 3D MODELING AUTOMATIC CORRECTION}

The present invention relates to a system for creating realistic objects through automatic correction of 3D modeling, and more specifically, a video image corresponding to a photographed video frame is obtained as photographing of an object is performed in multiple directions through a camera of an electronic device. In this case, the shooting angle information and the video image information are received from the electronic device, the virtual object, which is a 3D stereoscopic model corresponding to the shooting object, is implemented in a 3D virtual space, and the analysis of the virtual object is started, and the virtual object is created through the analysis result. If there is at least one defective pixel among the constituent pixels, correction of the defective pixel is completed by using at least one of color code information and depth value information matched to each of pixels adjacent to the at least one defective pixel, thereby realizing the realization. It relates to a technique that completes the creation of an object.

Recently, as the price drop and quality improvement cycle of 3D printers are gradually shortened, 3D printers are becoming more common for individual hobbies and daily necessities, unlike in the past, when they were mainly used in manufacturing. However, in order to do 3D modeling for 3D printing, professional programs such as AutoCAD and 3D Studio Max must be acquired, and there is a problem that it takes several months to several years to master. In addition, even if the user performs 3D modeling through a professional program, unless the user is a skilled expert, he or she cannot perform a highly precise work on the object of 3D modeling, so the output of the 3D printer may not be in good condition. .

Accordingly, the 3D printing industry is developing technologies that easily perform high-precision 3D modeling without professional knowledge of 3D modeling.

As an example, Korea Patent Publication No. 10-2019-0074562 (3D printing system using a 3D modeling authoring tool based on VR technology) utilizes VR technology to allow anyone to easily convert 3D modeling data into three dimensions without training in a professional 3D modeling tool. A technique for outputting is disclosed.

However, in the prior art described above, only a technology for authoring a VR object output from an HMD through a user's motion is disclosed, and shooting is performed for a shooting object in multiple directions through a camera of an electronic device. When a video image corresponding to the video frame is obtained, the shooting angle information and the video image information are received from the electronic device, and a virtual object, which is a 3D stereoscopic model corresponding to the shooting object, is implemented in a 3D virtual space to analyze the virtual object. and if at least one defective pixel exists among the pixels constituting the virtual object through the analysis result, at least one of color code information and depth value information matched to each of pixels adjacent to the at least one defective pixel is used. A technology for completing the correction of a defective pixel to complete generation of a sensory object has not been disclosed, and thus the need for a technology capable of solving this problem is emerging.

Accordingly, the present invention, when a video image corresponding to a photographed image frame is acquired as photographing of a photographed object in multiple directions is performed by a camera of an electronic device through a realistic object generation system through 3D modeling automatic correction, electronic By receiving photographic angle information and video image information from the device, a virtual object, a 3D three-dimensional model corresponding to the photographed object, is implemented in a 3D virtual space to start analyzing the virtual object, and the pixels constituting the virtual object through the analysis result If there is at least one defective pixel among the defective pixels, correction of the defective pixel is completed using at least one of color code information and depth value information matched to each of the pixels adjacent to the at least one defective pixel to create a tactile object. By completing the 3D modeling, the virtual object is automatically corrected through the pixels constituting the virtual object without professional knowledge of 3D modeling, providing work convenience to the user and at the same time providing work based on 3D modeling in good condition (e.g., modeling data or Its purpose is to provide 3D printer output).

In the realistic object generation system through 3D modeling automatic correction implemented in a computing device including one or more processors and one or more memories storing instructions executable by the processors according to an embodiment of the present invention, the camera of the electronic device When a video image corresponding to a photographed video frame is acquired as photographing of a photographed object in multiple directions is performed through the electronic device, photographing angle information based on a sensing value of a gyro sensor mounted in the electronic device and the electronic device a photographing information receiver configured to receive video image information acquired as the apparatus photographs the photographing object; When the reception of the photographing angle information and the video image information is completed, the received photographing angle information and the video image information are analyzed, and based on the analysis result, a virtual object that is a 3D stereoscopic model corresponding to the photographing object is created in 3D a virtual object analysis unit implemented in a virtual space and checking whether at least one defective pixel among pixels constituting the virtual object exists; and when it is confirmed that at least one defective pixel exists among the pixels constituting the virtual object due to the performance of the function of the virtual object analyzer, color code information and depth matched to each of pixels adjacent to the at least one defective pixel. A sensory object creation unit that checks value information and uses at least one of the identified color code information and depth value information to complete correction of the defective pixel to complete creation of a sensory object; Reality through 3D modeling automatic correction It is characterized in that it includes an object creation system.

The virtual object analysis unit, when the reception of the photographing angle information and the video image information is completed, starts analysis of the video image information, extracts a depth value and color of a photographed object on the video image information, and an image conversion unit for converting a 2D image of a photographed object photographed from multiple directions into a 3D image in which the extracted depth value and color are reflected; and when the conversion of the 3D image is completed, the size of the 3D image is corrected so that depth values of pixels indicating the same region of the photographing object become the same, and the 3D image is measured in a direction corresponding to the photographing angle information. It is preferable to include; a virtual object creation unit that creates a virtual object that is a 3D three-dimensional model by matching the images.

The virtual object analysis unit may include: a process starting unit that starts an analysis process for pixels constituting the virtual object when the creation of the virtual object is completed and implementation of the virtual object in the 3D virtual space is completed; When the analysis process starts, a first pixel among pixels constituting the virtual object is designated based on a pre-stored defective pixel derivation algorithm, and range pixels located within a designated range based on the first pixel are analyzed. a condition satisfaction checking unit that first checks whether at least one of the first pixel and the range pixels satisfies a designated defective pixel condition through the analysis result; and a defective pixel designation unit assigning identification information to a pixel that satisfies the specified defective pixel condition and designating the pixel as a defective pixel when at least one of the first pixel and the range pixels satisfies the designated defective pixel condition. it is possible

The pre-stored defective pixel derivation algorithm, when the analysis process starts, checks each of the pixels constituting the virtual object so that the depth value of the pixels and range pixels to which the color code information is not reflected is within a preset threshold range. It is possible that it is an algorithm that identifies at least one of the non-contiguous pixels exceeding or less than , assigning identification information to it, and designating it as the defective pixel.

The sensory object creation unit, when the defective pixel is identified among the pixels constituting the virtual object by performing the function of the defective pixel designation unit, excludes the defective pixel, but is located within a designated range based on the defective pixel. The average value of the color variation of each pixel in the range is checked, color code information corresponding to the checked average value is matched to the defective pixel, and the color based on the matched color code information is reflected to the defective pixel. a first color correction unit that performs correction; and when the function of the first color correction unit is completed, determining a range pixel located within the specified range and a depth value of the defective pixel based on the defective pixel, and determining the brightness based on the determined depth value of the defective pixel. and a second color correction implementer for performing secondary correction by reflecting the color reflected in the color and re-matching color code information matched to the defective pixel with color code information corresponding to the secondary corrected color. possible.

The sensory object creation unit checks whether a lighting object based on a lighting effect for the virtual object exists before the function of the first color correction unit is performed, and determines whether or not a lighting object exists in an area including defective pixels of the virtual object. When a lighting object is detected, the shape, color, and texture of the detected lighting object are analyzed to determine the effect of lighting, including the intensity and direction of lighting on the shooting object corresponding to the virtual object, the When the function of the second color correction unit is performed, the second color correction unit checks the depth value of the range pixels located within the designated range and the defective pixel, and determines the brightness based on the checked depth value. It is possible to request that the secondary correction be performed by reflecting the influence.

The sensory object generator may, when the virtual object analyzer identifies a non-contiguous pixel whose depth value versus range pixels exceeds or falls below a preset threshold range through the previously stored defective pixel derivation algorithm, adjacent to the identified non-contiguous pixel; It is possible to perform correction by calculating a depth average value of depth values of adjacent pixels and reflecting the depth value corresponding to the calculated depth average value to the identified non-contiguous pixels.

The system for creating realistic objects through automatic correction of 3D modeling according to the present invention creates realistic objects by automatically correcting a virtual object corresponding to a photographed object photographed through a camera, thereby providing work convenience to the user and at the same time, 3D modeling in good condition (e.g., modeling data or 3D printer output) can be provided.

In addition, the user can create a sensory object with a remarkably small error rate compared to a photographed object without acquiring specialized knowledge of 3D modeling.

1 is a block diagram of a sensory object generation system through automatic correction of 3D modeling according to an embodiment of the present invention.
2 is a block diagram of a virtual object analysis unit of a system for generating sensory objects through automatic correction of 3D modeling according to an embodiment of the present invention.
3 is another block diagram of a virtual object analysis unit of a system for generating sensory objects through automatic correction of 3D modeling according to an embodiment of the present invention.
4 is a diagram for explaining a pre-stored defective pixel derivation algorithm of a system for generating sensory objects through automatic correction of 3D modeling according to an embodiment of the present invention.
5 is a block diagram illustrating a sensory object generation unit of a sensory object generation system through automatic correction of 3D modeling according to an embodiment of the present invention.
6 is another block diagram illustrating a sensory object generation unit of a sensory object generation system through automatic correction of 3D modeling according to an embodiment of the present invention.
7 is a diagram for explaining an example of an internal configuration of a computing device according to an embodiment of the present invention.

In the following, various embodiments and/or aspects are disclosed with reference now to the drawings. In the following description, for purposes of explanation, numerous specific details are set forth in order to facilitate a general understanding of one or more aspects. However, it will also be appreciated by those skilled in the art that such aspect(s) may be practiced without these specific details. The following description and accompanying drawings describe in detail certain illustrative aspects of one or more aspects. However, these aspects are exemplary and some of the various methods in principle of the various aspects may be used, and the described descriptions are intended to include all such aspects and their equivalents.

References to “embodiment,” “example,” “aspect,” “example,” etc., used in this specification should not be construed as indicating that any aspect or design described is preferable to or advantageous over other aspects or designs. .

Also, the terms "comprises" and/or "comprising" mean that the feature and/or element is present, but excludes the presence or addition of one or more other features, elements and/or groups thereof. It should be understood that it does not.

In addition, terms including ordinal numbers such as first and second may be used to describe various components, but the components are not limited by the terms. These terms are only used for the purpose of distinguishing one component from another. For example, a first element may be termed a second element, and similarly, a second element may be termed a first element, without departing from the scope of the present invention. The terms and/or include any combination of a plurality of related recited items or any of a plurality of related recited items.

In addition, in the embodiments of the present invention, unless otherwise defined, all terms used herein, including technical or scientific terms, are those commonly understood by those of ordinary skill in the art to which the present invention belongs. have the same meaning. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and unless explicitly defined in the embodiments of the present invention, an ideal or excessively formal meaning not be interpreted as

1 is a block diagram of a sensory object generation system through automatic correction of 3D modeling according to an embodiment of the present invention.

Referring to FIG. 1 , a sensory object generation system 100 (hereinafter referred to as an automatic correction system) through 3D modeling automatic correction includes a shooting information receiver 101, a virtual object analyzer 103, and a sensory object generator 105. ) may be included.

According to an embodiment, when a video image corresponding to a captured image frame is obtained as a photographing object is captured in multiple directions through a camera of the electronic device 101a, the photographing information receiving unit 101 may perform the photographing of an object in multiple directions. Photography angle information based on a sensing value of a gyro sensor mounted in the electronic device 101 and video image information acquired as the electronic device 101 photographs the photographing object may be received from the electronic device 101. .

In the present invention, the electronic device 101a includes, for example, a display (or touch screen) as output and input means together with a processor and a memory in which a camera, a gyro sensor, and the automatic calibration system 100 are implemented. This may be implemented, and this may be understood as the same concept as the computing device mentioned in the description of FIG. 7 to be described later.

According to an embodiment, the camera of the electronic device 101a refers to a device (or module) that captures still images or videos.

According to another embodiment, the camera may include a camera including a depth vision function. A camera installed in a typical recent smart phone may refer to a camera in the present invention. Depth vision is classified as a Time-of-Flight (ToF) camera. The distance is calculated by the time it takes for the light shot toward the object to bounce off and return. You can accurately recognize the three-dimensional effect, spatial information, and movement of objects. A corresponding distance value is measured as a depth value in the present invention, and depth values for each pixel, that is, pixels within a boundary value of a photographed object among pixels constituting a photographed image frame may be measured.

Accordingly, when capturing a photographing object through the camera, the electronic device 101a may use various known techniques for identifying a photographing object from images constituting each image frame, such as boundary value identification and gray conversion identification.

According to an embodiment, the gyro sensor measures the rotational speed of the X, Y, and Z axes, and through this, the degree of tilt or rotation of the electronic device 101a can be measured. In relation to the foregoing, the gyro sensor may be replaced with, for example, an acceleration sensor, which is used for motion recognition and detection by measuring acceleration. Through this, when a user rotates or moves an electronic device such as a smartphone and photographs a photographing object, matches measurement values for each frame according to time stamps, and electronic device 101a when photographing a video image constituting each frame. ) may be used to generate photographing angle information and match them.

According to an embodiment, the photographing angle information is a value derived based on a sensing value of the gyro sensor, and means 3D angle information derived according to X, Y, and Z-axis coordinate values of the photographing object as an origin. can

According to an embodiment, the video image information is a kind of media obtained by combining the photographing angle information for each video image constituting a frame of an image acquired during a photographing time when a user captures an image using the electronic device 101a. It means a combination of meta data. In this case, the video image information may be information including a plurality of video images generated by photographing the photographing object in multiple directions.

According to an embodiment, when receiving the video image information, the photographing information receiving unit 101 may additionally receive photographing angle information about a photographed video frame corresponding to each video image based on the video image information.

According to an embodiment, when the reception of the photographing angle information and the video image information is completed, the virtual object analyzer 103 corresponds to the photographing object based on the received photographing angle information and the video image information. By implementing a virtual object, which is a 3D stereoscopic model, in a 3D virtual space, it is possible to check whether at least one defective pixel exists among pixels constituting the virtual object.

According to an embodiment, the virtual object analyzer 103 starts analyzing the received video image information when the reception of the photographing angle information and the video image information is completed, and the captured object on the video image information It is possible to extract a depth value and color for , and convert a photographing object, which is a 2D image based on the video image information, into a 3D image in which the extracted depth value and color are reflected.

In relation to the foregoing, when generating a 3D image of the photographed object, the virtual object analyzer 103 selects and utilizes images corresponding to six side views and perspective views of the photographed object as matching target information, It is possible to secure a constant quality of 3D modeling without being affected by the shooting situation or the user's shooting, and when a video is taken while moving around the shooting target naturally without professional knowledge, a certain drawing is automatically selected to create a 3D model. By performing modeling, the convenience can be greatly improved.

According to an embodiment, the depth value may mean a depth value measured for pixels constituting a photographed object in a video image corresponding to a photographed video frame used to generate a 3D image of the photographed object. . Accordingly, the virtual object analyzer 103 analyzes the pixels included in the image boundary of the captured object based on the video image information received from the electronic device 101a, and constructs the captured object through the analysis result. It is possible to extract a depth value matched to each of the pixels.

In the present invention, generating 3D image data using a combined video image may refer to a 3D image generation technology using a known 2D image, and in this case, depth values of pixels included in each video image information described above Among them, virtual objects and sensory objects may be created using pixels constituting a photographed object and depth values of the pixels. At this time, when generating 3D image data using the video image information, the virtual object analyzer 103 extracts depth values of pixels constituting a captured object included in each video image information, and extracts the extracted depth values. It is possible to implement pixels constituting the photographing object based on , match depth value information corresponding to the depth value to each of the implemented pixels, and match color code information of the pixels based on the extracted color. there is.

According to an embodiment, the virtual object analyzer 103 may generate a virtual object 103a that is a 3D stereoscopic model by implementing the 3D image in the 3D virtual space.

According to an embodiment, the virtual object analyzer 103 determines whether at least one defective pixel 103b exists among the pixels constituting the virtual object 103a when the generation of the virtual object 103a is completed. You can check. In relation to the foregoing, a detailed description of how the virtual object analyzer 103 determines whether the defective pixel 103b exists will be described with reference to FIGS. 3 and 4 .

According to an exemplary embodiment, the defective pixel 103b is that at least one of the pixels constituting the virtual object 103a is formed in an abnormal shape unlike the surrounding pixels, or the color and color of the surrounding pixels are different. Otherwise, it may mean a pixel formed in an abnormal shape.

According to an embodiment, the sensory object generating unit 105 has at least one defective pixel 103b among pixels constituting the virtual object 103a due to the function of the virtual object analyzing unit 103 being performed. If it is confirmed that this is the case, color code information 105a and depth value information 105b matched to each of pixels adjacent to the at least one defective pixel 103b are extracted, correction of the defective pixel is completed, and sensory object can create

According to an embodiment, the color code information 105a is pixels constituting the virtual object and information matched to the pixels, and more specifically, the color code information 105a corresponds to a color of a photographed object photographed through the camera. It may be information including a based color code (eg, BLUE(#0000FF, (0, 0, 255))).

According to an embodiment, the sensory object generator 105 extracts color code information 105a and depth value information 105b matched to each of pixels adjacent to the defective pixel, and then extracts the color code information 105a. ) to reflect the color corresponding to the average value of the color based on the defective pixel to perform color correction, or reflect the depth value corresponding to the average depth value of depth values based on the depth value information 105b to the defective pixel. Shape correction can be performed. A detailed description of how the sensory object generator 105 corrects the defective pixel will be described with reference to FIGS. 5 and 6 .

2 is a block diagram of a virtual object analysis unit of a system for generating sensory objects through automatic correction of 3D modeling according to an embodiment of the present invention.

Referring to FIG. 2, a realistic object generation system through 3D modeling automatic correction (eg, the realistic object creation system 100 through 3D modeling automatic correction in FIG. 1) (hereinafter referred to as an automatic correction system) is a virtual object analyzer 200 (eg, the virtual object analyzer 103).

According to an embodiment, the virtual object analyzer 200, when receiving the capturing angle information and the video image information from the capturing information receiving unit (eg, the capturing information receiving unit 101 of FIG. 1 ) is completed, the received capturing angle Based on the analysis result by analyzing information and the video image information, a virtual object, which is a 3D stereoscopic model corresponding to a photographed object, is implemented in a 3D virtual space, so that at least one defective pixel among pixels constituting the virtual object is You can check if it exists.

According to an embodiment, the virtual object analyzer 200 may include an image converter 201 and a virtual object generator 203 as detailed components for performing the above functions.

According to an embodiment, the image conversion unit 201 analyzes the received video image information to extract a depth value and color of a photographed object, and converts a 2D image of the photographed object photographed in multiple directions into the extracted It can be converted into a 3D image in which depth values and colors are reflected.

According to an embodiment, the video image information 201a is a kind of combination of the above-described photographing angle information for each video image constituting a frame of an image acquired during a photographing time when a user captures an image using an electronic device. It means a combination of media and meta data. In this case, the video image information 201a may be information including a plurality of video images generated by photographing the photographing object in multiple directions.

According to an embodiment, the image conversion unit 201 may generate a 3D image 201b through a 2D image of a photographed object based on the video image information. In more detail, the image conversion unit 201 may start analyzing the video image information and extract a depth value and color of a photographed object on the video image information based on an analysis result. At this time, the image conversion unit 201 implements pixels constituting the photographing object based on the depth value, matches depth value information corresponding to the depth value, and determines the color based on the extracted color. The 2D image may be converted into a 3D image 201b by matching color code information for pixels.

In relation to the above, the image conversion unit 201 may analyze the video image information through a pre-stored image analysis algorithm when analyzing the video image information. In relation to the above, the pre-stored image analysis algorithm may be an algorithm capable of generating a depth map similar to the measured depth map while learning other video image information and the measured depth map. Accordingly, the pre-stored image analysis algorithm may start analyzing the video image information and generate a depth map of the captured object based on the video image information.

In this case, the generated depth map may be information formed by pixels constituting the photographing object. Accordingly, the image conversion unit 201 extracts a depth value of the photographing object through the generated depth map, and provides depth value information corresponding to the extracted depth value to each of the pixels constituting the photographing object. can be matched. Also, the pre-stored image analysis algorithm may identify the color of the captured object based on the video image information. According to an embodiment, the virtual object generator 203 converts the 3D image 201b. When this is completed, the size of the 3D image 201b is corrected so that the depth values of pixels indicating the same area of the photographing object become the same, and the 3D image 201b is resized in a direction corresponding to the photographing angle information. A virtual object 203a, which is a 3D three-dimensional model, can be created by matching the .

In more detail, the image conversion unit 201 generates partial 3D images of each side from 2D images using depth values of pixels constituting a photographing object included in the video image information, and each side of each side The size of each partial 3D image is corrected so that the depth value of the pixel indicating the same point of the photographing object is the same through each partial 3D image by comparing the depth value and pixels of the pixels included in the partial 3D image. It is possible to create a virtual object implemented as a top, bottom, left, and right 3D image by matching partial 3D images in all directions (or multi-directions).

3 is another block diagram of a virtual object analysis unit of a system for generating sensory objects through automatic correction of 3D modeling according to an embodiment of the present invention.

Referring to FIG. 3, a realistic object generation system through automatic 3D modeling correction (eg, the realistic object generation system 100 through 3D modeling automatic correction in FIG. 1) (hereinafter referred to as an automatic correction system) is a virtual object analyzer 300 (eg, the virtual object analyzer 103).

According to an embodiment, the virtual object analyzer 300, when receiving the capturing angle information and the video image information from the capturing information receiving unit (eg, the capturing information receiving unit 101 of FIG. 1 ) is completed, the received capturing angle Based on the information and the video image information, a virtual object 301a, which is a 3D stereoscopic model corresponding to a photographed object, is implemented in a 3D virtual space, and analysis of the virtual object is started. It can be confirmed whether at least one defective pixel exists.

In relation to the above, the virtual object analyzer 300 implements the virtual object 301a, which is a 3D stereoscopic model corresponding to the captured object, in a 3D virtual space based on the received photographing angle information and the video image information. For a detailed description, refer to FIG. 2 .

According to an embodiment, the virtual object analyzer 300 starts analyzing the virtual object 301a and determines whether at least one defective pixel among the pixels constituting the virtual object 301a exists. As a configuration, it may include a process start unit 301, a condition satisfaction check unit 303, and a defective pixel designation unit 305.

According to an embodiment, the process initiator 301 is configured to generate the 3D object 301a as the creation of the virtual object 301a is completed by the virtual object generator (eg, the virtual object generator 203 of FIG. 2 ). When the implementation of the virtual object 301a in the virtual space is completed, an analysis process for pixels constituting the virtual object 301a may be started. The analysis process may be a process for determining whether the color and shape of the pixels constituting the virtual object are abnormal.

According to an embodiment, when the analysis process starts, the condition satisfaction checker 303 determines, based on a pre-stored defective pixel derivation algorithm, the first one of the pixels 305a constituting the virtual object 301a. A pixel 303a is designated, an analysis of range pixels located within a designated range based on the first pixel 303a is started, and at least one of the first pixel 303a and the range pixels is determined through the analysis result. It can be checked whether satisfies the designated defect pixel condition.

According to an embodiment, the pre-stored defective pixel derivation algorithm is a correlation between depth values of the first pixel and the range pixels and depth values, shapes, positions, and colors of the first pixel 303a and the range pixels, respectively. It may be a machine learning-based algorithm that determines

According to an embodiment, the condition satisfaction checking unit 303 analyzes the pixels 305a constituting the virtual object through the pre-stored defective pixel derivation algorithm, and among the pixels 305a constituting the virtual object, A correlation between the first pixel 303a and the depth value, shape, position, and color of a range pixel located within a range designated with respect to the first pixel 303a may be checked.

When the condition satisfaction checker 303 checks the correlation between the depth value, shape, position, and color of the first pixel 303a cell and the range pixels, the check result satisfies the specified defective pixel condition. you can check if it does.

According to an embodiment, the designated defective pixel condition is condition information set to determine whether some of the pixels 305a constituting the virtual object are abnormal pixels, and is changed by a manager or the previously stored defective pixels. They can be added and changed by derivation algorithms.

According to an embodiment, the condition satisfaction checking unit 303 analyzes the correlation between the depth value, shape, position, and color of the first pixel 303a and the range pixels through the pre-stored defective pixel derivation algorithm, , It is possible to determine whether the color and shape of the first pixel 303a and the range pixels are abnormal.

For example, the condition satisfaction checking unit 303 may start analyzing a correlation between the depth value, shape, position, and color of the first pixel 303a and the range pixels through the pre-stored defective pixel derivation algorithm. there is.

In relation to the above, the condition satisfaction checking unit 303 arbitrarily designates a first pixel among pixels constituting the bone net of the virtual object 301a having the shape of an SUV through a pre-stored defective pixel derivation algorithm to obtain the first pixel. By checking the color code information of one pixel 303a, it can be identified that the color of the first pixel 303a is white pearl. At this time, the condition satisfaction checking unit 303 checks color code information of pixels located within a designated range based on the first pixel 303a, and identifies that the color of the pixels in the range is aurora black pearl. .

In relation to the above, the condition satisfaction checking unit 303 identifies the shape of the first pixel 303a of the white pearl color and the shape of the range pixel of the aurora black pearl color, so that the shape of the range pixel is It can be identified that it is an upper area, and it can be confirmed that the shape of the first pixel 303a is an abnormal shape located in one area of the upper area of the bonnet. Accordingly, since the shape of the first pixel is abnormal, the condition satisfaction checking unit 303 determines that the color code information corresponding to the aurora black pearl color is not matched and the color code information corresponding to the white pearl color is matched. , it can be determined that the first pixel 303a is included in the predetermined defective pixel condition.

According to an exemplary embodiment, when at least one of the first pixel 303a and the range pixels satisfies the specified defective pixel condition, the defective pixel specifying unit 305 determines a pixel satisfying the specified defective pixel condition. Identification information may be assigned to designate a defective pixel.

According to an embodiment, the identification information is information given to a pixel when at least one of the first pixel 303a and the range pixels satisfies the designated defective pixel condition, and is provided to a sensory object generator (eg: This may be information allowing the sensory object generator 105 of FIG. 1 to recognize the defective pixel.

For example, the defective pixel designation unit 305, when it is determined by the condition satisfaction checking unit 303 that the first pixel 303a is included in the predetermined defective pixel condition, the first pixel ( 303a) may be assigned the identification information. At this time, the identification information given to the first pixel 303a causes the sensory object generator to determine whether the first pixel 303a has an abnormal pixel shape among the predetermined defective pixel conditions, and abnormal color code information. It may be information for identifying that is matched.

4 is a diagram for explaining a pre-stored defective pixel derivation algorithm of a system for generating sensory objects through automatic correction of 3D modeling according to an embodiment of the present invention.

Referring to FIG. 4, a sensory object creation system through automatic correction of 3D modeling (eg, the sensory object creation system 100 through automatic correction of 3D modeling in FIG. 1) (hereinafter referred to as an automatic correction system) is a condition satisfaction confirmation unit ( 401) (eg, condition satisfaction confirmation unit 303).

According to an embodiment, when the analysis process is started by the process starting unit (eg, the process starting unit 301 of FIG. 3 ), the condition satisfaction checking unit 401 is based on a previously stored defective pixel deriving algorithm 405. , designate a first pixel among the pixels 403 constituting the virtual object, start analyzing range pixels located within a designated range based on the first pixel, and analyze the first pixel and It may be checked whether at least one of the range pixels satisfies a designated defective pixel condition.

According to an embodiment, the pre-stored defective pixel derivation algorithm 405 analyzes the correlation between the depth value, shape, position, and color of each of the pixels 403 constituting the virtual object, and based on the analysis result, It may be an algorithm for checking whether at least one of the pixels satisfies a predetermined defective pixel condition.

In addition, when the analysis process is started, the pre-stored defective pixel derivation algorithm 405 checks each of the pixels constituting the virtual object, and determines a depth value relative to a pixel to which color code information is not reflected and a range pixel. It may be an algorithm that identifies at least one of the non-contiguous pixels that exceed or fall below a set threshold range.

That is, the pre-stored defective pixel derivation algorithm 405 derives a defective pixel through a correlation between the depth value, shape, position, and color of each of the first pixel and the range pixels among the pixels 403 constituting the virtual object. It may be an algorithm that identifies non-contiguous pixels whose depth value is greater than or less than a preset threshold range compared to pixels and range pixels to which the color code information is not reflected.

In relation to the above, the predetermined threshold range is configured based on an average depth value that is an average of depth values of pixels in the range, and to identify pixels whose depth value exceeds or is less than a specified value compared to the average depth value. It may be reference information for

For example, the pre-stored defective pixel derivation algorithm 405 may be a PointNet-based deep learning algorithm. The condition satisfaction confirmation unit 401 learns previously obtained or input virtual objects through the PointNet-based deep learning algorithm, thereby determining the depth value, shape, location, and depth value of each of the pixels 403 included in the virtual objects. The state of the pixel of the virtual object may be checked by determining the color or the like. In addition, the pre-stored defective pixel derivation algorithm 405 may be an algorithm that checks the depth value, arrangement (shape and location) of various abnormal pixels, and color correlation between neighboring pixels through other virtual objects.

That is, the previously stored defective pixel derivation algorithm 405 constructs a virtual object through a newly input virtual object by machine learning not only a PointNet-based deep learning algorithm but also a previously acquired or input virtual object. It is not limited to this if it is a machine learning-based algorithm capable of determining whether or not pixels are abnormal and confirming pixels in an abnormal state.

For another example, the pre-stored defective pixel derivation algorithm 405 may be an automatic recognition standardization algorithm. The automatic recognition standardization algorithm is based on machine learning for the condition satisfaction checking unit 401 to check the correlation between the depth value, shape, position and color of each pixel constituting the virtual object and classify it as an abnormal pixel. It can be an algorithm.

5 is a block diagram illustrating a sensory object generation unit of a sensory object generation system through automatic correction of 3D modeling according to an embodiment of the present invention.

Referring to FIG. 5, a sensory object generation system through automatic correction of 3D modeling (eg, the sensory object generation system 100 through automatic correction of 3D modeling in FIG. 1) (hereinafter referred to as an automatic correction system) includes a sensory object generator ( 500) (eg, the sensory object generator 105).

According to an embodiment, the sensory object generator 500 detects at least one defect among pixels constituting a virtual object due to the performance of the function of the virtual object analyzer (eg, the virtual body analyzer 103 of FIG. 1 ). If it is confirmed that the pixel exists, color code information and depth value information matched to each of pixels adjacent to the at least one defective pixel are extracted, and at least one of the extracted color code information and depth value information is used to find the defect. By completing the pixel correction, the creation of the realistic object can be completed.

According to an embodiment, the sensory object creation unit 500 may include a first color correction unit 501 and a second color correction unit 503 as a detailed configuration for performing the above functions. .

According to an embodiment, the first color correction unit 501 determines pixels 501a constituting a virtual object by performing a function of a defective pixel designation unit (eg, the defective pixel designation unit 305 of FIG. 3 ). If the defective pixel 501b is identified among the defective pixels 501b, the average value of color variation of each of the range pixels located within the designated range based on the defective pixel 501b is checked, and the defective pixel 501b is excluded. First correction may be performed by matching color code information corresponding to the checked average value to the defective pixel 501b and reflecting the color based on the matched color code information to the defective pixel 501b.

According to an embodiment, the first color correction unit 501 checks whether identification information matches each of the pixels 501a constituting the virtual object, and defects the pixel to which the identification information matches. It can be identified as pixel 501b. In this case, the first color correction unit 501 may check whether the defective pixel 501b satisfies one of the predetermined defective pixel conditions through the identification information.

According to an exemplary embodiment, when the identification of the defective pixel 501b is completed, the first color correction unit 501 determines the color variation of each of the range pixels located within a designated range based on the defective pixel 501b. You can check the average value. In this case, when checking the average value of the color variation, the first color correction unit 501 may check the average value of the color variation through only the range pixels excluding the defective pixel 501b.

According to an embodiment, the average value of the color variation may mean an average color according to a color change based on color code information matched to each pixel in the range. For example, when the range pixels are sequentially arranged in the order of a second pixel, a third pixel, a fourth pixel, and a fifth pixel, the first color correction unit 501 performs the second pixel, the third pixel, and the third pixel. Check the color code information matched to each of the pixels, the 4th pixel and the 5th pixel, and the color of the 2nd pixel is blue, the color of the 3rd pixel is navy, the color of the 4th pixel is dark blue, and the color of the 5th pixel is blue. It can be identified that the color of is black.

At this time, the first color correction unit 501 determines a first weight matched to blue, a second weight matched to navy, a third weight matched to dark blue, and black through a pre-stored color code table. A fourth weight matched to a color is checked, and a value obtained by summing the first weight, second weight, third weight, and fourth weight and dividing the result by the number of pixels in the range can be checked as an average value for color variation.

In relation to the above, the pre-stored color code table may be a data table in which colors corresponding to the color code information and information in which weights are matched for each color are stored for each color.

Accordingly, when the average value is confirmed, the first color correction unit 501 identifies a color matched with a weight corresponding to the average value through the pre-stored color code table, and assigns the color to the identified color. By matching the color code information for the color code to the defective pixel 501b, a color based on the matched color code information may be reflected in the defective pixel 501b.

According to an exemplary embodiment, when the function of the first color correction unit 501 is completed, the second color correction unit 503 is located within the specified range based on the defective pixel 501b. and the depth value of the defective pixel 501b is checked, the lightness based on the checked depth value is applied to the color reflected in the defective pixel to perform secondary correction, and the color code information matched to the defective pixel is reflected. It can be re-matched with color code information corresponding to the secondary corrected color.

In relation to the above, the function of the second color correction unit 503 may or may not be performed based on the identification information of the defective pixel 501b identified by the first color correction unit 501. there is.

In more detail, the defective pixel condition for the identification information given to the defective pixel 501b identified by the first color correction unit 501 relates only to the color of the pixel and not to the shape of the pixel. If it is confirmed as , only the color of the defective pixel 501b may be corrected by the first color correction unit 501 . In addition, when the defective pixel condition for the identification information assigned to the defective pixel 501b identified by the first color correction unit 501 is confirmed as a condition related to the color and shape of the pixel, the first Functions of the color correction unit 501 and the second color correction unit 503 may be performed.

According to an embodiment, the second color correction unit 503 may check a range pixel located within the designated range based on the defective pixel 501b and a depth value of the defective pixel 501b. In this case, the second color correction unit 503 may reflect the brightness corresponding to the confirmed depth value to the color reflected in the defective pixel 501b. In more detail, the second color correction unit 503 checks brightness information to which the checked depth value is matched among a plurality of pieces of brightness information stored in a pre-stored brightness table, and determines the brightness based on the checked brightness information. This can be reflected in the defective pixel 501b. The plurality of pieces of lightness information may be information in which lightness information corresponding to the depth value is matched.

Accordingly, as the brightness of the defective pixel 501b whose color is reflected by the first color correction unit 501 is reflected by the second color correction unit 503, the matched color code information is changed. can In this case, the changed color code information may be information corresponding to a color in which the brightness reflected by the second color correction unit 503 is reflected in the color reflected by the first color correction unit 501 .

According to an embodiment, before the function of the first color correction unit 501 is performed, the sensory object generator 500 checks whether a lighting object based on a lighting effect for the virtual object exists, When the lighting object is detected in one area including the defective pixel 501b of the virtual object, the intensity of illumination of the photographing object corresponding to the virtual object by analyzing the shape, color and texture of the detected lighting object And it is possible to determine the influence of lighting including the irradiation direction.

According to an embodiment, the lighting object may be a graphic effect having a shape corresponding to a lighting effect located in one area of the virtual object. Accordingly, when confirming that the lighting object exists in one area of the virtual object, the sensory object generator 500 may analyze the shape, color, and texture of the lighting object. In more detail, the sensory object generator 500 may analyze the shape, color, and texture of the lighting object for pixels constituting the virtual object through an image analysis algorithm.

Accordingly, the sensory object generator 500 may determine the effect of lighting based on the shape, color, and texture of the lighting object analyzed through the pre-stored image analysis algorithm. In relation to the foregoing, the influence of the lighting may include an intensity of lighting and a direction in which the lighting is radiated to pixels constituting the virtual object.

According to an embodiment, the pre-stored image analysis algorithm may be an algorithm that analyzes the shape, color, and texture of a lighting object for pixels constituting other virtual objects, and machine-learns the analysis result. Accordingly, the pre-stored image analysis algorithm may determine the effect of lighting on the pixels constituting the virtual object by analyzing the shape, color, and texture of the lighting object for the pixels constituting the virtual object.

According to an embodiment, the function of the second color correction unit 503 is performed in a state in which the sensory object generator 500 has completed determining the effect of lighting on the pixels constituting the virtual object. In this case, the second color correction unit 503 checks the depth value of the range pixels located within the designated range and the defective pixel 501b, and reflects the influence of the lighting on the brightness based on the checked depth value. A second correction can be made.

That is, before the function of the second color correction unit 503 is performed, the sensory object creation unit 500 determines the influence of the lighting, and determines the intensity of the lighting and irradiation of the lighting based on the determined influence of the lighting. The direction may be reflected together with the brightness reflected in the defective pixel 501b.

6 is another block diagram illustrating a sensory object generation unit of a sensory object generation system through automatic correction of 3D modeling according to an embodiment of the present invention.

Referring to FIG. 6, a sensory object generation system through automatic correction of 3D modeling (eg, the sensory object generation system 100 through automatic correction of 3D modeling in FIG. 1) (hereinafter referred to as an automatic correction system) includes a sensory object generator ( 601) (eg, the sensory object generator 105).

According to an embodiment, the sensory object generating unit 601 is configured to at least among pixels constituting a virtual object due to the performance of the function of the virtual object analyzing unit 603 (eg, the virtual object analyzing unit 103 of FIG. 1 ). If it is confirmed that one defective pixel exists, color code information and depth value information matched to each of pixels adjacent to the at least one defective pixel are extracted, and at least one of the extracted color code information and depth value information is extracted. It is possible to complete the creation of realistic objects by completing corrections for defective pixels.

In relation to the foregoing, a detailed description of how the sensory object generator 601 corrects the color reflected in the defective pixel will be referred to FIGS. 4 and 5 . In FIG. 6, only a description for correcting the depth value of a defective pixel is disclosed.

According to an embodiment, the sensory object generator 601 determines that the virtual object analyzer 603 detects non-contiguous areas in which a depth value versus range pixels exceeds or falls below a preset threshold range through the pre-stored defective pixel derivation algorithm. When a pixel is identified, a depth average value of depth values of adjacent pixels adjacent to the identified non-contiguous pixel is calculated, and a depth value corresponding to the calculated depth average value is reflected to the identified non-contiguous pixel for correction. can be carried out.

In relation to the foregoing, the predetermined threshold range is determined by the virtual object analyzer 603 as an average of depth values of pixels within a designated range based on a first pixel, which is one of pixels constituting a virtual object. It may be a configuration in which a depth value is reflected. For example, when the average depth value of the range pixels is 5, the predetermined threshold range may be 3 to 7 to which -2 and +2 are applied based on 5.

In relation to the foregoing, the sensory object generator 601 is configured to select a predetermined threshold range of 3 pixels among pixels included in a region (eg, first region) included in the range pixels among pixels constituting the virtual object. to identify non-contiguous pixels greater than or less than 7. Accordingly, the sensory object generating unit 601 calculates a depth average value of depth values of adjacent pixels adjacent to the identified non-contiguous pixel, and converts the depth value corresponding to the calculated depth average value to the identified non-contiguous pixel. Correction can be performed by reflecting it to the pixel.

That is, the sensory object generation unit 601 identifies non-contiguous pixels whose depth value compared to range pixels exceeds or falls below a preset threshold range through the pre-stored defective pixel derivation algorithm, and identifies non-contiguous pixels adjacent to the identified non-contiguous pixels. Pixel correction for a local range may be performed by calculating the average depth value of adjacent pixels and correcting the non-contiguous pixels.

According to an embodiment, the sensory object generation unit 601 may complete generation of the sensory object by reflecting an average value of the adjacent pixels to the non-contiguous pixels. The sensory object may refer to a completed virtual object created as the automatic correction system automatically corrects a virtual object corresponding to a photographed object.

7 is a diagram for explaining an example of an internal configuration of a computing device according to an embodiment of the present invention.

7 illustrates an example of an internal configuration of a computing device according to an embodiment of the present invention, and in the following description, descriptions of unnecessary embodiments overlapping with those of FIGS. 1 to 6 will be omitted. do it with

As shown in FIG. 7, a computing device 10000 includes at least one processor 11100, a memory 11200, a peripheral interface 11300, an input/output subsystem ( It may include at least an I/O subsystem (11400), a power circuit (11500), and a communication circuit (11600). In this case, the computing device 10000 may correspond to a user terminal connected to the tactile interface device (A) or the aforementioned computing device (B).

The memory 11200 may include, for example, high-speed random access memory, magnetic disk, SRAM, DRAM, ROM, flash memory, or non-volatile memory. there is. The memory 11200 may include a software module, a command set, or other various data necessary for the operation of the computing device 10000.

In this case, access to the memory 11200 from other components, such as the processor 11100 or the peripheral device interface 11300, may be controlled by the processor 11100.

Peripheral interface 11300 may couple input and/or output peripherals of computing device 10000 to processor 11100 and memory 11200 . The processor 11100 may execute various functions for the computing device 10000 and process data by executing software modules or command sets stored in the memory 11200 .

Input/output subsystem 11400 can couple various input/output peripherals to peripheral interface 11300. For example, the input/output subsystem 11400 may include a controller for coupling a peripheral device such as a monitor, keyboard, mouse, printer, or touch screen or sensor to the peripheral interface 11300 as needed. According to another aspect, input/output peripherals may be coupled to the peripheral interface 11300 without going through the input/output subsystem 11400.

The power circuit 11500 may supply power to all or some of the terminal's components. For example, power circuit 11500 may include a power management system, one or more power sources such as a battery or alternating current (AC), a charging system, a power failure detection circuit, a power converter or inverter, a power status indicator or power It may contain any other components for creation, management and distribution.

The communication circuit 11600 may enable communication with another computing device using at least one external port.

Alternatively, as described above, the communication circuit 11600 may include an RF circuit and transmit/receive an RF signal, also known as an electromagnetic signal, to enable communication with another computing device.

The embodiment of FIG. 7 is only an example of the computing device 10000, and the computing device 11000 may omit some of the components shown in FIG. 7, further include additional components not shown in FIG. It may have a configuration or arrangement combining two or more components. For example, a computing device for a communication terminal in a mobile environment may further include a touch screen or a sensor in addition to the components shown in FIG. , Bluetooth, NFC, Zigbee, etc.) may include a circuit for RF communication. Components that may be included in the computing device 10000 may be implemented as hardware including one or more signal processing or application-specific integrated circuits, software, or a combination of both hardware and software.

Methods according to embodiments of the present invention may be implemented in the form of program instructions that can be executed through various computing devices and recorded in computer readable media. In particular, the program according to the present embodiment may be configured as a PC-based program or a mobile terminal-only application. An application to which the present invention is applied may be installed in a user terminal through a file provided by a file distribution system. For example, the file distribution system may include a file transmission unit (not shown) that transmits the file according to a request of a user terminal.

The device described above may be implemented as a hardware component, a software component, and/or a combination of hardware components and software components. For example, devices and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), It may be implemented using one or more general purpose or special purpose computers, such as a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. A processing device may run an operating system (OS) and one or more software applications running on the operating system. A processing device may also access, store, manipulate, process, and generate data in response to execution of software. For convenience of understanding, there are cases in which one processing device is used, but those skilled in the art will understand that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that it can include. For example, a processing device may include a plurality of processors or a processor and a controller. Other processing configurations are also possible, such as parallel processors.

Software may include a computer program, code, instructions, or a combination of one or more of the foregoing, which configures a processing device to operate as desired or processes independently or collectively. You can command the device. Software and/or data may be any tangible machine, component, physical device, virtual equipment, computer storage medium or device, intended to be interpreted by or to provide instructions or data to a processing device. may be permanently or temporarily embodied in Software may be distributed on networked computing devices and stored or executed in a distributed manner. Software and data may be stored on one or more computer readable media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. - includes hardware devices specially configured to store and execute program instructions, such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter, as well as machine language codes such as those produced by a compiler. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

As described above, although the embodiments have been described with limited examples and drawings, those skilled in the art can make various modifications and variations from the above description. For example, the described techniques may be performed in an order different from the method described, and/or the components of the described system, structure, device, circuit, etc. may be combined or combined in a different form than the method described, or other components may be used. Or even if it is replaced or substituted by equivalents, appropriate results can be achieved. Therefore, other implementations, other embodiments, and equivalents of the claims are within the scope of the following claims.

Claims (7)

A realistic object generation system through automatic correction of 3D modeling implemented in a computing device including one or more processors and one or more memories storing instructions executable by the processors,
When a video image corresponding to a photographed video frame is obtained as a photograph of a photographed object is performed in multiple directions through a camera of the electronic device, photographing based on a sensing value of a gyro sensor mounted in the electronic device from the electronic device a photographing information receiver configured to receive angle information and video image information acquired as the electronic device photographs the photographing object;
When the reception of the photographing angle information and the video image information is completed, the received photographing angle information and the video image information are analyzed, and based on the analysis result, a virtual object that is a 3D stereoscopic model corresponding to the photographing object is created in 3D a virtual object analysis unit implemented in a virtual space and checking whether at least one defective pixel among pixels constituting the virtual object exists; and
When it is confirmed that at least one defective pixel exists among the pixels constituting the virtual object due to the performance of the function of the virtual object analyzer, color code information and depth value matched to each of pixels adjacent to the at least one defective pixel A sensory object generator configured to confirm information and complete generation of a sensory object by completing correction for the defective pixel using at least one of the identified color code information and depth value information;
The virtual object analysis unit,
When the reception of the photographing angle information and the video image information is completed, analysis of the video image information is started, a depth value and a color of a photographed object on the video image information are extracted, and the photographing taken in multiple directions is performed. an image conversion unit which converts a 2D image of an object into a 3D image in which the extracted depth value and color are reflected; and
When the conversion of the 3D image is completed, the size of the 3D image is corrected so that depth values of pixels indicating the same region of the photographing object become the same, and the 3D image is viewed in a direction corresponding to the photographing angle information. Including; virtual object generation unit for generating a virtual object that is a 3D three-dimensional model by matching the
The virtual object analysis unit,
a process initiation unit that starts an analysis process for pixels constituting the virtual object when implementation of the virtual object in the 3D virtual space is completed as the creation of the virtual object is completed;
When the analysis process starts, a first pixel among pixels constituting the virtual object is designated based on a pre-stored defective pixel derivation algorithm, and range pixels located within a designated range based on the first pixel are analyzed. a condition satisfaction checking unit that first checks whether at least one of the first pixel and the range pixels satisfies a designated defective pixel condition through the analysis result; and
When at least one of the first pixel and the range pixels satisfies the designated defective pixel condition, a defective pixel designation unit assigning identification information to a pixel satisfying the designated defective pixel condition and designating the pixel as a defective pixel;
The pre-stored defective pixel derivation algorithm,
When the analysis process starts, each of the pixels constituting the virtual object is identified, and at least among pixels to which the color code information is not reflected and non-contiguous pixels whose depth value is greater than or less than a preset threshold range compared to range pixels An immersive object creation system through automatic correction of 3D modeling, characterized in that it is an algorithm that identifies one and assigns identification information to it as the defective pixel.
delete delete delete According to claim 1,
The sensory object creation unit,
When the defective pixel is identified among the pixels constituting the virtual object due to the performance of the function of the defective pixel specifying unit, the defective pixel is excluded, but the color variation of each of the range pixels located within the designated range based on the defective pixel A first correction is performed by checking an average value for , matching color code information corresponding to the checked average value to the defective pixel, and reflecting a color based on the matched color code information to the defective pixel. Color correction implementation unit; and
When the function of the first color correction unit is completed, a range pixel located within the designated range and a depth value of the defective pixel are checked based on the defective pixel, and brightness based on the checked depth value is assigned to the defective pixel. and a second color correction unit for performing secondary correction by reflecting the reflected color and re-matching color code information matched to the defective pixel with color code information corresponding to the secondary corrected color. Realistic object creation system through automatic correction of 3D modeling.
According to claim 5,
The sensory object creation unit,
Before the function of the first color correction unit is performed, it is checked whether a lighting object based on the lighting effect for the virtual object exists, and the lighting object is detected in an area including a defective pixel of the virtual object. , In a state in which the shape, color, and texture of the detected lighting object are analyzed, and the influence of lighting including the intensity and irradiation direction of lighting on the shooting object corresponding to the virtual object is determined, the second color correction unit When the function is performed, the second color correction unit checks the depth value of the range pixels located within the designated range and the defective pixel, reflects the influence of the lighting on the brightness based on the checked depth value, A system for creating realistic objects through automatic correction of 3D modeling, characterized in that for requesting to perform difference correction.
According to claim 6,
The sensory object creation unit,
When the virtual object analyzer identifies a non-contiguous pixel having a depth value relative to range pixels greater than or less than a preset threshold range through the previously stored defective pixel derivation algorithm, a depth value for an adjacent pixel adjacent to the identified non-contiguous pixel A realistic object generation system through automatic correction of 3D modeling, characterized in that by calculating a depth average value of , and performing correction by reflecting a depth value corresponding to the calculated depth average value to the identified non-contiguous pixels.

Images