KR102402643B1 - 3D color modeling optimization processing system - Google Patents

Abstract

According to the present invention, an optimization processing system of a 3D modeling comprises: a 3D image generating module generating a 3D image from an object image; a detection module for detecting an outline and a surface surrounded by an outline from the 3D image; a region dividing module generating a plurality of regions by dividing the surface; a color recognition module for recognizing colors of the divided regions; and an optimization execution module including a group setting unit configured to set a group including at least one of the regions and a correction execution unit configured to collectively correct color of the group.

Description

Color optimization processing system of 3D modeling {3D color modeling optimization processing system}

The present invention relates to a color optimization processing system for three-dimensional modeling, and more specifically, through a method of dividing a surface of a three-dimensional image in modeling a three-dimensional image, and collectively correcting the color of the divided surface, 3 It relates to a color optimization processing system for three-dimensional modeling, which enables the capacity to be optimized for a three-dimensional image.

Recently, as broadcasting services with high definition (HD) resolution are expanding not only domestically but also worldwide, many users are getting used to high-resolution and high-definition images, and accordingly, many institutions are spurring the development of next-generation imaging devices. Also, along with HDTV, as interest in Ultra High Definition (UHD) having a resolution four times higher than that of HDTV is increasing, compression technology for higher resolution and high-definition images is required.

The image compression technology includes an inter prediction technology that predicts a pixel value included in the current picture from temporally previous and/or later pictures, and an intra prediction technology that predicts a pixel value included in the current picture using pixel information in the current picture. (intra) prediction technology, a transformation technology that decomposes an original video signal into a signal in the frequency domain, an inverse transformation technology that restores a frequency domain signal to an original video signal, assigns a short code to a symbol with a high frequency of appearance and assigns a short code to the frequency of appearance There is an entropy encoding technique in which a long code is assigned to a symbol having a low . Image data may be efficiently compressed by these image compression techniques.

As a prior art for such image compression, 'shape-adaptive model-based codec for lossy and lossless image compression' is disclosed in Korean Patent Laid-Open No. 10-2018-0117155. The invention relates to a method and codec for image and video compression. Embodiments of the invention provide binary shape as well as matte and soft segmentation image compression by decomposing input shapes into deterministic and probabilistic components for flexible lossy and lossless coding. Includes a new shape-adaptive model-based codec (SAM) that supports The invention can provide inter/intraprediction, and is flexibly adapted between lossy and lossless modes with various parameters for compression quality control. The compression module may also be adapted with a number of other compression techniques.

However, further, there is a need to provide a color optimization processing system for 3D modeling that optimizes capacity while maintaining quality by color-based optimization of a 3D image with a huge capacity.

The main object of the present invention is to optimize capacity while maintaining quality by collectively correcting three-dimensional images based on color.

Another object of the present invention is to specify a region division configuration for correction.

Another object of the present invention is to reflect interference of a light source in a three-dimensional image arranged in metaverse space.

It is a further object of the present invention to diversify the light source and to reflect the color interference according to the diversified light source.

In order to achieve the above object, the color optimization processing system of the three-dimensional modeling according to the present invention, a three-dimensional image generating module for generating a three-dimensional image from the object image; a detection module for detecting an outline and a surface surrounded by the outline from the 3D image; a region dividing module for generating a plurality of regions by dividing the surface; a color recognition module for recognizing a color of the divided region; and an optimization performing module including a group setting unit for setting a group including at least one of the regions, and a correction performing unit for collectively correcting the colors of the group.

Furthermore, the color recognition module includes: a color recognition unit for recognizing a Hue-Saturation-Brightness (HSB) value of the region; wherein the group setting unit sets a group including the at least one region according to a high and low level of the color error degree, and the optimization performing module is configured to calculate an average HSB value of the region included in the group. and a calculation unit, wherein the correction performing unit collectively corrects colors of regions included in the group with the average HSB value of each group.

In addition, the 3D image is a 3D object displayed in a metaverse space including a virtual light source, and the optimization performing module includes a correction color calculation unit that corrects the average HSB value according to the brightness of the light source, , the correction performing unit may include a function of collectively correcting colors of regions included in the group with the corrected average HSB value.

In addition, the corrected color calculating unit is characterized in that according to the brightness of the light source, the B (Brightness) value included in the average HSB value is corrected.

According to the color optimization processing system of the three-dimensional modeling of the present invention,

1) Create a three-dimensional image from an object image, but minimize the capacity while maintaining the quality of the three-dimensional image by simplifying the colors included in the three-dimensional image and optimizing the capacity increased due to various color configurations,

2) Grouping areas with similar colors to each other, and batch-correcting the color of the area based on the average color value of the group, achieves the purpose of capacity optimization while maximizing the quality of the 3D image by preventing the color from changing significantly. to be able to preserve

3) By making it possible to reflect the apparent color change that may occur in an environment where there is a light source in the color correction of the 3D image, it is possible to equalize the color by area and add the effect of the light source that can be additionally placed in the virtual space. It provides an effect that can increase the sense of reality.

delete

1 is a conceptual diagram showing a schematic configuration of a system of the present invention.
2 is a block diagram showing the configuration of the main server of the present invention.
3 is a process diagram illustrating an optimization processing system of the present invention;
4 is a conceptual diagram illustrating an example of region division.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. The accompanying drawings are not drawn to scale, and like reference numbers in each drawing refer to like elements.

1 is a conceptual diagram showing a schematic configuration of a system of the present invention.

Referring to FIG. 1 , the color optimization processing system for 3D modeling of the present invention may be configured to include a user terminal 2 and a main server 1 .

The user terminal 2 is a terminal possessed by a user who uploads an object image to perform 3D modeling, and the user terminal 2 basically includes at least one of a smartphone, a tablet PC, a desktop PC, and a notebook computer. may be doing The user terminal 2 may upload the object image stored in the terminal to the optimization processing system of the present invention, and receive the optimized three-dimensional image.

The main server 1 generates a 3D image from an object image in the optimization processing system of the present invention, and performs a role of lowering the capacity while maintaining 3D quality by automatically optimizing the color of the 3D image. In other words, in the present invention, the subject performing the 3D modeling optimization processing system can be called the main server 1, and unless otherwise specified, the main server 1 is assumed to be the same as the system. .

The main server 1 is a series of subjects for realizing the system of the present invention, and includes a server PC and a network communication network together. In addition, the main server 1 has a central processing unit (CPU) and a program that can be executed in the central processing unit on a hardware basis having storage means such as a memory and a hard disk, that is, software is installed and this software can be executed. A series of specific configurations of software will be described later as structural units such as 'modules', 'parts', and 'parts'.

Such 'modules' or 'parts' or 'interfaces' or 'parts' are software or hardware such as FPGAs or ASICs installed and stored in the storage means of the main server and executed via the CPU and memory. means

In this case, the configuration of 'module', 'unit', and 'interface' is not limited to hardware, and may be configured to be in an addressable storage medium or may be configured to reproduce one or more processors.

As an example, 'module' or 'part' or 'interface' refers to components such as software components, object-oriented software components, class components and task components, processes, functions, and properties. It includes fields, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays and variables.

Functions provided by these 'modules' or 'units' or 'interfaces' may be combined into a smaller number of components and 'units' or 'modules' or additional components and 'units' or 'modules' can be further separated.

In addition, the main server 1 means all kinds of hardware devices including at least one processor, and may be understood as encompassing software configurations operating in the corresponding hardware devices according to embodiments.

For example, a computing device as an example of a server may be understood as meaning including a smartphone, a tablet PC, a desktop, a notebook computer, and a user client and an application running in each device, and is not limited thereto.

Hereinafter, an optimization processing system for 3D modeling of the present invention will be described based on the configuration of the main server 1 as follows.

Figure 2 is a block diagram showing the configuration of the main server of the present invention, Figure 3 is a process diagram showing the optimization processing system of the present invention.

2 and 3 , the 3D modeling optimization processing system of the present invention is a 3D image generation module 100 , a detection module 200 , a region division module 300 , and a color recognition module 400 . , characterized in that it includes an optimization performing module 500 .

The 3D image generating module 100 performs a function of generating a 3D image from the object image input from the user terminal 2 . Here, the object image and the 3D image have nothing to do with any image or image, but basically, each frame of an image can be considered an image, so it is okay to judge that 3D data refers to an image unless otherwise specified. .

If the object image is input as a three-dimensional image, the object image may soon become a three-dimensional image. However, if the object image is a two-dimensional image, a plurality of object images photographed at various angles and distances are input for the object. A single 3D image may be generated by synthesizing a plurality of object images.

In this case, when generating a 3D image, it is to generate an intermediate image based on a plurality of object images taken from multiple angles of a specific object, and then perform assignment processing and rendering of polygons included in the intermediate image to generate a 3D image. It is possible.

In this case, a polygon is a polygon of the smallest unit used to express a three-dimensional shape, that is, an object on a three-dimensional image. It is used to express the curve by following a straight line, and it is used to indicate the edge of an object. In addition, rendering refers to a process of creating a three-dimensional image by injecting realism into a two-dimensional image in consideration of external information such as light source, position, and color. As described above, a technique of producing a three-dimensional image by performing polygon assignment processing and rendering according to the level value of the intermediate image synthesized from a plurality of object images is a well-known technique, and thus a more detailed description thereof will be omitted.

The detection module 200 serves to detect an outline and a surface surrounded by the outline from the 3D image. When the 3D image includes a 3D shape image of a specific object, the 3D object includes an outline and a surface surrounded by the outline. For example, when the 3D image includes an image of a cube, the cube corresponds to a total of 12 outlines and 6 faces.

Accordingly, it can be said that each surface of an object included in the 3D image is detected by detecting the outline of the generated 3D image and the surface surrounded by the outline.

More preferably, in order to detect an outline and a surface through the detection module 200 , the detection module 200 may include an outline detection unit 210 and a surface detection unit 220 .

The outline detection unit 210 detects an edge from the 3D image and performs a function of acquiring an outline from the detected edge. Here, the edge indicates the boundary of a region within the image, and performs a function of obtaining a pixel corresponding to an outline by indicating a discontinuity of image brightness.

This may refer to the conventional edge detection, and in this case, it is also possible to detect an edge from a 3D image from which noise is removed by first performing noise removal before edge detection. In addition, when detecting an edge, a step discontinuity point and a line discontinuity point are detected, respectively, an edge is identified based on the detection, and an outline can be obtained based on the edge.

The surface detection unit 220 detects a portion surrounded by the outline detected by the outline detection unit 210 as a surface, and detects a closed curve closed by the outline or an area inside the closed line segment as a surface. perform the function

4 is a conceptual diagram illustrating an example of region division.

Referring to FIG. 4 , the region dividing module 300 performs a function of generating a plurality of regions by dividing each detected surface into a plurality of pieces. In this case, since there is no limit to the size and number of divided regions, the size and number of divided regions may vary according to settings by the system administrator, and thus the size of the regions may be the same or different.

As in the above description, when the three-dimensional image is an image of a cube, a total of six faces are detected. At this time, each face is divided into nine and each face is divided into nine regions each having the same area. do. Therefore, a cube with 6 faces can be divided into 54 regions in total.

Alternatively, when various colors are distributed on the surface, it is possible to set the area using the color as a boundary. As shown in FIG. 4 , when several colors are mixed and distributed on a specific surface, only parts having the same or similar colors may be divided into the same area.

The color recognition module 400 performs a function of recognizing the color of each divided area. In this case, in the color recognition method, color recognition may be performed through any one of the conventional RGB, HSB, and CMYK methods. In this case, if each divided area has the same color, only the color of a specific coordinate among the divided areas can be recognized. However, if the colors of each divided area are all different, it is also possible to determine the average color value for each area.

Taking the Hue-Saturation-Brightness (HSB) value as an example, the HSB value is determined for each area, and if the HSB value is different for each pixel even within the same area, the average HSB value of the corresponding area can be identified.

The optimization performing module 500 sets a group including a plurality of regions and identifies a function of collectively correcting the colors of the group, and includes a group setting unit 510 and a correction performing unit 520 for this purpose. .

The group setting unit 510 sets a group including at least one of the regions, and in this case, regions having similar colors are preferably set as one group. At this time, the standard of color similarity is not limited, but each value (R, G, B) error within 5% in RGB, within 5% error of each value (H, S, B) in HSB standard, each value based on CMYK (C ,M,Y,K) After selecting a similarity range such as within 5% of an error, it can be said that the regions within the range are set as a group.

Accordingly, regions having similar colors are grouped into one group, preferably a color group is set, and the basis for performing optimization is laid through the set color group.

The correction performing unit 520 performs a function of collectively correcting group colors. In this case, collectively correcting the colors is basically to equalize the colors of the regions included in the group so that all colors of the corresponding group are the same without color difference. At this time, there is no restriction on the equalization method, but preferably, the colors of all areas in the group are collectively corrected with the color of a specific area included in the group, or the average RGB value, average HSB value, and average CMYK value of the area included in the group. In any one of the methods, the color of the regions included in the group can be collectively corrected.

According to the 3D modeling optimization processing system of the present invention, a 3D image is generated from an object image, but the color included in the 3D image is simplified to optimize the capacity increased due to various color configurations. It has the advantage of being able to minimize the capacity while maintaining the quality.

Here, more preferably, a configuration for group setting and color optimization will be described. The color recognition module 400 of the present invention includes a color recognition unit 410 and an error degree determination unit 420 to determine the color similarity between regions. can figure out

The color recognition unit 410 performs a function of recognizing a Hue-Saturation-Brightness (HSB) value of each area. In this case, the HSB value is also referred to as a hue, saturation, and brightness model, and the H, S, and B values have values of 0 to 100%, respectively. Therefore, the HSB value of each area is identified through a conventional design program or color recognition tool. If the color is non-uniform within the area, the H, S, and B values of the area are averaged, respectively, and the HSB value of the area is identified. .

The error degree determination unit 420 calculates a color error degree by comparing the HSB values of the specific area and the surrounding area based on the specific area. In this case, the color error degree is to determine how much the difference between the corresponding area and the surrounding area is, and in this case, preferably, only the absolute value of the error is identified.

If the HSB value of area A is H 48, S 35, B 52, and the HSB value of area B is H 47, S 40, B 40, H is 1 difference, S is 5 difference, B differs by 12. That is, there is a difference of 1% for H, 5% for S, and 12% for B. That is, if the HSB is averaged, it can be determined that there is a color error of 6%.

In other words, after obtaining the absolute value of the difference between H, S, and B, the average value can be calculated as a color error degree. can be a criterion.

When the color error degree is identified in this way, the group setting unit 510 sets a group including at least one region according to the calculated height of the color error degree. In this case, preferably, as in the above-described embodiment, after a standard is set such that the color error is within 5%, regions meeting the standard are grouped.

Accordingly, regions having a color error degree within a reference range with respect to one region are grouped into one group, and in this case, additional grouping may be performed using another region among the remaining ungrouped regions as a reference again.

At this time, as an embodiment, there are nine regions A, B, C, D, E, F, G, H, and I, and the grouping standard is within a color error range of 5%.

If the area with a color error within 5% of area A is C, E, G, A, C, E, G is a group, and the remaining B, D, F, H, I Wait for grouping.

Next, when re-extracting an area with a color error within 5% based on area B, if there is no extracted area, set a group for area B alone.

Then, based on group D, a region with a color error degree within a range of 5% is extracted again, and when the extracted area is F, one group is set for D and F.

Finally, when re-extracting an area with a color error within 5% based on group H, if there is no extracted area, H and I are set to different groups.

When grouping is repeatedly performed using a specific region as a reference in this way, at least one region is included in each group, and at this time, the color error degree of the regions included in each group is calculated to be within 5%.

In this case, since there is no restriction on the method of setting the criteria for setting the group or on which region to use as the reference, any one of the randomly selected regions may be the criterion for setting the group.

When the group is set in this way, the average color calculating unit 530 included in the optimization performing module 500 may calculate the average HSB value of the region included in the group. In this case, the average HSB value is the average H value, S value, and B value of the regions included in the group. If, as in the above embodiment, regions D and F are a group, the HSB values of regions D are H(45), S(20), and B(5), and the HSB values of regions F are H(42), For S(15), B(10), the average HSB values are H(44), S(18), B(8). At this time, since each H, S, and B value preferably recognizes an integer range, when a decimal value is calculated, it is preferable to calculate an integer value by rounding to the first decimal place.

When the average HSB value of each group is calculated through the above-described average color calculating unit 530 , the correction performing unit 520 collectively corrects the colors of the regions included in the group with the calculated HSB value of each group.

Therefore, by grouping regions with similar colors to each other, and by batch-correcting the colors of the regions based on the average color value of the group, the purpose of capacity optimization is achieved while the color does not change significantly to preserve the quality of the 3D image as much as possible. can do.

In this case, most preferably, the 3D image may be a 3D object displayed in the metaverse space, where there may not be a separate light source in the metaverse space, but a virtual star or sun, a separate light or a street lamp, and a separate light or street lamp in the metaverse space The same virtual light source may be installed.

Accordingly, a virtual light source may be included in the metaverse space, and the 3D image may be a 3D object displayed in the metaverse space including the virtual light source.

At this time, in the real world, the brightness of objects located near the light source is slightly distorted and enters our eyes, but it is difficult to reflect even the color change due to the light source in the metaverse space.

However, in order to realize a more realistic metaverse space, it may be more preferable to reflect even the apparent color change of the object due to the light source. can do.

The corrected color calculator 540 may perform a function of correcting the average HSB value calculated according to the brightness of the light source. As the brightness of the light source increases, the apparent color is likely to be distorted. , to correct the B value. Preferably, the average HSB value may be corrected by adjusting the H value and the B value so that at least one of the H value and the B value increases as the brightness of the light source increases. At this time, there are no restrictions on the level of correction or whether to correct H or B.

However, most preferably, the B (Brightness) value included in the average HSB value is corrected according to the brightness of the light source, and if the B value reaches 100%, it is most preferable to additionally correct the H value.

In one embodiment, when the brightness of the virtual light source is 300 lx, the B value is increased by 10%, when the brightness of the virtual light source is 600 lx, the B value is increased by 15%, and when the brightness of the virtual light source is 1000 lx, the B value is 20% and the H value increase by 10%. If the B value reaches the maximum (100%), only the H value is raised.

Accordingly, when the average HSB value is corrected in this way, the correction performing unit collectively corrects the colors of the regions included in the specific group with the corrected average HSB value.

According to such a correction configuration according to the light source, the apparent color change that may appear in an environment where there is a light source can be reflected in the color correction of the three-dimensional image, so that the color is equalized for each area and additionally arranged in a virtual space It has the advantage of increasing the sense of reality by adding the effect of possible light sources.

At this time, it is preferable that the corrected color calculating unit 540 corrects the B (Brightness) value included in the average HSB value according to the brightness of the light source when correcting the average HSB value. In this case, there is no special limitation, but most preferably, the corrected B value can be calculated according to Equation 1 below.

Equation 1,

는 보정 처리된 B값,

는 그룹에 포함된 k번째 영역의 B(Brightness)값으로서

, n은 영역의 개수, B(s)는 광원의 밝기(lx), B(c)는 기준 밝기(lx)를 의미한다.)(here,

is the corrected B value,

is the B (Brightness) value of the k-th area included in the group.

, n is the number of regions, B(s) is the brightness of the light source (lx), and B(c) is the reference brightness (lx).

Here, the reference brightness is to remove a unit from the brightness of a light source using a unit of lux (lx), and may be set in units of lx. In this case, the range of the standard brightness can be set to around 100 lx, which is between the brightness of the underground parking lot (50 lx), which is the brightness enough to identify an object, and the brightness inside the subway (150 lx), but there is no limitation on the exact value of the standard brightness. , this can be set by the system administrator.

The brightness of the light source is the brightness of a virtual light source placed in the metaverse space, and the brightness of the light source may vary depending on the type of lighting and whether it is a star or a sun in a natural object.

For example, 5 regions are included in a specific group, and the B values of each region are 50, 48, 45, 52, 54. 300 lx of lighting is placed in the metaverse space, and the standard brightness is 100 lx.

At this time, the corrected B value is,

can be corrected with

That is, the B value of the average HSB value is converted to 54, and the color of the corresponding group can be collectively corrected according to the converted HSB value.

In Equation 1, a logarithm is taken of the value obtained by dividing the brightness of the light source by the reference brightness, the hypertangent is corrected, and this is reflected in the average of the B values.

This is because lx, which is used as the brightness of the light source, generally has a range of greatly fluctuating values, so take the logarithm and observe only the change in logarithmic units. However, it is multiplied by a weight to reflect the decrease in the change due to the difference through the logarithm.

Therefore, according to the configuration of the correction processing of the B value through Equation 1, it is possible to take a logarithm of the brightness difference that varies by more than 10 units and reflect it, and at the same time, it is possible to observe the nonlinear behavior through the hypertangent function. Furthermore, there is an effect of controlling excessive change in color by adjusting the limit of correction to around 10% through the characteristic of the hypertangent function converging to 1.

Accordingly, it is possible to reflect the apparent color change that may appear in the environment where there is a light source in the color correction of the 3D image, so that the color is equalized for each area and the effect of the light source that can be additionally placed in the virtual space is reduced. It can be added to increase the realism.
As described so far, the configuration and operation of the color optimization processing system for three-dimensional modeling according to the present invention are expressed in the above description and drawings, but this is merely an example and the spirit of the present invention is not limited to the above description and drawings. It goes without saying that various changes and modifications can be made without departing from the technical spirit of the present invention.

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

1: main server 2: user terminal
100: 3D image generation module 200: detection module
210: outline detection unit 220: surface detection unit
300: region division module 400: color recognition module
410: color recognition unit 420: error determination unit
500: optimization performing module 510: group setting unit
520: correction performing unit 530: average color calculating unit
540: correction color calculation unit 600: solar information identification module

Claims (8)

As a color optimization processing system of 3D modeling,
a three-dimensional image generating module for generating a three-dimensional image that is a three-dimensional object displayed in a metaverse space including a virtual light source from the object image;
a detection module for detecting an outline and a surface surrounded by the outline from the 3D image;
a region dividing module for generating a plurality of regions by dividing the surface;
a color recognition module comprising: a color recognition unit recognizing a hue-saturation-brightness (HSB) value of the divided area; and an error degree determination unit calculating a color error degree by comparing HSB values of the area and surrounding areas;
a group setting unit for setting a group including at least one region according to the high and low levels of the color error degree; and an average color calculation unit for calculating an average HSB value of regions included in the group; A correction color calculation unit for correcting the B (Brightness) value included in the average HSB value, and a correction performing unit for collectively correcting the colors of the regions included in the group with the average HSB value including the corrected B value. and an optimization performing module to
The corrected value of B is,
A color optimization processing system for three-dimensional modeling, characterized in that calculated according to the following Equation (1).
Equation 1,

(here,

is the corrected B value,

is the B (Brightness) value of the k-th area included in the group.

, n is the number of regions, B(s) is the brightness of the light source (lx), B(c) is the reference brightness (lx)) The method of claim 1,
The detection module is
an edge detection unit for detecting an edge from the three-dimensional image and obtaining an outline from the detected edge;
A color optimization processing system for three-dimensional modeling, characterized in that it comprises a surface detection unit for detecting a surface surrounded by the detected outline. delete delete delete delete delete delete

Images