CN117635830A - Metaverse scene construction method, system, server and storage medium - Google Patents

Abstract

The invention provides a method, system, server and storage medium for constructing a metaverse scene. The method includes: constructing a three-dimensional space model of the metaverse scene corresponding to the real scene based on multiple multi-angle real scene pictures uploaded by the user; receiving the The uploaded original sound file and the recorded file, the recorded file is a file recorded when the user plays the original sound file in the real scene; perform sound rendering processing on the original sound file in the three-dimensional space model; The recorded file and the rendered audio file are subjected to spectrum analysis and comparison, and based on the comparison result, it is judged whether the constructed three-dimensional space model is accurate. In this invention, the dependence on the UI designer is greatly reduced under the premise of restoring reality, and after creating the three-dimensional space model of the metaverse scene, the sound space verification of the three-dimensional space model is performed based on the audio file, ensuring that the constructed three-dimensional space model is accuracy.

Description

Metaverse scene construction method, system, server and storage medium

Technical field

Embodiments of the present invention relate to the technical field of three-dimensional modeling, and in particular to a method, system, server and storage medium for constructing a metaverse scene.

Background technique

In the existing technology, when creating metaverse scenes, the interface design (UI) designer is mainly relied on to provide materials and draw them, and the scenes are mostly constructed from imagination and lack correspondence with reality.

Contents of the invention

Embodiments of the present invention provide a method, system, server and storage medium for constructing metaverse scenes, which are used to solve the problem that existing metaverse scenes are mostly constructed based on imagination and lack correspondence with reality.

In order to solve the above technical problems, the present invention is implemented as follows:

In the first aspect, embodiments of the present invention provide a method for constructing a metaverse scene, including:

Construct a three-dimensional space model of the metaverse scene corresponding to the real scene scene based on multiple multi-angle real scene pictures uploaded by the user;

Receive original sound files and recorded files uploaded by users, where the recorded files are files recorded when the user plays the original sound files in the real-life scene;

Perform sound rendering processing on the original sound file within the three-dimensional space model to obtain a rendered audio file;

Perform spectrum analysis and comparison on the recorded file and the rendered audio file, and determine whether the constructed three-dimensional space model is accurate based on the comparison results.

Optionally, the real scene pictures include at least one of the following: pictures taken by the user, pictures uploaded by the user from the local photo album, pictures selected by the user from pictures matching the real scene in the picture library, the picture library The pictures in include at least one of the following: pictures obtained through web crawlers, pictures uploaded by users.

Optionally, the method also includes:

Build a picture library, where the picture library includes:

Carry out a rough classification on target pictures obtained through web crawlers and/or uploaded by users. The rough classification includes: labeling some of the target pictures with scene categories; using a neural network model to extract targets with labeled scene categories. digital features of the picture and the target picture to be classified, and calculate the distance between the target picture to be classified and the target picture of each category of marked scene categories, and determine the scene to which the target picture to be classified belongs based on the distance categories;

Subdividing the roughly classified target pictures. The subdivided classification includes: performing object recognition on the first target picture and the second target picture belonging to the target scene category, obtaining an object list, and extracting the first target picture and the second target picture. regular-shaped objects among the objects in the second target picture; if the number of objects in the intersection of the object lists of the first target picture and the second target picture is equal to the number of objects in the first target picture and the If the ratio of the union of the object lists of the second target picture is greater than or equal to the first preset ratio, it is determined that the first target picture and the second target picture belong to the same scene subcategory; if the intersection is empty, it is determined that the The first target picture and the second target picture do not belong to the same scene subcategory; if the ratio of the intersection and the union is less than the first preset ratio, the regular shape in the intersection is The objects are compared for comparison. If the similarity of the regular-shaped objects in the intersection is greater than or equal to the second preset ratio, it is determined that the first target picture and the second target picture do not belong to the same scene. kind.

Optionally, the method also includes:

Obtain the relevant text in the web page where the image obtained through the web crawler is located;

The text is analyzed, and the scene classification of the pictures obtained through the web crawler is determined according to the analysis results.

Optionally, building a three-dimensional space model of the metaverse scene corresponding to the real scene based on multiple multi-angle real scene pictures uploaded by the user includes:

Perform object recognition on the multiple multi-angle real scene pictures to obtain an object list for each of the real scene pictures;

Merge the object lists of real scene pictures under the same scene classification to obtain a collection of object lists;

Determine the size data of objects in each of the real scene pictures under the same scene classification;

Determine the target size data of each object in the object list set according to all size data of each object in the object list set in different real scene pictures;

According to the target size data of each object in the multiple multi-angle real scene pictures, a three-dimensional space model of the metaverse scene corresponding to the real scene is constructed.

Optionally, based on the target size data of each object in the multiple multi-angle real scene pictures, construct a three-dimensional space model of the metaverse scene corresponding to the real scene, including:

Obtain the outer contour and inner contour of the object in the real scene picture;

Obtain depth information of objects in the real scene picture;

According to the target size data, outer contour, inner contour and depth information of the object in the real scene picture, a three-dimensional space model of the metaverse scene corresponding to the real scene scene is constructed.

Optionally, the original sound file is subjected to sound rendering processing in the three-dimensional space model to obtain a rendered audio file, including:

Determine the spatial coordinates of the sound source corresponding to the original sound file and the listener corresponding to the recorded file in the three-dimensional space model. The spatial coordinates of the sound source are related to the position of the user when the original sound file is played in the real scene. Correspondingly, the spatial coordinates of the listener correspond to the user's position when recording the recording file in the real scene;

According to the spatial coordinates of the sound source corresponding to the original sound file and the listener corresponding to the recording file in the three-dimensional space model, the original sound file is subjected to sound rendering processing in the three-dimensional space model to obtain the rendered audio document.

In the second aspect, embodiments of the present invention provide a metaverse scene construction system, which is characterized by including:

The first building module is used to construct a three-dimensional space model of the metaverse scene corresponding to the real-life scene based on multiple multi-angle real-life scene pictures uploaded by the user;

A receiving module, configured to receive original sound files and recorded files uploaded by users. The recorded files are files recorded when the user plays the original sound file in the real-life scene;

A rendering module, used to perform sound rendering processing on the original sound file in the three-dimensional space model to obtain a rendered audio file;

A verification module, configured to perform spectrum analysis and comparison on the recorded file and the rendered audio file, and determine whether the constructed three-dimensional space model is accurate based on the comparison results.

In a third aspect, an embodiment of the present invention provides a server, which is characterized in that it includes: a processor, a memory, and a program stored on the memory and executable on the processor, and the program is processed by the When the processor is executed, the steps of the method for constructing a metaverse scene as described in the first aspect are implemented.

In a fourth aspect, embodiments of the present invention provide a computer-readable storage medium. A computer program is stored on the computer-readable storage medium. When the computer program is executed by a processor, the elements described in the first aspect are implemented. Steps in the construction method of universe scene.

In the embodiment of the present invention, multiple multi-angle real scene pictures uploaded by users are used to construct a three-dimensional space model of the metaverse scene corresponding to the real scene scene, which greatly reduces the dependence on UI designers on the premise of restoring reality, and After creating the three-dimensional space model of the metaverse scene, the sound space verification of the three-dimensional space model is performed based on the audio file to ensure the accuracy of the constructed three-dimensional space model.

Description of drawings

Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are for the purpose of illustrating preferred embodiments only and are not to be construed as limiting the invention. Also throughout the drawings, the same reference characters are used to designate the same components. In the attached picture:

Figure 1 is a schematic flow chart of a metaverse construction method according to an embodiment of the present invention;

Figure 2 is a schematic diagram of a user interface for users to upload real scene pictures according to an embodiment of the present invention;

Figure 3 is a schematic diagram of a user interface for saving a constructed three-dimensional space model according to an embodiment of the present invention;

Figures 4 and 5 are schematic diagrams of the principles of the four-dimensional light field function according to the embodiment of the present invention;

Figure 6 is a schematic diagram of a user interface for automatically fine-tuning a three-dimensional space model through audio data uploaded by users according to an embodiment of the present invention;

Figure 7 is a schematic structural diagram of a metaverse construction system according to an embodiment of the present invention;

Figure 8 is a schematic structural diagram of a server according to an embodiment of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without making creative efforts fall within the scope of protection of the present invention.

Please refer to Figure 1. An embodiment of the present invention provides a method for constructing a metaverse scene, including:

Step 11: Construct a three-dimensional space model of the metaverse scene corresponding to the real-life scene based on multiple multi-angle real-life scene pictures uploaded by the user;

In the embodiment of the present invention, the plurality of multi-angle real scene pictures are pictures in the same real scene space.

In the embodiment of the present invention, the angle may include at least one of the following: the entire surface, the left side, the right side, etc.

Step 12: Receive the original sound file and the recorded file uploaded by the user. The recorded file is the file recorded when the user plays the original sound file in the real-life scene;

Step 13: Perform sound rendering processing on the original sound file in the three-dimensional space model to obtain a rendered audio file;

Step 14: Perform spectrum analysis and comparison on the recorded file and the rendered audio file, and determine whether the constructed three-dimensional space model is accurate based on the comparison results.

In the embodiment of the present invention, multiple multi-angle real scene pictures uploaded by users are used to construct a three-dimensional space model of the metaverse scene corresponding to the real scene scene, which greatly reduces the dependence on UI designers on the premise of restoring reality, and After creating the three-dimensional space model of the metaverse scene, the sound space verification of the three-dimensional space model is performed based on the audio file to ensure the accuracy of the constructed three-dimensional space model.

In this embodiment of the present invention, as shown in Figure 2, a user interface can be provided for users to upload real scene pictures. Users can perform the following operations in this user interface:

1. The user can enter the scene name in the text input box, such as a certain palace or a certain gate, or other customized scene names, such as my ideal castle, etc. Or, more specifically, the study room of a certain palace, etc.

2. Users upload real scene pictures.

In the embodiment of the present invention, the real scene pictures include at least one of the following: pictures taken by the user, pictures uploaded by the user from the local photo album, and pictures selected by the user from the pictures in the picture library that match the real scene.

That is, users can upload real scene pictures by taking photos or browsing local photo albums. If the user checks to use the picture library, the system will search the picture library for pictures matching the scene name according to the scene name entered by the user in the first step. Optionally, each picture in the picture library has a scene classification label. The scene classification label is used to indicate the real scene to which the image belongs. If a picture matching the scene name input by the user is found in the picture library, the picture can be displayed in the browsing area on the right side of the user interface shown in Figure 2 for the user to select and use. Users can select pictures in the browsing area by clicking or dragging. In the browsing area, users can also browse more pictures by sliding and other methods. If the user's scene name is a customized scene name such as "My Ideal Castle", there may be no matching photos in the gallery, and a prompt can be given in the browsing area, for example, "No matching picture found."

In this embodiment of the present invention, the angle of the real scene may include at least one of the following: the entire surface, the left side, the right side, etc. For the sake of modeling accuracy, in the embodiment of the present invention, the number of pictures from each angle can be limited, for example, there are at least two pictures from each angle. In addition to photos from specified angles, users can also upload photos from other angles.

3. The user clicks the Confirm Submit button to upload the real scene picture to the server.

Optionally, before uploading to the server, a simple data verification can also be performed. For example, the data verification includes at least one of the following: whether the number of pictures meets the standard, whether the scene name is entered, etc.

The following describes the method for constructing a picture library according to the embodiment of the present invention.

In this embodiment of the present invention, the pictures in the picture library include at least one of the following: pictures obtained through web crawlers, and pictures uploaded by users.

When obtaining pictures through a web crawler, you can first set the scene, crawl the web pictures according to the scene, download the obtained pictures to a local specific resource folder, and set the scene classification tags of these pictures.

In the embodiment of the present invention, please refer to Figure 3. On the user interface for saving the constructed three-dimensional space model, the user can be prompted whether to authorize the sharing of pictures to the picture library. If the user does not click to authorize use, when the user clicks the completion button, the pictures uploaded by the user in the user interface shown in Figure 2 will be regarded as temporary data and deleted in the background; if the user clicks to authorize use, the pictures will be uploaded to the server , and after subsequent processing (such as setting scene classification labels), add it to your own picture library.

In the embodiment of the present invention, for pictures obtained through web crawlers, it is also necessary to perform scene classification on these pictures and set scene classification tags. Of course, for the sake of accuracy, scene classification can also be performed on pictures uploaded by users and scenes can be set. Classification tags.

In this embodiment of the present invention, building a picture library includes:

1. Roughly classify the target images obtained through web crawlers and/or uploaded by users. The rough classification includes:

1.1 Mark scene categories for some of the target images in the target images; optionally, when manually annotating scene categories for some of the target images, it is necessary to ensure that the number of images corresponding to each scene category is no less than the preset value. , for example, there are no less than 5 pictures, and the scene categories can be, for example, five scene categories: external environment of XX Palace, building of XX Palace, internal environment of XX Palace, XX Palace other, and other wrong pictures.

1.2 Use the neural network model to extract the digital features of the target images that have been labeled with the scene categories and the target images to be classified, and calculate the distance between the target images to be classified and the target images that have been labeled with the scene categories of each category. According to the The distance determines the scene category to which the target picture to be classified belongs;

In the embodiment of the present invention, the distance may be a Euclidean distance, and the calculation formula of the Euclidean distance d(x, μ) is as follows:

Among them, x is the digital feature of the target image to be classified, u _i is the mean digital feature of the i-th category of images manually labeled (because there is not only one image under each category, the average is taken), n is the manually labeled scene The number of categories.

That is, calculate which of the above-mentioned manually labeled scene categories the digital features of the target image to be classified have the closest mean distance to, and then determine which scene category the target image to be classified belongs to.

Taking the above five major scene categories as an example, after the classification is completed, the image resources under the "Other Error Pictures" category can be deleted, and the remaining four categories of images will enter the subsequent processing process.

2. Carry out detailed classification on the target images after rough classification.

The subdivisions include:

2.1 Perform object recognition on the first target picture and the second target picture belonging to the target scene category, obtain an object list, and extract regular-shaped objects among the objects in the first target picture and the second target picture;

Optionally, the regular shape includes at least one of the following: rectangle, circle, triangle.

2.2 If the ratio of the number of objects in the intersection of the object lists of the first target picture and the second target picture to the union of the object lists of the first target picture and the second target picture is greater than or equal to A first preset ratio (such as 40%) determines that the first target picture and the second target picture belong to the same scene subcategory;

If the intersection is empty, it is determined that the first target picture and the second target picture do not belong to the same scene subcategory;

If the ratio of the intersection and the union is less than the first preset ratio, compare the regular-shaped objects in the intersection, and if the regular-shaped objects in the intersection If the similarity is greater than or equal to the second preset ratio (such as 60%), it is determined that the first target picture and the second target picture do not belong to the same scene subcategory.

Taking the scene category of the internal environment of a certain palace as an example, the scene subcategories obtained after subdivision can include: dining room, study room, bedroom, kitchen, bathroom, etc.

In the embodiment of the present invention, optionally, the creation method of the metaverse scene also includes: obtaining relevant text in the web page where the picture obtained through a web crawler is located; analyzing the text (such as word segmentation processing), and based on the analysis The result determines the scene classification of the pictures obtained through the web crawler. This reverse tracing method is suitable for the above-mentioned situation where the scene classification of the picture cannot be determined through subdivision. For example, assuming that the image ImgA is crawled from the URL h1, the text in the text content with tags such as <p>/<div> before and after the <image> (ImgA) tag can be extracted and recorded as ImgA_h1_T0. Perform word segmentation processing on the text to obtain the nouns in it. Determine whether the scene classification appears in the noun list. If it exists, find the label of the picture by analyzing the grammatical structure of the sentence. If it does not exist, the noun list is directly analyzed and the scene classification label of the picture is given based on the context. If the scene classification labels of images ImgA and ImgB are the same, they can be correctly judged as one category even if there are no overlapping item lists.

In the embodiment of the present invention, optionally, building a three-dimensional space model of the metaverse scene corresponding to the real-life scene based on multiple multi-angle real-life scene pictures uploaded by the user includes:

Step 111: Perform object recognition on the multiple multi-angle real scene pictures to obtain an object list for each of the real scene pictures;

Step 112: Merge the object lists of real scene pictures under the same scene classification to obtain a set of object lists;

That is, pictures belonging to the same scene category are viewed as a whole. Each picture is a different angle of the whole, providing different angle details and data for drawing the whole.

The object list collection of a certain scene classification can be expressed as:

Among them, i represents the i-th picture in the scene category (such as the "exterior wall" scene subcategory under the broad category of XX palace architectural scene, or the "restaurant" scene subcategory under the major category of interior environment scenes), n The total number of pictures in this scene category.

Step 113: Determine the size data of the objects in each real scene picture under the same scene classification; the size data may include length, width, and height.

Assume that the object list set ObjList={a, b, c, d, e, f}, and the picture ImgA has only three objects {b, e, f}, then only the three objects {b, e, The dimensions of f} (length, width, height) are enough.

Step 114: Determine the target size data of each object in the object list set based on all size data of each object in the object list set in different real scene pictures;

In the embodiment of the present invention, the adjacency list in the data structure may be used to store the size data of the object.

For example, for object a in the object list set of a certain scene classification, after step 113, a ka (1<=ka<=n) group of size data can be obtained (n is the total number of pictures under the scene classification); similarly, For object b in the object list set of this scene classification, kb (1<=kb<=n) group size data can be obtained, and ka has no quantitative relationship with kb.

In the embodiment of the present invention, determining the target size data of each object in the object list set may be: normalizing all the size data of the target objects in the object list set, and normalizing the size data Find the mode and average. When the proportion of the mode value exceeds the preset threshold, the mode is directly used as the normalized size data of the object; otherwise, the average of the mode and the average is used as the normalized size data of the object. The final size data.

For example, the ka group size data of object a are normalized respectively (for example, the width is unified to 100) to complete the unification of the scale. Because although the size of the same object in different pictures may be different, the length, width, and height ratio of the object itself is similar. Calculate the mode and average of the length (or height or width) in the normalized ka group size data. When the number of mode values exceeds 50%, the mode is directly used as the normalized size of the object. Length (or height or width); otherwise use the average of the mode and mean as the normalized length (or height or width) of the object.

Step 115: Construct a three-dimensional space model of the metaverse scene corresponding to the real scene scene based on the target size data of each object in the multiple multi-angle real scene scene pictures.

Because the real scene is usually a famous stadium and other buildings in reality, some real data of the real scene can be obtained. By comparing the partial real data with the target size data (estimated size data) of the object obtained above, the estimate can be made. The size data is corrected to obtain the true size data of the object. Assume that the estimated size data of object a obtained above is (La,100,Ha), and the estimated size data of object b is (Lb,100,Hb). When only the real size data of object a exists, the estimated size data of object b Real size data can also be obtained by mapping.

In the embodiment of the present invention, optionally, based on the target size data of each object in the multiple multi-angle real scene pictures, a three-dimensional space model of the metaverse scene corresponding to the real scene scene is constructed, including:

Step 116: Obtain the outer contour and inner contour of the object in the real scene picture;

Obtaining the outer contour of an object means: tracing the same object (there are multiple pictures containing this object) under the same scene category (the subdivision obtained previously, that is, the scene subcategory). Optionally, multiply the recognition frame of the object recognition by a preset multiple (such as 1.3 times) before cropping (the multiple can be adjusted according to the actual situation, but must be greater than 1 to retain it to cover a sufficient area), and obtain the object's The main image (the image contains almost no other objects) is then segmented through the neural network ReSeg model to obtain the outer contour of the object on the two-dimensional plane.

The method of obtaining the inner contour of an object can be as follows: with the help of the idea of active contour model and edge detection algorithm, first define a curve, and then obtain the energy function based on the picture data, trigger the curve change by minimizing the energy function, and use the pixels in adjacent areas Due to the discontinuous nature of the value, the first-order or second-order derivative is used to detect the edge point, so that it gradually approaches the target edge, and finally finds the target edge, and then obtains the inner contour of the object on the two-dimensional plane.

Step 117: Obtain the depth information of the objects in the real scene picture;

Alternatively, a four-dimensional light field function can be used to obtain a two-dimensional sliced epipolar plane image (EPI), which includes a spatial size dimension and an angle dimension. An object point in the scene is displayed as an inclined straight line in the EPI , the slope of the straight line is proportional to the distance from the object point to the camera. Therefore, the depth information of an object can be obtained through the slope of the corresponding object point.

Please refer to Figure 4 and Figure 5. The principle of the four-dimensional light field function is as follows: a ray passes through two points on two parallel planes, the coordinates are (u, v) and (s, t) respectively, and the distance between the two parallel planes The distance is F, and the two-dimensional angle information of the light can be obtained from the position coordinates of the two points, so it can be represented by the four-dimensional light field function LF (s, t, u, v). The relationship between the light field function LF(s,t,u,v) defined by the lens surface and the sensor surface and the light field function LF′(s,t,u,v) defined by the lens surface and the refocusing surface can be Obtained through geometric transformation, the propagation of the four-dimensional light field function in space is equivalent to a shear transformation. According to the four-dimensional light field function, the image radiance from the u, v plane to a certain point on the s, t plane can be expressed by the integral formula of the light radiance, so that a two-dimensional slice expression can be obtained.

Step 118: Construct a three-dimensional space model of the metaverse scene corresponding to the real scene scene based on the target size data, outer contour, inner contour and depth information of the object in the real scene scene picture.

In the embodiment of the present invention, optionally, the original sound file is subjected to sound rendering processing in the three-dimensional space model to obtain a rendered audio file, including:

Determine the spatial coordinates of the sound source corresponding to the original sound file and the listener corresponding to the recorded file in the three-dimensional space model. The spatial coordinates of the sound source are related to the position of the user when the original sound file is played in the real scene. Correspondingly, the spatial coordinates of the listener correspond to the user's position when recording the recording file in the real scene;

According to the spatial coordinates of the sound source corresponding to the original sound file and the listener corresponding to the recording file in the three-dimensional space model, the original sound file is subjected to sound rendering processing in the three-dimensional space model to obtain the rendered audio document.

Please refer to Figure 6, which is a schematic diagram of a user interface for automatically fine-tuning a three-dimensional spatial model through audio data uploaded by the user. In this user interface, the user can position the sound source (the little sun in Figure 6) and the listener (the smiling face in Figure 6) into the three-dimensional space model by dragging the icons. During the dragging process, the (X, Y, Z) coordinates on the right side are updated in real time. In addition to dragging, the user can also output specific values in (X, Y, Z) on the right to determine the spatial coordinates of the sound source and the listener within the three-dimensional space model. The spatial coordinates of the sound source correspond to the position of the user when playing the original sound file in the real scene, and the spatial coordinates of the listener correspond to the position of the user when recording the recording file in the real scene. In addition, during the dragging process, it can also be judged whether the distance between two points set by the user exceeds the nearest distance threshold. If it does not exceed the value, the user will be prompted to reset it. Submit the recording file and original sound file by browsing and uploading, and click the Confirm Optimization button to verify the three-dimensional space model.

In the embodiment of the present invention, optionally, the user's position when playing the original sound file in the real scene scene and the user's position when recording the recorded file in the real scene scene are multiple groups.

Examples are given below. First, multiple sets of double-point positions in the scene are taken as the collection points of the recording file, including but not limited to the following sets of double-point positions: <boundary, opposite boundary>, <boundary, center point>. Secondly, play the original sound file at one of the above-mentioned two-point positions, place a microphone at the other point for recording, and obtain the recording file. In this embodiment of the present invention, the length of the recording file can be set as needed, for example, audio data within 30 seconds to 1 minute. Perform spectrum analysis and sampling on the collected recording files, and record them as data tables. In the constructed three-dimensional space model, determine the sound source corresponding to the original sound file and the spatial coordinates of the listener corresponding to the recorded file in the three-dimensional space model. The spatial coordinates of the sound source are consistent with the user playing the original sound file in the real scene. The spatial coordinates of the listener correspond to the position of the user when recording the recording file in the real scene. After the spatial coordinates are determined, sound rendering processing is performed on the original sound file. The data generated by rendering are also subjected to spectrum analysis and sampling, and recorded as data Test. Compare the similarity between the data Table and the data Test. If the two are very close, it is judged that the three-dimensional space model is successfully constructed and accurate; if there is a large difference between the two, it is judged that the three-dimensional space model is not constructed accurately. At this point, the accuracy of three-dimensional spatial model construction can be effectively guaranteed.

In the embodiment of the present invention, while verifying whether the three-dimensional space model constructed through audio data is accurate, the difference between the above-mentioned recorded file and the rendered audio file can also be analyzed, and the three-dimensional space model can be automatically adjusted based on the analysis results.

In the embodiment of the present invention, after the three-dimensional space model is constructed, the user can also manually adjust the constructed three-dimensional space model. For example, a user interface is provided, the three-dimensional space model is displayed on the user interface, and the user selects the three-dimensional space model to be adjusted. After selecting the item, you can increase or decrease the length by sliding the slider and other operations. In addition, parameters such as color and position can also be adjusted.

Referring to Figure 7, an embodiment of the present invention provides a metaverse scene construction system 70, which includes:

The first building module 71 is used to construct a three-dimensional space model of the metaverse scene corresponding to the real-life scene based on multiple multi-angle real-life scene pictures uploaded by the user;

The receiving module 72 is used to receive the original sound file and the recorded file uploaded by the user. The recorded file is the file recorded when the user plays the original sound file in the real-life scene;

The rendering module 73 is used to perform sound rendering processing on the original sound file in the three-dimensional space model to obtain a rendered audio file;

The verification module 74 is used to perform spectrum analysis and comparison on the recorded file and the rendered audio file, and determine whether the constructed three-dimensional space model is accurate based on the comparison results.

Optionally, the real scene pictures include at least one of the following: pictures taken by the user, pictures uploaded by the user from the local photo album, pictures selected by the user from pictures matching the real scene in the picture library, the picture library The pictures in include at least one of the following: pictures obtained through web crawlers, pictures uploaded by users.

Optionally, the build system 70 also includes:

The second building module is used to build a picture library, wherein said building a picture library includes:

Carry out a rough classification on target pictures obtained through web crawlers and/or uploaded by users. The rough classification includes: labeling some of the target pictures with scene categories; using a neural network model to extract targets with labeled scene categories. digital features of the picture and the target picture to be classified, and calculate the distance between the target picture to be classified and the target picture of each category of marked scene categories, and determine the scene to which the target picture to be classified belongs based on the distance categories;

Subdividing the roughly classified target pictures. The subdivided classification includes: performing object recognition on the first target picture and the second target picture belonging to the target scene category, obtaining an object list, and extracting the first target picture and the second target picture. regular-shaped objects among the objects in the second target picture; if the number of objects in the intersection of the object lists of the first target picture and the second target picture is equal to the number of objects in the first target picture and the If the ratio of the union of the object lists of the second target picture is greater than or equal to the first preset ratio, it is determined that the first target picture and the second target picture belong to the same scene subcategory; if the intersection is empty, it is determined that the The first target picture and the second target picture do not belong to the same scene subcategory; if the ratio of the intersection and the union is less than the first preset ratio, the regular shape in the intersection is The objects are compared for comparison. If the similarity of the regular-shaped objects in the intersection is greater than or equal to the second preset ratio, it is determined that the first target picture and the second target picture do not belong to the same scene. kind.

Optionally, the build system 70 also includes:

The acquisition module is used to obtain the relevant text in the web page where the image obtained through the web crawler is located;

A determination module, configured to analyze the text and determine the scene classification of the pictures obtained through the web crawler according to the analysis results.

Optionally, the first building module 71 is used to perform object recognition on the multiple multi-angle real scene pictures to obtain an object list for each of the real scene pictures; merge the real scene pictures under the same scene classification The object list is obtained to obtain an object list set; determine the size data of the objects in each of the real-scene scene pictures under the same scene classification; according to each object in the object list set, in different real-scene scene pictures All size data in the object list set is determined to determine the target size data of each object in the object list set; based on the target size data of each object in the multiple multi-angle real scene pictures, the element corresponding to the real scene scene is constructed. A three-dimensional space model of a cosmic scene.

Optionally, the first building module 71 is used to obtain the outer contour and inner contour of the object in the real scene picture; obtain the depth information of the object in the real scene picture; according to the real scene picture The target size data, outer contour, inner contour and depth information of the object are used to construct a three-dimensional space model of the metaverse scene corresponding to the real scene scene.

Optionally, the rendering module 73 is used to determine the spatial coordinates of the sound source corresponding to the original sound file and the listener corresponding to the recorded file in the three-dimensional space model. The spatial coordinates of the sound source are consistent with the location of the user. The real scene scene corresponds to the position when the original sound file is played, and the spatial coordinates of the listener correspond to the user's position when the real scene scene records the recording file; according to the sound source corresponding to the original sound file and the The listener's spatial coordinate corresponding to the recorded file is in the three-dimensional space model, and the original sound file is subjected to sound rendering processing in the three-dimensional space model to obtain a rendered audio file.

Please refer to Figure 8. This embodiment of the present invention also provides a server 80, which includes a processor 81, a memory 82, and a computer program stored on the memory 82 and executable on the processor 81. The computer program is processed by the processor 81. During execution, each process of the above embodiment of the method for constructing a metaverse scene is realized, and the same technical effect can be achieved. To avoid repetition, details will not be described here.

Embodiments of the present invention also provide a computer-readable storage medium. A computer program is stored on the computer-readable storage medium. When the computer program is executed by a processor, each process of the above embodiment of the method for constructing a metaverse scene is implemented, and can achieve the same technical effect, so to avoid repetition, we will not repeat them here. Wherein, the computer-readable storage medium is such as read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk, etc.

It should be noted that, in this document, the terms "comprising", "comprises" or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article or device that includes a series of elements not only includes those elements, It also includes other elements not expressly listed or inherent in the process, method, article or apparatus. Without further limitation, an element defined by the statement "comprises a..." does not exclude the presence of additional identical elements in a process, method, article or apparatus that includes that element.

Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus the necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is better. implementation. Based on this understanding, the technical solution of the present invention can be embodied in the form of a software product in essence or the part that contributes to the existing technology. The computer software product is stored in a storage medium (such as ROM/RAM, disk, CD), including several instructions to cause a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in various embodiments of the present invention.

The embodiments of the present invention have been described above in conjunction with the accompanying drawings. However, the present invention is not limited to the above-mentioned specific implementations. The above-mentioned specific implementations are only illustrative and not restrictive. Those of ordinary skill in the art will Under the inspiration of the present invention, many forms can be made without departing from the spirit of the present invention and the scope protected by the claims, all of which fall within the protection of the present invention.

Claims (10)

1. The construction method of the meta-universe scene is characterized by comprising the following steps of:

constructing a three-dimensional space model of a meta-universe scene corresponding to a live-action scene according to a plurality of multi-angle live-action scene pictures uploaded by a user;

receiving an original sound file and a recorded file uploaded by a user, wherein the recorded file is a file recorded by the user when the original sound file is played in the live-action scene;

performing sound rendering processing on the original sound file in the three-dimensional space model to obtain a rendered audio file;

and carrying out spectrum analysis and comparison on the recorded file and the rendered audio file, and judging whether the constructed three-dimensional space model is accurate or not according to the comparison result.

2. The method of claim 1, wherein the live-action scene picture comprises at least one of: the method comprises the steps of shooting pictures by a user, uploading pictures from a local album by the user, and selecting pictures from pictures matched with the live-action scene in a picture library by the user, wherein the pictures in the picture library comprise at least one of the following: and the pictures are acquired by the web crawlers, and uploaded by the users.

3. The method as recited in claim 2, further comprising:

constructing a picture library, wherein the constructing the picture library comprises:

performing rough classification on target pictures acquired by a web crawler and/or uploaded by a user, wherein the rough classification comprises the following steps: labeling a scene major class for part of the target pictures in the target pictures; extracting the digital characteristics of the target pictures of the marked scene major categories and the target pictures to be classified by using a neural network model, calculating the distance between the target pictures to be classified and the target pictures of each type of marked scene major categories, and determining the scene major categories to which the target pictures to be classified belong according to the distance;

performing fine classification on the roughly classified target pictures, wherein the fine classification comprises the following steps: object recognition is carried out on a first target picture and a second target picture belonging to a target scene main class, an object list is obtained, and regularly-shaped objects in the first target picture and the second target picture are extracted; if the ratio of the number of objects in the intersection of the object lists of the first target picture and the second target picture to the union of the object lists of the first target picture and the second target picture is greater than or equal to a first preset proportion, determining that the first target picture and the second target picture belong to the same scene subclass; if the intersection is empty, determining that the first target picture and the second target picture do not belong to the same scene subclass; and if the ratio of the intersection set to the union set is smaller than the first preset proportion, comparing the degrees of comparison of the regularly-shaped objects in the intersection set, and if the similarity of the regularly-shaped objects in the intersection set is larger than or equal to a second preset proportion, determining that the first target picture and the second target picture do not belong to the same scene subclass.

4. A method according to claim 3, further comprising:

acquiring a related text in a webpage where a picture acquired by a web crawler is located;

and analyzing the text, and determining scene classification of the pictures acquired by the web crawlers according to an analysis result.

5. The method of claim 1, wherein constructing a three-dimensional spatial model of a metauniverse scene corresponding to the live-action scene from a plurality of multi-angle live-action scene pictures uploaded by a user comprises:

performing object recognition on the multiple multi-angle live-action scene pictures to obtain an object list of each live-action scene picture;

merging object lists of the live-action scene pictures under the same scene classification to obtain an object list set;

determining size data of objects in each of the live-action scene pictures under the same scene classification;

determining target size data of each object in the object list set according to all size data of each object in the object list set in different live-action scene pictures;

and constructing a three-dimensional space model of the meta-universe scene corresponding to the real scene according to the target size data of each object in the multi-angle real scene pictures.

6. The method of claim 5, wherein constructing a three-dimensional spatial model of a metauniverse scene corresponding to the live-action scene from target size data of each object in the plurality of multi-angle live-action scene pictures comprises:

acquiring the outer outline and the inner outline of an object in the live-action scene picture;

acquiring depth information of an object in the live-action scene picture;

and constructing a three-dimensional space model of the metauniverse scene corresponding to the live-action scene according to the target size data, the outer contour, the inner contour and the depth information of the object in the live-action scene picture.

7. The method of claim 1, wherein performing a sound rendering process on the acoustic file in the three-dimensional space model to obtain a rendered audio file, comprising:

determining the space coordinates of a sound source corresponding to the original sound file and a listener corresponding to the recorded file in the three-dimensional space model, wherein the space coordinates of the sound source correspond to the position of a user when the original sound file is played in the live-action scene, and the space coordinates of the listener correspond to the position of the user when the recorded file is recorded in the live-action scene;

and carrying out sound rendering processing on the original sound file in the three-dimensional space model according to the space coordinates of the sound source corresponding to the original sound file and the listener corresponding to the recorded file in the three-dimensional space model to obtain a rendered audio file.

8. A system for building a meta-cosmic scene, comprising:

the first construction module is used for constructing a three-dimensional space model of a meta-universe scene corresponding to the live-action scene according to a plurality of multi-angle live-action scene pictures uploaded by a user;

the receiving module is used for receiving an original sound file and a recorded file uploaded by a user, wherein the recorded file is a file recorded by the user when the original sound file is played in the live-action scene;

the rendering module is used for performing sound rendering processing on the original sound file in the three-dimensional space model to obtain a rendered audio file;

and the verification module is used for carrying out spectrum analysis and comparison on the recorded file and the rendered audio file, and judging whether the constructed three-dimensional space model is accurate or not according to the comparison result.

9. A server, comprising: a processor, a memory and a program stored on the memory and executable on the processor, which when executed by the processor, implements the steps of the method of constructing a metauniverse scene as claimed in any one of claims 1 to 7.

10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, implements the steps of the method of constructing a metauniverse scene according to any one of claims 1 to 7.