CN116974416A - Data processing method, device, equipment and readable storage medium - Google Patents
Data processing method, device, equipment and readable storage medium Download PDFInfo
- Publication number
- CN116974416A CN116974416A CN202310203020.8A CN202310203020A CN116974416A CN 116974416 A CN116974416 A CN 116974416A CN 202310203020 A CN202310203020 A CN 202310203020A CN 116974416 A CN116974416 A CN 116974416A
- Authority
- CN
- China
- Prior art keywords
- cultural relic
- virtual
- resource
- shooting
- display
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title abstract description 18
- 238000000034 method Methods 0.000 claims abstract description 80
- 230000000007 visual effect Effects 0.000 claims description 46
- 230000008569 process Effects 0.000 claims description 34
- 238000009877 rendering Methods 0.000 claims description 34
- 238000012545 processing Methods 0.000 claims description 32
- 238000005259 measurement Methods 0.000 claims description 26
- 239000000463 material Substances 0.000 claims description 23
- 230000006870 function Effects 0.000 claims description 21
- 238000004590 computer program Methods 0.000 claims description 20
- 230000008859 change Effects 0.000 claims description 17
- 238000004891 communication Methods 0.000 claims description 11
- 238000013507 mapping Methods 0.000 claims description 9
- 230000002452 interceptive effect Effects 0.000 claims description 5
- 238000007654 immersion Methods 0.000 abstract description 8
- 238000001514 detection method Methods 0.000 description 17
- 230000003993 interaction Effects 0.000 description 17
- 238000010586 diagram Methods 0.000 description 16
- 238000004422 calculation algorithm Methods 0.000 description 9
- 230000007613 environmental effect Effects 0.000 description 6
- 230000033001 locomotion Effects 0.000 description 6
- 238000004458 analytical method Methods 0.000 description 4
- 238000013459 approach Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 238000005286 illumination Methods 0.000 description 4
- 230000008439 repair process Effects 0.000 description 4
- 230000004044 response Effects 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 3
- 238000010276 construction Methods 0.000 description 3
- 238000000605 extraction Methods 0.000 description 3
- 230000009467 reduction Effects 0.000 description 3
- 230000001133 acceleration Effects 0.000 description 2
- 230000004397 blinking Effects 0.000 description 2
- 239000003086 colorant Substances 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 230000001419 dependent effect Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- UDHXJZHVNHGCEC-UHFFFAOYSA-N Chlorophacinone Chemical compound C1=CC(Cl)=CC=C1C(C=1C=CC=CC=1)C(=O)C1C(=O)C2=CC=CC=C2C1=O UDHXJZHVNHGCEC-UHFFFAOYSA-N 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 230000003190 augmentative effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 230000001747 exhibiting effect Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000007499 fusion processing Methods 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000003825 pressing Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
- G06F3/04815—Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/29—Geographical information databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9537—Spatial or temporal dependent retrieval, e.g. spatiotemporal queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F3/04845—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range for image manipulation, e.g. dragging, rotation, expansion or change of colour
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/006—Mixed reality
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Data Mining & Analysis (AREA)
- Computer Graphics (AREA)
- Computer Hardware Design (AREA)
- Software Systems (AREA)
- Remote Sensing (AREA)
- Processing Or Creating Images (AREA)
Abstract
The application discloses a data processing method, a device, equipment and a readable storage medium, wherein the method comprises the following steps: responding to the cultural relic navigation operation in the display application, and displaying a shooting picture aiming at a target site and shot by a shooting component in the display application; the target site is an object site triggering the cultural relic navigation operation; when the shooting picture contains a target area for displaying virtual cultural relic resources in a target site, displaying the virtual cultural relic resources positioned on the target area in the shooting picture; when the position relation between the shooting component and the target area changes and the shooting picture continuously contains the target area, the display style of the virtual cultural relic resource in the shooting picture is updated. The embodiment of the application can be applied to the map field, and can enrich the mode of displaying the online relics and improve the immersion of the sightseeing of the online relics.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a data processing method, apparatus, device, and readable storage medium.
Background
When the museum is exhibited under the line, the cultural relics can not be exhibited due to the reasons of lending, repairing and the like, so that the characteristic appearance of the cultural relics can not be known when the tourist is in downstream of the line, the exhibition mode of the cultural relics is still too single, and real-time navigation of the museum in the cultural relics can not be provided, and the overall tour continuity and immersion feel are poor.
Disclosure of Invention
The embodiment of the application provides a data processing method, a device, equipment and a readable storage medium, enriches the mode of displaying the line offerings and improves the immersion of the sightseeing of the line offerings.
In one aspect, an embodiment of the present application provides a data processing method, including:
responding to the cultural relic navigation operation in the display application, and displaying a shooting picture aiming at a target site and shot by a shooting component in the display application; the target site is an object site triggering the cultural relic navigation operation;
when the shooting picture contains a target area for displaying virtual cultural relic resources in a target site, displaying the virtual cultural relic resources positioned on the target area in the shooting picture; the cultural relic type of the virtual cultural relic resource is associated with a target site, and the display position of the virtual cultural relic resource in a shooting picture is associated with the display position of a target area in the shooting picture;
When the position relation between the shooting component and the target area changes and the shooting picture continuously contains the target area, the display style of the virtual cultural relic resource in the shooting picture is updated.
Wherein, still include:
responding to navigation operation aiming at a target site in the display application, and displaying object position information, site position information and route navigation information between the object position information and the site position information on an electronic map in the display application; the object position information is used for indicating the position of an object triggering the navigation operation; the site location information is used for indicating the location of the target site; the route navigation information is used to guide the object to a target site for the display relics.
Wherein, still include:
when the shooting picture contains the virtual cultural relic resource, responding to the triggering operation of a shooting control in the display application, and storing the cultural relic shooting picture aiming at the virtual cultural relic resource and shot by the shooting component in the display application; the cultural relic shooting picture comprises a target site and virtual cultural relic resources.
Wherein, still include:
displaying a virtual cultural relic route from an object to a target area in a shooting picture, and displaying a guide control for the virtual cultural relic route;
And responding to the triggering operation of the guide control in the shooting picture, and playing the voice cultural relic introduction aiming at the virtual cultural relic resource.
Wherein, still include:
displaying a virtual field map aiming at a target field in a shooting picture, and displaying field display position information corresponding to one or more cultural relic resources to be navigated in the virtual field map; the site display position information is used for representing the position of a to-be-navigated cultural relic resource in a target site, and the one or more to-be-navigated cultural relic resources comprise virtual cultural relic resources;
if the shooting picture also comprises the virtual cultural relic resource, the display mode of the site display position information of the virtual cultural relic resource in the virtual site map is different from the display mode of the site display position information of the residual cultural relic resource to be guided in the virtual site map, and the residual cultural relic resource to be guided comprises one or more cultural relic resources to be guided except the virtual cultural relic resource in the cultural relic resource to be guided.
Wherein, still include:
responding to triggering operation aiming at virtual cultural relic resources in a shooting picture, displaying a cultural relic display window in the shooting picture, and displaying the virtual cultural relic resources in the cultural relic display window;
And responding to the interactive operation aiming at the virtual cultural relic resources in the cultural relic display window, and performing rotation or scaling operation on the virtual cultural relic resources in the cultural relic display window.
The method for displaying the virtual cultural relic resource on the target area in the shooting picture comprises the following steps:
when an obstacle exists between the object and the target area, determining an occlusion area of the obstacle to the target area along a shooting view angle of the shooting assembly, and displaying non-occluded virtual cultural relic resources in the virtual cultural relic resources on the target area in a shooting picture; the virtual cultural relic resource which is not shielded is a virtual cultural relic resource except for the virtual cultural relic resource covered by the shielding area in the virtual cultural relic resource.
Wherein, still include:
responding to the triggering operation of a model collection control for the virtual cultural relic resources in a shooting picture, displaying a cultural relic collection window in the shooting picture, and displaying the virtual cultural relic resources in the cultural relic collection window;
responding to detailed operation of virtual cultural relic resources in the cultural relic collection window, and displaying historical information of the cultural relic for the virtual cultural relic resources; the historical information of the cultural relics is a word description of the cultural relics corresponding to the virtual cultural relic resources.
The position relationship comprises a distance relationship and an azimuth relationship, and the display style comprises a display size and a display view angle; when the position relation between the shooting component and the target area changes and the shooting picture continuously contains the target area, updating the display style of the virtual cultural relic resource in the shooting picture comprises the following steps:
When the distance relation between the shooting component and the target area is changed and the shooting picture continuously comprises the target area, changing the display size of the virtual cultural relic resource in the shooting picture; displaying the virtual cultural relic resources in the size change in a target area all the time;
when the azimuth relation between the shooting component and the target area is changed and the shooting picture continuously comprises the target area, changing the display view angle of the virtual cultural relic resource in the shooting picture; virtual cultural relic resources in the display view angle change are always displayed in the target area.
Wherein, still include:
acquiring point cloud materials of a target site, and constructing a site virtual space aiming at the target site according to the point cloud materials; the site virtual space has the same three-dimensional space information as the target site;
determining site display position information of one or more to-be-navigated relic resources in a target site;
acquiring a shooting position of a shooting picture in a field virtual space;
detecting a matching relation between site display position information and shooting positions corresponding to each cultural relic resource to be navigated respectively;
if the field display position information matched with the shooting position exists, determining the to-be-navigated cultural relic resource corresponding to the field display position information matched with the field display position information as a virtual cultural relic resource, and determining that the shooting picture contains a target area for displaying the virtual cultural relic resource in the target field.
Wherein acquiring a shooting position of a shooting picture in a venue virtual space includes:
performing regional characteristic recognition on a shot picture to obtain picture regional characteristics, acquiring inertial measurement information of an object, and determining regional visual angle characteristics of the object on a target site based on the inertial measurement information and the picture regional characteristics;
and carrying out global image retrieval in the field virtual space according to the regional visual angle characteristics, and determining the space position matched with the regional visual angle characteristics and retrieved in the field virtual space as the shooting position of the shooting picture in the field virtual space.
The method for detecting the matching relation between the site display position information and the shooting position corresponding to each to-be-navigated cultural relic resource comprises the following steps:
acquiring a first coordinate range of site display position information corresponding to each cultural relic resource to be navigated, and acquiring a second coordinate range of a shooting position;
if a first coordinate range which is overlapped with the second coordinate range is present, determining that the field display position information corresponding to the first coordinate range which is overlapped with the second coordinate range is matched with the shooting position;
wherein, still include:
and determining a region overlapping with the region where the first coordinate range exists in the shooting picture as a target region.
The method for displaying the virtual cultural relic resource on the target area in the shooting picture comprises the following steps:
determining a parallax angle between the shooting position and the site display position information and a parallax depth between the shooting position and the site display position information based on the shooting position of the shooting picture and the site display position information of the virtual cultural relics resource;
determining a display view angle of the virtual cultural relic resource in a shooting picture based on the parallax angle;
determining the display size of the virtual cultural relic resource in a shooting picture based on the parallax depth;
and acquiring material texture information and cultural relic model vertex coordinate information associated with the virtual cultural relic resources, and rendering the virtual cultural relic resources at a target area in a shooting picture according to a rendering mapping relation, a parallax angle and a parallax depth between the cultural relic model vertex coordinate information and the material texture information.
Wherein, still include:
when the shooting picture does not contain a target area in the target field and the distance between the position of the object in the target field and the target area is smaller than a distance threshold value, playing a voice prompt for prompting the object to browse virtual cultural relic resources; the voice prompts include location navigation prompts that are determined based on a positional relationship between the location of the object in the target site and the target area.
Wherein, still include:
when the shooting picture contains virtual cultural relic resources, playing a voice cultural relic introduction aiming at the virtual cultural relic resources;
when the position relation between the position of the object in the target site and the target area is changed and the shooting picture does not contain the target area any more, suspending the introduction of the voice cultural relics aiming at the virtual cultural relic resources, and switching and playing the voice cultural relic navigation to the next cultural relic resource to be navigated; the next cultural relic resource to be browsed refers to a cultural relic resource which is adjacent to the virtual cultural relic resource in the target site and is not browsed by the object.
Wherein, still include:
when the shooting picture contains virtual cultural relic resources, and the proportion of the virtual unique identifiers in the virtual cultural relic resources to the shooting picture is larger than or equal to a proportion threshold value, playing the voice cultural relic introduction aiming at the virtual cultural relic resources and associated with the virtual unique identifiers;
when the duration of the key part in the virtual cultural relic resource is equal to the duration threshold value in the process of playing the voice cultural relic introduction, the played voice cultural relic introduction is switched to the voice introduction content aiming at the key part.
In one aspect, an embodiment of the present application provides a data processing apparatus, including:
The cultural relic guiding module is used for responding to cultural relic guiding operation in the display application and displaying a shooting picture aiming at a target site and shot by the shooting component in the display application; the target site is an object site triggering the cultural relic navigation operation;
the cultural relic display module is used for displaying the virtual cultural relic resources positioned on the target area in the shooting picture when the shooting picture contains the target area for displaying the virtual cultural relic resources in the target field; the cultural relic type of the virtual cultural relic resource is associated with a target site, and the display position of the virtual cultural relic resource in a shooting picture is associated with the display position of a target area in the shooting picture;
and the cultural relic updating module is used for updating the display view angle of the virtual cultural relic resource in the shooting picture when the position relation between the shooting component and the target area is changed and the shooting picture continuously comprises the target area.
Wherein, still include:
the navigation module is used for responding to the navigation operation aiming at the target site in the display application, and displaying the object position information, the site position information and the route navigation information between the object position information and the site position information on the electronic map in the display application; the object position information is used for indicating the position of an object triggering the navigation operation; the site location information is used for indicating the location of the target site; the route navigation information is used to guide the object to a target site for the display relics.
Wherein, still include:
the shooting and saving module is used for responding to the triggering operation of the shooting control in the display application when the shooting picture contains the virtual cultural relic resource, and saving the cultural relic shooting picture aiming at the virtual cultural relic resource and shot by the shooting component in the display application; the cultural relic shooting picture comprises a target site and virtual cultural relic resources.
Wherein, still include:
the guiding and broadcasting module is used for displaying a virtual cultural relic route from the object to the target area in the shooting picture and displaying guiding and broadcasting controls aiming at the virtual cultural relic route;
and the guide module is also used for responding to the triggering operation of the guide control in the shooting picture and playing the voice cultural relic introduction aiming at the virtual cultural relic resource.
Wherein, still include:
the virtual map module is used for displaying a virtual field map aiming at a target field in a shooting picture, and displaying field display position information corresponding to one or more cultural relics to be guided respectively in the virtual field map; the site display position information is used for representing the position of a to-be-navigated cultural relic resource in a target site, and the one or more to-be-navigated cultural relic resources comprise virtual cultural relic resources; if the shooting picture also comprises the virtual cultural relic resource, the display mode of the site display position information of the virtual cultural relic resource in the virtual site map is different from the display mode of the site display position information of the residual cultural relic resource to be guided in the virtual site map, and the residual cultural relic resource to be guided comprises one or more cultural relic resources to be guided except the virtual cultural relic resource in the cultural relic resource to be guided.
Wherein, still include:
the cultural relic interaction module is used for responding to the triggering operation aiming at the virtual cultural relic resources in the shooting picture, displaying a cultural relic display window in the shooting picture, displaying the virtual cultural relic resources in the cultural relic display window, and responding to the interaction operation aiming at the virtual cultural relic resources in the cultural relic display window, and performing rotation or scaling operation on the virtual cultural relic resources in the cultural relic display window.
The cultural relic display module is specifically used for determining a shielding area of the obstacle to the target area along the shooting view angle of the shooting assembly when the obstacle exists between the object and the target area, and displaying the non-shielded virtual cultural relic resource in the virtual cultural relic resources on the target area in the shooting picture; the virtual cultural relic resource which is not shielded is a virtual cultural relic resource except for the virtual cultural relic resource covered by the shielding area in the virtual cultural relic resource.
Wherein, still include:
the cultural relic collection module is used for responding to the triggering operation of the model collection control for the virtual cultural relic resources in the shooting picture, displaying a cultural relic collection window in the shooting picture and displaying the virtual cultural relic resources in the cultural relic collection window; responding to detailed operation of virtual cultural relic resources in the cultural relic collection window, and displaying historical information of the cultural relic for the virtual cultural relic resources; the historical information of the cultural relics is a word description of the cultural relics corresponding to the virtual cultural relic resources.
Wherein, the historical relic updates the module, include:
the cultural relic size updating unit is used for changing the display size of the virtual cultural relic resource in the shooting picture when the distance relation between the shooting component and the target area is changed and the shooting picture continuously comprises the target area; displaying the virtual cultural relic resources in the size change in a target area all the time;
the cultural relic visual angle updating unit is used for changing the display visual angle of the virtual cultural relic resource in the shooting picture when the azimuth relation between the shooting component and the target area is changed and the shooting picture continuously comprises the target area; virtual cultural relic resources in the display view angle change are always displayed in the target area.
Wherein, still include:
the voice prompt module is used for playing a voice prompt for prompting the object to browse the virtual cultural relic resources when the shooting picture does not contain a target area in the target site and the distance between the position of the object in the target site and the target area is smaller than a distance threshold value; the voice prompts include location navigation prompts that are determined based on a positional relationship between the location of the object in the target site and the target area.
Wherein, still include:
the voice cultural relic introduction module is used for playing voice cultural relic introduction aiming at the virtual cultural relic resources when the shooting picture contains the virtual cultural relic resources;
the voice cultural relic introduction module is also used for suspending voice cultural relic introduction aiming at the virtual cultural relic resource when the position of the object in the target site changes with the position relation between the target area and the target area is not contained in the shooting picture, and switching and playing the voice cultural relic navigation aiming at the next cultural relic resource to be navigated; the next cultural relic resource to be browsed refers to a cultural relic resource which is adjacent to the virtual cultural relic resource in the target site and is not browsed by the object.
Wherein, still include:
the cultural relic part introduction module is used for playing the voice cultural relic introduction aiming at the virtual cultural relic resource, which is associated with the virtual unique identifier, when the photographed picture contains the virtual cultural relic resource and the proportion of the virtual unique identifier in the virtual cultural relic resource in the photographed picture is larger than or equal to a proportion threshold value;
and the relic part introduction module is also used for switching the played voice relic introduction to the voice introduction content aiming at the key part when the duration of the key part in the virtual relic resource contained in the shot picture is equal to the duration threshold in the process of playing the voice relic introduction. Wherein, still include:
The site virtual space module is used for acquiring point cloud materials of the target site and constructing a site virtual space aiming at the target site according to the point cloud materials; the site virtual space has the same three-dimensional space information as the target site;
the processing module is used for determining site display position information of one or more cultural relics resources to be navigated in a target site and acquiring shooting positions of shooting pictures in a site virtual space;
the detection module is used for detecting the matching relation between the site display position information and the shooting position corresponding to each cultural relic resource to be navigated respectively; if the field display position information matched with the shooting position exists, determining the to-be-navigated cultural relic resource corresponding to the field display position information matched with the field display position information as a virtual cultural relic resource, and determining that the shooting picture contains a target area for displaying the virtual cultural relic resource in the target field.
Wherein, processing module includes:
the regional visual angle characteristic unit is used for carrying out regional characteristic identification on a shot picture to obtain picture regional characteristics, acquiring inertial measurement information of an object, and determining regional visual angle characteristics of the object on a target site based on the inertial measurement information and the picture regional characteristics;
And the global image retrieval unit is used for performing global image retrieval in the field virtual space according to the regional visual angle characteristics, and determining the space position matched with the regional visual angle characteristics and retrieved in the field virtual space as the shooting position of the shooting picture in the field virtual space.
Wherein, detection module includes:
the coordinate range acquisition unit is used for acquiring a first coordinate range of the site display position information corresponding to each cultural relic resource to be navigated and acquiring a second coordinate range of the shooting position;
a coordinate range matching unit, configured to determine that the site display position information corresponding to the first coordinate range overlapping with the second coordinate range matches the shooting position if there is the first coordinate range overlapping with the second coordinate range;
and the coordinate range matching unit is also used for determining the area overlapping with the area where the first coordinate range exists in the shooting picture as a target area.
Wherein, the historical relic display module includes:
the model parallax unit is used for determining a parallax angle between the shooting position and the site display position information and a parallax depth between the shooting position and the site display position information based on the shooting position of the shooting picture and the site display position information of the virtual cultural relics resource;
The model parallax unit is also used for determining the display view angle of the virtual cultural relic resource in the shooting picture based on the parallax angle; determining the display size of the virtual cultural relic resource in a shooting picture based on the parallax depth;
the model rendering unit is used for acquiring material texture information and cultural relic model vertex coordinate information associated with the virtual cultural relic resources, and rendering the virtual cultural relic resources at a target area in a shooting picture according to a rendering mapping relation, a parallax angle and a parallax depth between the cultural relic model vertex coordinate information and the material texture information.
In one aspect, an embodiment of the present application provides a computer device, including: a processor, a memory, and a network interface;
the processor is connected with the memory and the network interface, wherein the network interface is used for providing a data communication function, the memory is used for storing a computer program, and the computer program when executed by the processor causes the computer device to execute the method provided by the embodiment of the application.
In one aspect, an embodiment of the present application provides a readable storage medium storing a computer program adapted to be loaded and executed by a processor, so that a computer device having the processor performs the method provided by the embodiment of the present application.
In one aspect, embodiments of the present application provide a computer program product comprising a computer program stored in a readable storage medium. A processor of a computer device reads the computer program from a readable storage medium, and the processor executes the computer program to cause the computer device to execute the method provided by the embodiment of the present application.
According to the embodiment of the application, the cultural relics guiding operation in the display application is responded, and the shooting picture aiming at the target site and shot by the shooting component is displayed in the display application; the target site is an object site triggering the cultural relic navigation operation; when the shooting picture contains a target area for displaying virtual cultural relic resources in a target site, displaying the virtual cultural relic resources positioned on the target area in the shooting picture; the cultural relic type of the virtual cultural relic resource is associated with a target site, and the display position of the virtual cultural relic resource in a shooting picture is associated with the display position of a target area in the shooting picture; when the position relation between the shooting component and the target area changes and the shooting picture continuously contains the target area, the display style of the virtual cultural relic resource in the shooting picture is updated. Therefore, the virtual cultural relic resource which is not associated with the exhibition is displayed in the target area of the shooting picture, the virtual cultural relic resource can be combined with environmental objects (scenes in the real environment) in the shooting picture so as to restore the off-line watching experience of the exhibition-free cultural relic, and the method can enrich the exhibition mode of the off-line cultural relic and improve the immersion sense of the sightseeing of the off-line cultural relic.
Drawings
In order to more clearly illustrate the embodiments of the application or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic diagram of a network architecture according to an embodiment of the present application;
FIG. 2 is a schematic diagram of a data processing scenario provided in an embodiment of the present application;
FIG. 3a is a schematic flow chart of a data processing method according to an embodiment of the present application;
FIG. 3b is a schematic diagram of another data processing scenario provided by an embodiment of the present application;
FIG. 3c is a schematic diagram of another data processing scenario provided by an embodiment of the present application;
FIG. 4 is a flowchart of another data processing method according to an embodiment of the present application;
FIG. 5 is a flowchart of another data processing method according to an embodiment of the present application;
FIG. 6 is a schematic diagram of a virtual cultural relic resource exhibition architecture according to an embodiment of the present application;
FIG. 7 is a schematic diagram of a data processing apparatus according to an embodiment of the present application;
Fig. 8 is a schematic structural diagram of a computer device according to an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
Referring to fig. 1, fig. 1 is a schematic diagram of a network architecture according to an embodiment of the present application. The network architecture may include the terminal device 100 and the platform server 200, where the terminal device 100 may have a communication connection with the platform server 200, where the communication connection is not limited to a connection manner, and may be directly or indirectly connected through a wired communication manner, may be directly or indirectly connected through a wireless communication manner, or may be otherwise connected through other manners, and the application is not limited herein.
It should be appreciated that the terminal device 100 as shown in fig. 1 may be installed with a presentation application that, when run on the terminal device 100, may interact with the platform server 200 as shown in fig. 1. The exhibition application can be an instant messaging application, a live broadcast application, a short video application, a music application, a social application, a shopping application, a game application, a novel application, a payment application, a browsing application and the like. The presentation application may be an independent client, or may be an embedded sub-client integrated in an application (such as an instant messaging application, a social application, a video application, etc.), which is not limited herein.
Specifically, the application can collect high-precision point cloud data (point data set of the appearance surface of the article obtained by a measuring instrument) of a target site (such as a museum, a souvenir hall, a cultural relic exhibition and the like) through the point cloud and the laser equipment in advance, and can collect high-precision point cloud data through photo materials of the target site.
The platform server 200 may establish the site virtual space of the target site through the high-precision point cloud data, and virtual cultural relic resources in the site virtual space (i.e., a cultural relic model established by cultural relic resources to be displayed on the target site), where the cultural relic resources may be cultural relics being displayed (i.e., entity cultural relics being displayed in the target scene), or cultural relics not being displayed due to repair, lending, etc. According to the planned display position of the cultural relic resource on the target site, site display position information of the virtual cultural relic resource corresponding to the cultural relic resource in the site virtual space can be determined.
The platform server 200 may make a calibration (set its spatial coordinates or set a spatial region) for the service area of the target site in the site virtual space, so that the site virtual space has the same three-dimensional spatial information as the target site. The service area may include location information of the cultural relic resource (site display location information of the virtual cultural relic resource), and the service area may further include a main meeting place, an entrance, a road, an obstacle, and other space areas of the target site.
The terminal device 100 triggers a navigation operation in the presentation application, determines a target site to which the user is going, and transmits route navigation information (which may include the user's location, the location of the target site, and a traffic pattern) to the platform server 200.
The platform server 200 creates a venue navigation route between a user location and a target venue location (which may also be referred to as venue location information) according to the route navigation information, and may also determine a navigation route distance (an estimated distance required for the user to complete the venue navigation route in the target venue) and a navigation route time (an estimated time required for the user to complete the venue navigation route in the target venue). The platform server 200 transmits the venue navigation route, the navigation route distance, and the navigation route time to the terminal device 100.
The terminal device 100 displays the venue navigation route, the navigation route distance, and the navigation route time on the electronic map of the presentation application, and the user can follow the venue navigation route to the target venue navigated by the user.
After reaching the target site, the terminal device 100 may trigger the cultural relic navigation operation in the exhibition application, start the AR (Augmented Reality ) camera function in the exhibition application, so as to call the shooting component (may be a camera) of the terminal device 100, obtain a shooting picture of the target site (may be a picture seen by a user through the shooting component at the target site) through shooting by the shooting component, and display the shooting picture of the target site in real time in the exhibition application.
The terminal 100 may display a virtual cultural relic route on the shooting picture, where the virtual cultural relic route may be used to guide the user to the location where the cultural relic resource to be navigated is located, and browse the cultural relic resource to be navigated, where the cultural relic resource to be navigated may include a cultural relic resource in a cultural relic resource that is in a exhibition and a cultural relic resource that is not in exhibition, where the cultural relic resource that is not in exhibition may perform AR display in the shooting picture by creating a corresponding virtual cultural relic resource. The terminal device 100 may also determine, as the cultural relic resource to be navigated, a cultural relic resource that the user does not browse the cultural relic by displaying log information in the application (may record the cultural relic resource that the user finishes browsing the cultural relic by displaying the application).
Alternatively, when the terminal device 100 triggers the navigation operation of the cultural relics, the terminal device 100 may call the shooting component to display an environment picture (a picture shot by the shooting component) while the terminal device 100 is in the way of going to the target site, and at this time, the terminal device 100 may display a virtual cultural relic route with a navigation arrow in the shot environment picture, so as to guide the user to the location where the cultural relic resource to be navigated is located.
The terminal device 100 may determine, in combination with log information of the display application, a cultural relic resource that is not browsed by the user as a cultural relic resource to be browsed, display a virtual field map of the target field in the photographing screen, and display field display position information corresponding to the cultural relic resource to be browsed in the virtual field map.
Optionally, the terminal device 100 may perform differential display on the displayed cultural relic resources to be navigated in the virtual field map of the shot picture, for example, the physical cultural relic resources and the virtual cultural relic resources in the cultural relic resources to be navigated may be differential displayed, or may perform differential display on the browsed cultural relic resources and the cultural relic resources to be navigated in the virtual field map, where the mode of differential display may include different colors, shapes, and display animations (for example, may be blinking, lighting, etc.).
It can be understood that the terminal device 100 can determine, by displaying the log information of the application, all the cultural relic resources (i.e. the cultural relic resources to be browsed by the user) in the target site, and according to the virtual cultural relic route for each cultural relic resource to be browsed, determine the total route from the current position (i.e. the position of the user, which can be outside the target site or can be in the target site) to the completion of browsing all the cultural relic resources to be browsed (which can be referred to as a cultural relic browsing route), and further determine the browsing route distance (the estimated distance required by the user to complete the cultural relic browsing route) and the browsing route time (the estimated time required by the user to complete the cultural relic browsing route). For example, the cultural relic guiding route may include a route that a user walks from a current location to a target site (i.e., a site navigation route), a route that enters from a certain entrance of the target site and completes browsing of cultural relic resources to be guided in sequence (i.e., a virtual cultural relic route), and a route that exits from a certain exit of the target site.
The cultural relic guiding route can further comprise a virtual cultural relic route of the cultural relic resource for the user to finish cultural relic browsing through the display application.
When the shooting view angle range determined by the shooting picture of the terminal device 100 overlaps with the preset model display range corresponding to a certain virtual cultural relic resource, the shooting picture can be considered to include a target area for displaying the virtual cultural relic resource, and the terminal device 100 displays the virtual cultural relic resource on the target area of the shooting picture.
The model display range may be a display range of the determined virtual cultural relic resource in a field virtual space according to field display position information and model information (which may include a model size and a placement direction of the virtual cultural relic resource) of the virtual cultural relic resource. The photographing view angle range may be a range of view angles corresponding to photographing positions in the venue virtual space.
According to the method, the virtual cultural relic resource which is associated with the non-exhibition cultural relic is displayed in the target area of the shooting picture, the virtual cultural relic resource can be combined with the environmental objects (the scenes in the real environment) in the shooting picture so as to restore the offline watching experience of the non-exhibition cultural relic, and the method can enrich the manner of displaying the offline cultural relic and improve the immersion sense of the online cultural relic tour.
Referring to fig. 2, fig. 2 is a schematic diagram of a data processing scenario provided in an embodiment of the present application. Taking the target place as a museum B (a museum containing virtual cultural relic resources) as an example, a user may respond to a navigation operation on the museum B in the display application through the terminal device 100, and display, on an electronic map in a service scene 101 of the display application, a user position a, a position of the museum B, and route navigation information between the user position a and the position of the museum B, where the user position may be "my position a", the position of the museum B may be "museum B", the route navigation information may include a route between the user position a and the position of the museum B, a guiding route distance (i.e., "143 m" in the service scene 101), and a guiding route time (i.e., "3 min" in the service scene 101), where the guiding route distance is used to indicate a distance required for the user to complete the browsing of the cultural relics in the museum B, and the guiding route time is used to indicate an estimated time required for the user to complete the browsing of the cultural relics in the museum B. The user is guided to the museum B by the route guidance information.
The user may operate the live-action navigation control C in the service scene 101 (the triggering operation on the live-action navigation control C may also be referred to as a cultural relic guiding operation, which may trigger the AR camera function in the display application to be started), enter live-action navigation (i.e. the service scene 102), the terminal device 100 photographs the photographed image of the target site through the photographing component, and displays the photographed image for the museum B in the display application, where the photographed image may display a virtual cultural relic route with a navigation arrow, and the user may follow the virtual cultural relic route to reach the museum B.
When the shooting view angle range determined by the shooting picture of the terminal device 100 overlaps with the preset model display range corresponding to a certain virtual cultural relic resource, the shooting picture can be considered to include a target area D (i.e., an area D in the service scene 102) for displaying the virtual cultural relic resource (which may be the virtual cultural relic resource E), and the site display position information of the certain cultural relic resource can be considered to be matched with the shooting position.
The model display range may be a range of the determined virtual cultural relic resource in a field virtual space according to the field display position information and model information (which may include a model size and a placement direction of the virtual cultural relic resource) of the virtual cultural relic resource. The photographing view angle range may be a range in which a view angle corresponding to the photographing position is determined in the venue virtual space.
The terminal device 100 displays, on the target area D of the shot picture, the virtual cultural relic resource E (i.e., the virtual cultural relic resource E in the service scene 103) after rendering processing corresponding to the cultural relic resource (the entity of the cultural relic resource is not in the target site). Virtual cultural relic resources in a shooting picture can be fixed on a target area of a field virtual space through an AR technology, so that the virtual cultural relic resources can change a display style along with movement of a user. As shown in the service scenario 103 and the service scenario 104, when a user approaches, a larger virtual cultural relic resource E is seen; when the user walks away, smaller virtual cultural relic resources E can be seen; and when the virtual cultural relic resource is watched from different angles, the virtual cultural relic resource E under different angles is displayed so as to simulate and realize the visual effect of the cultural relic resource during exhibition.
Referring to fig. 3a, fig. 3a is a flowchart of a data processing method according to an embodiment of the present application, where the data processing method may be performed by a computer device, and the computer device may include a terminal device 100 or a platform server 200 shown in fig. 1, and as shown in fig. 3a, the method may include:
step S101, responding to a cultural relic guiding operation in a display application, and displaying a shooting picture aiming at a target site, shot by a shooting component, in the display application; the target site is an object site triggering the cultural relic navigation operation;
specifically, the computer device may trigger a cultural relic navigation operation in the exhibition application to start an AR camera function in the exhibition application, so as to call a shooting component (may be a camera) of the terminal device 100, obtain a shooting picture of the target site (may be a picture seen by a user through the shooting component at the target site) through shooting by the shooting component, and display the shooting picture of the target site in real time in the exhibition application.
The AR camera function may be a function of combining virtual contents (which may be virtual images or characters) with environmental objects (which may be scenes in a real environment), and may achieve the purpose of enhancing information transmission or displaying virtual contents by superimposing the virtual contents on the environmental objects.
The shot screen in the display application of the computer device can also display an electronic map, a virtual cultural relic route and a virtual site map.
The specific process of displaying the electronic map may be: responding to navigation operation aiming at a target site in the display application, and displaying object position information, site position information and route navigation information between the object position information and the site position information on an electronic map in the display application; the object position information is used for indicating the position of an object triggering the navigation operation; the site location information is used for indicating the location of the target site; the route navigation information is used to guide the object to a target site for the display relics.
Specifically, referring to fig. 3b, fig. 3b is a schematic diagram of another data processing scenario provided in the embodiment of the present application. As shown in fig. 3b, the computer device may trigger a navigation operation in the presentation application, determining the target venue to which the user is going (which may be a museum Q), the user's location (user location F, which may also be referred to as my location F), and the way of traffic to the target venue (in the example of a walking action). The computer device may create a venue navigation route between the user location and the target venue location (which may be the location of museum Q or may be referred to as venue location information), and may also determine a tour route distance (an estimated distance required by the user to complete the tour route for the cultural relics in the target venue may be "143 meters" in the business scene 111, and the description of the tour route may be referred to below) and a tour route time (an estimated time required by the user to complete the tour route for the cultural relics in the target venue may be "3 minutes" in the business scene 111). The computer device displays the venue navigation route, the tour route distance, and the tour route time on an electronic map of the presentation application, along which the user may travel to a target venue navigated by the user.
The specific process of displaying the virtual cultural relic route can be as follows: and displaying the virtual cultural relic route from the object to the target area in the shooting picture, and displaying a guide control for the virtual cultural relic route.
Specifically, the computer device may display a virtual cultural relic route on the shooting picture, the virtual cultural relic route may be used to guide the user to a position where the navigation cultural relic resource is located, the cultural relic resource to be navigated is browsed, the cultural relic resource to be navigated may include a cultural relic resource in a exhibition and a cultural relic resource not in exhibition, and the cultural relic resource not in exhibition may be displayed in the shooting picture by creating a corresponding virtual cultural relic resource. The computer device can also determine the cultural relic resource which is not browsed by the user as the cultural relic resource to be browsed by displaying the log information in the application (the cultural relic resource which is browsed by the user through the display application can be recorded).
For example, as shown in fig. 3b, the computer device may display a virtual cultural relic route with a navigation arrow (i.e. a route K with a navigation arrow shown in the service scene 112 in fig. 3 b) in the captured environmental screen, and guide the user to the location where the cultural relic resource to be navigated is located.
The specific process of displaying the virtual cultural relic map may be: displaying a virtual field map aiming at a target field in a shooting picture, and displaying field display position information corresponding to one or more cultural relic resources to be navigated in the virtual field map; the site display position information is used for representing the position of a to-be-navigated cultural relic resource in a target site, and the one or more to-be-navigated cultural relic resources comprise virtual cultural relic resources; if the shooting picture also comprises the virtual cultural relic resource, the display mode of the site display position information of the virtual cultural relic resource in the virtual site map is different from the display mode of the site display position information of the residual cultural relic resource to be guided in the virtual site map, and the residual cultural relic resource to be guided comprises one or more cultural relic resources to be guided except the virtual cultural relic resource in the cultural relic resource to be guided.
Specifically, the computer device may determine, in combination with log information of the display application, a cultural relic resource that is not browsed by the user as a cultural relic resource to be browsed, display a virtual site map of the target site in the shooting picture, and display site display position information corresponding to the cultural relic resource to be browsed in the virtual site map.
The displaying of the virtual venue map may be as shown in a service scene 113 in fig. 3b, where a point F in the virtual scene map displayed by the service scene 113 may refer to a user position, and a point J is venue display position information of a to-be-navigated cultural relic resource (may be a virtual cultural relic resource) indicated by a virtual cultural relic route.
Optionally, the computer device may perform differential display on the displayed cultural relic resource to be navigated in the virtual field map of the shot picture, for example, the physical cultural relic resource and the virtual cultural relic resource in the cultural relic resource to be navigated may be differential displayed, or may perform differential display on the browsed cultural relic resource and the cultural relic resource to be navigated in the virtual field map, where the mode of differential display may include different colors, shapes, and display animations (for example, may be blinking, lighting, etc.).
Optionally, the virtual cultural relic route is displayed, and meanwhile, a voice prompt function is also provided, and the following process can be realized through the voice prompt function: when the shooting picture does not contain a target area in the target field and the distance between the position of the object in the target field and the target area is smaller than a distance threshold value, playing a voice prompt for prompting the object to browse virtual cultural relic resources; the voice prompts include location navigation prompts that are determined based on a positional relationship between the location of the object in the target site and the target area.
Specifically, when the shooting picture of the computer device does not include the target area, and the distance between the position of the shooting picture of the computer device (the position of the user) and the virtual cultural relic route of the target area is smaller than the distance threshold (may be a preset value, may be 10 meters, 15 meters, etc.), that is, when the shooting position is near the position (the site display position) of the virtual cultural relic resource (may also be the cultural relic resource to be navigated), a voice prompt is played to prompt the user of the virtual cultural relic resource around, and guide the user to the position of the virtual cultural relic resource. For example, the voice prompt may be: there is a virtual cultural relic resource in the left hand direction.
Wherein, the voice prompt may include a navigation prompt, the navigation prompt may be a navigation voice message to a virtual cultural relic resource in the voice prompt, the navigation prompt may be: forward 10 meters, turn left, and reach the virtual cultural relic resource.
It can be understood that the computer device can determine all the cultural relic resources (i.e. the cultural relic resources to be navigated) which are not browsed by the user in the target site through displaying the applied log information, and can determine the total route (which can be called as the cultural relic navigation route) from the current position (i.e. the position of the user, can be outside the target site, and can also be on the target site) to finish browsing all the cultural relic resources to be navigated according to the virtual cultural relic route for each cultural relic resource to be navigated, so as to determine the navigation route distance (the estimated distance required by the user to finish the cultural relic navigation route) and the navigation route time (the estimated time required by the user to finish the cultural relic navigation route). For example, the cultural relic guiding route may include a route that a user walks from a current location to a target site (i.e., a site navigation route), a route that enters from a certain entrance of the target site and completes browsing of cultural relic resources to be guided in sequence (i.e., a virtual cultural relic route), and a route that exits from a certain exit of the target site.
The cultural relic guiding route can further comprise a virtual cultural relic route of the cultural relic resource for the user to finish cultural relic browsing through the display application.
Step S102, when a shooting picture contains a target area for displaying virtual cultural relic resources in a target site, displaying the virtual cultural relic resources positioned on the target area in the shooting picture; the cultural relic type of the virtual cultural relic resource is associated with a target site, and the display position of the virtual cultural relic resource in a shooting picture is associated with the display position of a target area in the shooting picture;
specifically, when the shooting view angle range determined by the shooting picture of the computer device overlaps with the model display range, that is, the shooting picture contains the target area, it can be considered that the site display position information of a certain cultural relic resource is matched with the shooting position. And the computer equipment displays the virtual cultural relic resource after rendering processing corresponding to the cultural relic resource (the entity of the cultural relic resource is not in the target site) on the target area of the shot picture.
The model display range may be a range of the determined virtual cultural relic resource in the field virtual space according to the field display position information and the model information (which may include a model size and a placement direction of the virtual cultural relic resource). The photographing view angle range may be a range in which a view angle corresponding to the photographing position is determined in the venue virtual space.
Referring to fig. 3c, fig. 3c is a schematic diagram of another data processing scenario according to an embodiment of the present application. As shown in fig. 3c, in the business scenario 121, when the user moves to the vicinity of the site presentation position information of a certain cultural relic resource, and the shooting view angle range determined by the shooting picture overlaps with the model presentation range. The computer device may display the rendered virtual cultural relic resource M as shown in the service scene 122 on the target area L of the photographed picture. Wherein the target area E is associated with a virtual cultural relic resource M.
The specific process of the computer device responding to the triggering operation of the guide control in the shooting picture can be as follows: and responding to the triggering operation of the guide control in the shooting picture, and playing the voice cultural relic introduction aiming at the virtual cultural relic resource.
Specifically, as shown in fig. 3c, the computer device may respond to the triggering operation of the guide control Q in the shooting picture in the service scene 121 to play the voice cultural relic introduction of the virtual cultural relic resource M displayed in the target area E.
The voice cultural relic introduction content can be determined based on the view angle (i.e. the shooting view angle range) of the shooting picture, if the currently browsed cultural relic introduction content is the front side of the virtual cultural relic resource, the information of the front side of the cultural relic associated with the virtual cultural relic resource (such as the texture and the detailed description of the front side of the cultural relic corresponding to the virtual cultural relic resource) can be introduced, and if the currently browsed cultural relic is the back side of the virtual cultural relic resource, the information of the virtual cultural relic resource related to the back side of the cultural relic (such as the texture and the detailed description of the back side of the cultural relic corresponding to the virtual cultural relic resource) can be introduced. The voice relic introduction can also be combined with a relic guiding route (or a virtual relic route), for example, navigation voice information corresponding to the virtual relic route of the user going to the site display position of the virtual relic resource M can be played, for example, the navigation voice information can be 10 meters forward and turn left, and reach the vicinity of the site display position of the virtual relic resource M (the position of the virtual relic resource M in the target area E can be seen through the shooting picture).
Alternatively, the specific process of playing the introduction of the voice cultural relic may be: when the shooting picture contains virtual cultural relic resources, playing a voice cultural relic introduction aiming at the virtual cultural relic resources; when the position relation between the position of the object in the target site and the target area is changed and the shooting picture does not contain the target area any more, suspending the introduction of the voice cultural relics aiming at the virtual cultural relic resources, and switching and playing the voice cultural relic navigation to the next cultural relic resource to be navigated; the next cultural relic resource to be browsed refers to a cultural relic resource which is adjacent to the virtual cultural relic resource in the target site and is not browsed by the object.
Specifically, when the virtual cultural relic resource is displayed in the target area of the shooting picture of the computer equipment (that is, the shooting view angle range determined by the shooting picture overlaps with the model display range), the computer equipment can play the voice cultural relic introduction of the virtual cultural relic resource currently displayed.
When the shooting position moves and the shooting picture changes and does not contain the target area, the computer equipment can pause the voice cultural relic introduction of the current virtual cultural relic resource, switch and play the navigation voice information (also called voice cultural relic navigation) of the next cultural relic resource to be navigated, and the navigation voice information can be determined through the associated virtual cultural relic resource. For example, the navigation voice information may be: the front is 10 meters, turns left and goes to the next cultural relic resource to be navigated. As shown in fig. 3c, in the business scenario 121, when the position of the virtual cultural relic resource M in the target area E is seen from the shooting picture, the voice cultural relic introduction of the virtual cultural relic resource M can be played, when the user shooting picture no longer includes the target area E, the voice cultural relic introduction is paused, the next cultural relic resource to be guided by the user is determined according to the cultural relic guiding route, and the navigation voice information corresponding to the virtual cultural relic route to the cultural relic resource to be guided is played.
Alternatively, the specific process of playing the introduction of the voice cultural relic may be: when the shooting picture contains virtual cultural relic resources, and the proportion of the virtual unique identifiers in the virtual cultural relic resources to the shooting picture is larger than or equal to a proportion threshold value, playing the voice cultural relic introduction aiming at the virtual cultural relic resources and associated with the virtual unique identifiers; when the duration of the key part in the virtual cultural relic resource is equal to the duration threshold value in the process of playing the voice cultural relic introduction, the played voice cultural relic introduction is switched to the voice introduction content aiming at the key part.
Specifically, when the shot picture of the computer device includes that the proportion of the virtual unique identifier to the shot picture is greater than or equal to the proportion threshold (the preset value), the computer device may play the voice cultural relic introduction associated with the virtual unique identifier. The virtual unique identifier can be article information of a nameplate, a base, a key part and the like which are uniquely bound with the virtual cultural relic resource, and the virtual unique identifier can be in the expression forms of characters, images, two-dimensional codes and the like. The link of the voice cultural relic introduction is associated with the virtual unique identifier, and the computer equipment can acquire the voice cultural relic introduction corresponding to the link by identifying the virtual unique identifier.
In the process of playing the voice relic introduction, when the duration of the key part (which can be texture, appearance characteristic and the like) of the virtual relic resource in the shooting picture reaches a time threshold (a preset value can be 3 seconds, 5 seconds and the like), the played voice relic introduction is switched to the voice introduction content of the key part, and the voice introduction content of the key part can be the voice introduction of the texture, the appearance characteristic.
The computer device may perform feature extraction on a key part of a virtual cultural relic resource in a shot picture to obtain a picture key feature, match the key feature corresponding to the virtual cultural relic resource (the key feature determined when the cultural relic resource is modeled) through the picture key feature, and may match only the key feature of the virtual cultural relic resource, and if the picture key feature is successfully matched with a certain key feature corresponding to the virtual cultural relic resource, the key part associated with the picture key feature may be determined, and the voice introduction content linked with the key part may be played.
The specific process of responding to the triggering operation of shooting and saving can be as follows: when the shooting picture contains the virtual cultural relic resource, responding to the triggering operation of a shooting control in the display application, and storing the cultural relic shooting picture aiming at the virtual cultural relic resource and shot by the shooting component in the display application; the cultural relic shooting picture comprises a target site and virtual cultural relic resources.
Specifically, as shown in fig. 3c, when the shot frame includes a virtual cultural relic resource, the computer device may respond to a trigger operation of the shooting control N in the service scene 122 by displaying the application, and store, in the displaying application, the cultural relic shot frame shot by the shooting component, where the cultural relic shot frame may be a frame including the target site and the virtual cultural relic resource M after rendering, and the cultural relic shot frame may also be a display frame in the displaying application.
The specific process of responding to the triggering operation of the cultural relic display can be as follows: responding to triggering operation aiming at virtual cultural relic resources in a shooting picture, displaying a cultural relic display window in the shooting picture, and displaying the virtual cultural relic resources in the cultural relic display window; and responding to the interactive operation aiming at the virtual cultural relic resources in the cultural relic display window, and performing rotation or scaling operation on the virtual cultural relic resources in the cultural relic display window.
Specifically, when the shooting picture contains the virtual cultural relic resource, the computer equipment can respond to the triggering operation of the virtual cultural relic resource in the shooting picture, display a cultural relic display window in the shooting picture, and display the virtual cultural relic resource in the cultural relic display window. The computer device may rotate or scale the virtual cultural relic resources in the cultural relic display window in response to the interactive operation (may be a sliding in all directions) of the virtual cultural relic resources in the cultural relic display window.
As shown in a service scenario 122 of fig. 3c, the computer device may respond to a triggering operation of the virtual cultural relic resource M in the shooting picture (may be clicking or long pressing the virtual cultural relic resource M in the shooting picture), and display a cultural relic display window R in the shooting picture, where the cultural relic display window R may be as shown in the service scenario 123, and the computer device may respond to an interaction operation of the virtual cultural relic resource M in the cultural relic display window R (may be sliding in all directions), and rotate or scale the virtual cultural relic resource M in the cultural relic display window R.
The specific process of the triggering operation of the response model collection control can be as follows: responding to the triggering operation of a model collection control for the virtual cultural relic resources in a shooting picture, displaying a cultural relic collection window in the shooting picture, and displaying the virtual cultural relic resources in the cultural relic collection window; responding to detailed operation of virtual cultural relic resources in the cultural relic collection window, and displaying historical information of the cultural relic for the virtual cultural relic resources; the historical information of the cultural relics is a word description of the cultural relics corresponding to the virtual cultural relic resources.
Specifically, when the shooting picture contains the virtual cultural relic resource, the computer device can respond to the triggering operation of the model collection control for the virtual cultural relic resource in the shooting picture, display a cultural relic collection window in the shooting picture, and display the virtual cultural relic resource in the cultural relic collection window. The computer device may display, in a presentation application, historical information about the virtual cultural relic resource in response to a detailed view operation for the virtual cultural relic resource in the cultural relic collection window, where the historical information may be a textual description of the cultural relic corresponding to the virtual cultural relic resource.
As shown in the service scenario 122 of fig. 3c, the computer device may respond to the triggering operation of the model collection control P, and display a cultural relic collection window T in the shot image, where the cultural relic collection window T may display a virtual cultural relic resource M (or may be an image of a cultural relic corresponding to the virtual cultural relic resource M) in the cultural relic collection window T as shown in the service scenario 124. The computer device may respond to the control S in the cultural relic collection window in the service scenario 124 (i.e., the detailed view operation for the virtual cultural relic resource M is triggered), and may display a text description about the cultural relic corresponding to the virtual cultural relic resource M in the service scenario 124, where the text description may be displayed in the window U.
It can be appreciated that, in the specific embodiment of the present application, the cultural relic resource to be navigated, the virtual cultural relic resource, the cultural relic resource after rendering, and the like may refer to a cultural relic model of the cultural relic resource, where the interaction (such as operations of broadcasting, shooting and saving, cultural relic displaying, and model collecting) on the virtual cultural relic resource may also be applied to a cultural relic in which the virtual cultural relic resource has been established.
Step S103, when the position relation between the shooting component and the target area is changed and the shooting picture continuously comprises the target area, the display style of the virtual cultural relic resource in the shooting picture is updated.
Specifically, when the position relationship (or azimuth relationship) between the shooting position (or the position where the shooting component is located) and the target area changes, and the shooting picture continuously contains the target area, the display style of the virtual cultural relic resource in the shooting picture is updated.
The specific process of determining the display style of the virtual cultural relic resource may be: when the distance relation between the shooting component and the target area is changed and the shooting picture continuously comprises the target area, changing the display size of the virtual cultural relic resource in the shooting picture; displaying the virtual cultural relic resources in the size change in a target area all the time; when the azimuth relation between the shooting component and the target area is changed and the shooting picture continuously comprises the target area, changing the display view angle of the virtual cultural relic resource in the shooting picture; virtual cultural relic resources in the display view angle change are always displayed in the target area.
Specifically, when the photographed image changes, that is, the photographed position moves, the computer device may redetermine the photographed view angle range and the model display range. When the photographed picture continues to contain the target area (the photographed view angle range overlaps with the model display range), the computer device may render the virtual cultural relic resource again (may be rendered again on the virtual cultural relic resource after the previous rendering process only through the changed view angle and distance), and update the display style (display view angle and display size) of the virtual cultural relic resource in the photographed picture.
Referring to fig. 2 again, as shown in fig. 2, when the shooting frame changes, that is, the shooting position moves from the service scene 103 to the service scene 104, the computer device may redetermine, based on the virtual cultural relic resource C in the service scene 103, the view angle and the distance corresponding to the shooting frame of the service scene 104 according to the shooting frame of the service scene 104, adjust the display size of the virtual cultural relic resource C through the parallax depth, adjust the display view angle of the virtual cultural relic resource C through the parallax angle, and update the display style (display view angle and display size) of the virtual cultural relic resource C on the target area of the service scene 104.
Optionally, the computer device may adjust information such as illumination, brightness, saturation, contrast, and the like, which are displayed by the virtual cultural relic resource in the target area according to the photosensitive information. The photosensitive information may be obtained by performing hue, saturation and brightness analysis on a picture of a shot picture.
When there is an obstacle occluding the virtual cultural relic resource, the computer device may display the virtual cultural relic resource outside of the virtual cultural relic resource covered by the occlusion area (the area occluded by the obstacle).
The specific process of displaying the virtual cultural relic resource with the obstacle shielding can be as follows: when an obstacle exists between the object and the target area, determining an occlusion area of the obstacle to the target area along a shooting view angle of the shooting assembly, and displaying non-occluded virtual cultural relic resources in the virtual cultural relic resources on the target area in a shooting picture; the virtual cultural relic resource which is not shielded is a virtual cultural relic resource except for the virtual cultural relic resource covered by the shielding area in the virtual cultural relic resource.
Specifically, the computer device may determine, according to the image area characteristics of the image, an obstacle between the virtual cultural relics in the image and the image capturing position, determine an occlusion area of the obstacle along the image capturing view angle range, and display a virtual cultural relic resource that is not occluded by the occlusion area on the target area in the image capturing image.
The image region features can be obtained by extracting the shot image features. The photographing view angle range may be a range in which a view angle corresponding to the photographing position is determined in the venue virtual space.
According to the method, the virtual cultural relic resource in the shooting picture changes the display style along with the movement of the user. Namely, when a user approaches, larger virtual cultural relic resources can be seen; when the user walks away, smaller virtual cultural relic resources can be seen; and when the virtual cultural relic resource is watched from different angles, the corresponding virtual cultural relic resource under the visual angle range is shot so as to simulate and realize the visual effect of the cultural relic resource during exhibition. The user can experience different distances through walking observation, the display modes and the display effects of the virtual relics under different visual angles, and the immersion sense and the atmosphere sense of the on-line relics are improved.
It can be appreciated that the computer device can determine the to-be-navigated cultural relic resource to which the user is going through the cultural relic navigation route, and when the to-be-navigated cultural relic resource is a virtual cultural relic resource, the computer device can pre-render the virtual cultural relic resource (or a plurality of virtual cultural relic resources) according to the estimated time required by the virtual cultural relic route to the cultural relic resource. When the computer equipment identifies the corresponding picture area characteristics in the shooting picture obtained by calling the shooting component, the pre-rendered virtual cultural relic resources can be immediately displayed in the target area of the shooting picture.
The computer device can also determine the virtual cultural relic resources (or a plurality of virtual cultural relic resources) to be rendered in advance through the cultural relic resources to be navigated determined by the cultural relic navigation route, and when a user follows the guidance of the virtual cultural relic route, the computer device can only perform feature recognition on the picture region features according to the site display information corresponding to the virtual cultural relic resources to be rendered in advance, namely, can be matched with the picture region features of the current shooting picture through a small amount of site display information so as to recognize whether the current shooting picture contains a target region or not, and further can improve the recognition efficiency of the target region.
According to the embodiment of the application, the cultural relics guiding operation in the display application is responded, and the shooting picture aiming at the target site and shot by the shooting component is displayed in the display application; the target site is an object site triggering the cultural relic navigation operation; when the shooting picture contains a target area for displaying virtual cultural relic resources in a target site, displaying the virtual cultural relic resources positioned on the target area in the shooting picture; the cultural relic type of the virtual cultural relic resource is associated with a target site, and the display position of the virtual cultural relic resource in a shooting picture is associated with the display position of a target area in the shooting picture; when the position relation between the shooting component and the target area changes and the shooting picture continuously contains the target area, the display style of the virtual cultural relic resource in the shooting picture is updated. Therefore, the related virtual cultural relic resource is displayed in the target area, and the virtual cultural relic resource can update the display style along with the change of the position relation between the shooting component and the target area so as to restore the experience of watching the cultural relic under the line, so that the display mode of the virtual cultural relic can be enriched, and the display effect of the virtual cultural relic can be improved.
Referring to fig. 4, fig. 4 is a flowchart of another data processing method provided in an embodiment of the present application, where the data processing method may be performed by a computer device, and the computer device may include the terminal device 100 or the platform server 200 shown in fig. 1, and as shown in fig. 4, the method may include:
step S201, acquiring point cloud materials of a target site, and constructing a site virtual space aiming at the target site according to the point cloud materials; the site virtual space has the same three-dimensional space information as the target site;
specifically, the computer device may establish a site virtual space of a target site for displaying the cultural relics through the high-precision point cloud data, and virtual cultural relic resources in the site virtual space (i.e., a cultural relic model established through cultural relic resources to be displayed on the target site, where the cultural relic resources may be cultural relics being displayed (i.e., entity cultural relics being displayed in the target scene), or cultural relics not being displayed due to repair, lending, etc. According to the exhibition position of the planned cultural relic resource on the target site, site exhibition position information of a virtual cultural relic resource corresponding to the cultural relic resource in a site virtual space can be determined, and the computer equipment can also determine key characteristics of the virtual cultural relic resource through textures, appearance characteristics and the like given by the cultural relic resource when the model is built. The application is not limited in the manner of creating the cultural relic model through the cultural relic resources.
The computer device may calibrate (set its spatial coordinates or set the spatial region) the business region of the target site in the site virtual space, so that the site virtual space has the same three-dimensional spatial information as the target site. The service area may include location information of the cultural relic resource (i.e., site display location information of the virtual cultural relic resource), and the service area may further include a main meeting place, an entrance, a road, an obstacle, and other spatial areas of the target site.
It can be understood that for a museum, many cultural relics can not be conveniently exhibited for a long time due to lending, and the like. Besides, a brand new interaction mode is provided for the combined exhibition among the museums, the borrowed cultural relics can be carried out no longer, virtual cultural relics resources corresponding to the cultural relics are authorized, and the reduction of the combined exhibition threshold of the museums is reduced.
Step S202, determining site display position information of one or more to-be-navigated relic resources in a target site;
Specifically, the computer device may determine the target site to which the user is going according to route navigation information (may include the position of the user, the position of the target site, and the traffic mode), or may acquire a shot picture of the user at the target site, perform region feature recognition on the shot picture in the target site, obtain a picture region feature, and determine the target site where the user is located by identifying the picture region feature.
The computer equipment can determine the site display position information of the cultural relic resource to be navigated in the target site through the calibration condition of the position information of the cultural relic resource in the site virtual space associated with the target site (or the target site to be navigated) where the user is located.
The calibration condition refers to calibration (setting the space coordinates or setting the space area) which can be performed by the computer equipment on the position information of the cultural relic resource of the target site in the site virtual space. The cultural relic resources to be navigated can comprise the cultural relic resources in display and the cultural relic resources not in display in the cultural relic resources, wherein the cultural relic resources not in display can be displayed in a shooting picture by creating corresponding virtual cultural relic resources. The computer device can also determine the cultural relic resource which is not browsed by the user as the cultural relic resource to be browsed by displaying the log information in the application (the cultural relic resource which is browsed by the user through the display application can be recorded).
Step S203, acquiring a shooting position of a shooting picture in a field virtual space;
specifically, the computer device may acquire a shot image and inertial measurement information of the target site, and may perform region feature recognition on the shot image in the target site to obtain a picture region feature, and determine a shot position of the user on the target site according to the inertial measurement information and the picture region feature.
The inertial measurement information can be measured by an IMU sensor (Inertial Measurement Unit, i.e., an inertial measurement unit) mounted thereon. The inertial measurement information may be the user's three-axis attitude angle (or angular rate) and acceleration.
The specific process of determining the shooting position may be: performing regional characteristic recognition on a shot picture to obtain picture regional characteristics, acquiring inertial measurement information of an object, and determining regional visual angle characteristics of the object on a target site based on the inertial measurement information and the picture regional characteristics; and carrying out global image retrieval in the field virtual space according to the regional visual angle characteristics, and determining the space position matched with the regional visual angle characteristics and retrieved in the field virtual space as the shooting position of the shooting picture in the field virtual space.
Specifically, the computer device can determine the regional visual angle characteristics of the user on the target site through the picture regional characteristics and the inertial measurement information, perform characteristic matching in the site virtual space of the target site according to the regional visual angle characteristics, and calculate to obtain the shooting position in the site virtual space when the shooting picture is shot.
The feature matching may be that an algorithm is improved by Netvlad (a vlad (vector of locally aggregated descriptors, graphic retrieval algorithm), and by combining with a deep convolutional neural network, an aggregation k-means of local feature descriptors may be changed into 1by 1 convolution, and global descriptors are output), global image retrieval is performed on a field virtual space corresponding to a target field by using a global descriptor output, feature point extraction is performed by using SuperPoint (a way of encoding and decoding by convolution, feature points and calculation descriptors may be extracted), and a shooting position associated with a shooting picture is matched.
It will be appreciated that the computer device may also determine the voice presentation associated with a key location (texture, appearance feature) in the target area where the virtual cultural relic resource has been displayed by identifying that key location.
Step S204, detecting the matching relation between the site display position information and the shooting position corresponding to each cultural relic resource to be navigated; if the field display position information matched with the shooting position exists, determining the to-be-navigated cultural relic resource corresponding to the field display position information matched with the field display position information as a virtual cultural relic resource, and determining that the shooting picture contains a target area for displaying the virtual cultural relic resource in a target field;
Specifically, the computer device may determine a matching relationship between the site display position information and the shooting position, determine a to-be-navigated cultural relic resource corresponding to the matched site display position information as a virtual cultural relic resource, and determine a target area corresponding to the virtual cultural relic resource.
The specific process of determining the matching relationship between the site display position information and the shooting position may be: acquiring a first coordinate range of site display position information corresponding to each cultural relic resource to be navigated, and acquiring a second coordinate range of a shooting position; if a first coordinate range which is overlapped with the second coordinate range is present, determining that the field display position information corresponding to the first coordinate range which is overlapped with the second coordinate range is matched with the shooting position; and determining a region overlapping with the region where the first coordinate range exists in the shooting picture as a target region.
Specifically, the computer device may detect a matching relationship between the site display position information and the shooting position corresponding to each virtual cultural relic resource in the site virtual space, and may determine a model display range (may be referred to as a first coordinate range) in the site virtual space through the site display position information and model information (may include a model size and a placement direction of the virtual cultural relic resource) of each virtual cultural relic resource, and determine a shooting view angle range (may be referred to as a second coordinate range) of the user in the site virtual space according to the shooting position. The computer device may determine a first coordinate range having a region overlapping with a second coordinate range, determine that the field presentation position information corresponding to the first coordinate range matches the photographing position, and determine a region of the region overlapping as the target region.
The computer device may display the virtual cultural relic resource after rendering processing on a target area of a photographed picture of the presentation application.
The specific process of displaying the virtual cultural relic resource after rendering treatment can be as follows: determining a parallax angle between the shooting position and the site display position information and a parallax depth between the shooting position and the site display position information based on the shooting position of the shooting picture and the site display position information of the virtual cultural relics resource; determining a display view angle of the virtual cultural relic resource in a shooting picture based on the parallax angle; determining the display size of the virtual cultural relic resource in a shooting picture based on the parallax depth; and acquiring material texture information and cultural relic model vertex coordinate information associated with the virtual cultural relic resources, and rendering the virtual cultural relic resources at a target area in a shooting picture according to a rendering mapping relation, a parallax angle and a parallax depth between the cultural relic model vertex coordinate information and the material texture information.
Specifically, the computer device may calculate the user position (shooting position) in real time according to the shot picture by using a map engine (capable of acquiring the positioning position of the user) and a visual algorithm engine (capable of processing the shot picture to obtain information such as the position, the size, the shape, and the like of the object in the picture) in combination with a GPU (Graphic Processing Unit, a graphics processor) detection service and a CPU (Central Processing Unit, a central processing unit) positioning engine, where the visual algorithm may be ranac PnP (a method for solving three-dimensional to two-dimensional point-to-point motion by calculating a transformation matrix).
The computer device may calculate a parallax angle and a parallax depth between the photographing position and the field presentation position information through a visual algorithm, the photographing position, and the field presentation position information of the virtual cultural relic resource. According to the visual angle, the display visual angle of the virtual cultural relic resource in the shooting picture is adjusted, and according to the visual depth, the display size of the virtual cultural relic resource in the shooting picture is adjusted, namely the display visual angle and the display size can be adjusted by rotating and zooming the virtual cultural relic resource.
The computer device may obtain texture information associated with the virtual cultural relic resource and vertex coordinate information of the cultural relic model, where the texture information may include information such as a map of texture, color, and the like. The cultural relic model vertex coordinate information may include UV coordinates of texture information, and a set of UV coordinates may be represented by two coordinates (e.g., u=0.0f, v=1.0f), where coordinates (0.0f ) may represent an upper left corner of a map corresponding to the texture information, and coordinates (1.0f ) may represent a lower right corner of the map corresponding to the texture information. The platform server 200 may assign the vertex coordinate information of the cultural relic model to each vertex in the vertex list (each vertex in the cultural relic model), and determine a rendering mapping relationship between the texture and each vertex.
The computer device may render virtual cultural relic resources at a target area in the photographed picture according to the rendering mapping, the parallax angle and the parallax depth.
Optionally, the computer device may adjust information such as illumination, brightness, saturation, contrast, and the like of the virtual cultural relic resource during rendering processing according to the photosensitive information. The photosensitive information can be obtained by hue, saturation and brightness analysis of pictures of a shot picture or can be obtained by measuring photosensitive elements on computer equipment.
The virtual cultural relic resource in the shooting picture can change in angle, direction and size along with the change of the shooting position so as to restore the experience of watching the cultural relic under the line.
Specifically, when the shooting picture changes, that is, the shooting position moves, the computer device may repeat the above steps to perform content interaction, and redetermine the shooting position and the target area. When the photographed picture continues to contain the target area, the computer device may render the virtual cultural relic resource again (may be rendered again on the virtual cultural relic resource after the previous rendering process only by the parallax angle and the parallax depth), and update the display style (display view angle and display size) of the virtual cultural relic resource in the photographed picture.
Referring to fig. 2 again, as shown in fig. 2, when the shooting frame changes, that is, the shooting position moves from the service scene 103 to the service scene 104, the computer device may redetermine, based on the virtual cultural relic resource C in the service scene 103, a parallax angle and a parallax depth corresponding to the shooting frame of the service scene 104 according to the shooting frame of the service scene 104, adjust the display size of the virtual cultural relic resource C through the parallax depth, adjust the display view angle of the virtual cultural relic resource C through the parallax angle, and update the display style (display view angle and display size) of the virtual cultural relic resource C on the target area of the service scene 104.
Referring to fig. 5, fig. 5 is a flowchart illustrating another data processing method according to an embodiment of the application. As shown in fig. 5, the data processing method may be implemented through interaction between the terminal device 100 and the platform server 200.
The specific process of pre-establishing the three-dimensional virtual cultural relic resource can be as follows: the platform server 200 may collect high-precision point cloud data (a point data set of an appearance surface of an article obtained by a measuring instrument) of a target site (may be a museum, a memorial hall, a cultural relic exhibition, etc.) through a point cloud and a laser device in advance, or collect high-precision point cloud data through photo materials of the target site.
The platform server 200 may establish the site virtual space of the target site through the high-precision point cloud data, and virtual cultural relic resources in the site virtual space (i.e., a cultural relic model established through cultural relic resources to be displayed on the target site), where the cultural relic resources may be cultural relics being displayed (i.e., entity cultural relics being displayed in the target scene), or cultural relics not being displayed for repair, lending, etc. According to the exhibition position of the planned cultural relic resource on the target site, site exhibition position information of the virtual cultural relic resource corresponding to the cultural relic resource in the site virtual space can be determined.
The platform server 200 may make a calibration (set its spatial coordinates or set a spatial region) for the service area of the target site in the site virtual space, so that the site virtual space has the same three-dimensional spatial information as the target site. The service area may include location information of the cultural relic resource (i.e., site display location information of the virtual cultural relic resource), and the service area may further include a main meeting place, an entrance, a road, an obstacle, and other spatial areas of the target site.
Specifically, the terminal device 100 may trigger a cultural relic navigation operation in the exhibition application (may be to turn on an AR camera), start an AR camera function in the exhibition application, call a shooting component (may be a camera) of the terminal device 100, obtain a shooting picture of the target site (may be a picture seen by a user through the shooting component) through shooting by the shooting component, and display the shooting picture of the target site in real time in the exhibition application.
The terminal device 100 may transmit the target site photographing screen and inertial measurement information (may be a three-axis attitude angle (or angular rate) and acceleration) obtained by measurement of an inertial measurement unit (may be an IMU sensor mounted on the terminal device 100) to the platform server 200.
The platform server 200 acquires the photographed image and the inertial measurement information of the target site, and can perform region feature recognition on the photographed image in the target site to obtain image region features (which may also be referred to as environmental features, and extraction of the image region features may also be performed in the terminal device 100), and determine the target site where the user is located by recognizing the image region features. (this process may also be referred to as reconstructing the camera scene (shot).
When the user moves to a location of the cultural relic resource that is not being exhibited (i.e., the location of the virtual cultural relic resource in the site exhibiting location is near the location of the virtual cultural relic resource), the platform server 200 may determine the regional viewing angle characteristic of the user in the target site through the picture regional characteristic and the inertial measurement information, and the platform server 200 may perform feature matching (may also be referred to as target content matching) in the site virtual space of the target site according to the regional viewing angle characteristic, and the specific process of feature matching may refer to the description of step S204 in the embodiment of fig. 4, which is not repeated herein. The specific process of shooting the view angle range may also be called calibrating the position orientation of a three-dimensional space (field virtual space) camera.
The specific process of constructing the interaction between the virtual and the real content can be as follows: when the angle, the direction and the size of the virtual cultural relic resource are changed along with the change of the shooting position, the content interaction between the terminal equipment 100 and the platform server 200 is performed so as to restore the experience of watching the cultural relic offline.
The terminal device 100 displays the virtual cultural relic resource after rendering processing (presents the three-dimensional virtual cultural relic resource) on the target area of the photographing screen of the presentation application.
The specific process of constructing the virtual-real content interaction and presenting the three-dimensional virtual cultural relic resource may refer to the description of step S204 in the embodiment of fig. 4, and will not be repeated here.
When the photographed picture changes, that is, the user adjusts the movement, the terminal device 100 and the platform server 200 may repeat the above steps to perform the content interaction, and re-determine the photographed position and the target area. When the photographed picture continues to contain the target area, the platform server 200 may render the virtual cultural relic resource again (may be rendered again only by the parallax angle and the parallax depth on the virtual cultural relic resource after the last rendering process), and update the presentation style (presentation view angle and presentation size) of the virtual cultural relic resource in the photographed picture on the terminal device 100. According to the method, the virtual cultural relic resource in the shooting picture changes the display style along with the movement of the user. Namely, when a user approaches, larger virtual cultural relic resources can be seen; when the user walks away, smaller virtual cultural relic resources can be seen; and when the virtual cultural relic resource is watched from different angles, the corresponding virtual cultural relic resource under the visual angle range is shot so as to simulate and realize the visual effect of the cultural relic resource during exhibition. The user can experience different distances through walking observation, the display modes and the display effects of the virtual relics under different visual angles, and the immersion sense and the atmosphere sense of the on-line relics are improved.
Referring to fig. 6, fig. 6 is a schematic diagram of a virtual cultural relic resource exhibition architecture according to an embodiment of the present application. As shown in fig. 6, the virtual relic resource exhibition architecture can include a data initialization component, a point cloud data storage component, an AR engine component, a resource service component, a relic model exhibition component, and a virtual space construction component. The above components may also be integrated into one server, i.e., the platform server 200 in the network architecture diagram of fig. 1. The relic offline restoration component may also be the terminal device 100 in the network architecture diagram of fig. 1.
Specifically, the data initializing component can acquire high-precision point cloud data by collecting field photo materials of a set target field (such as a museum, a commemorative museum and a cultural relic exhibition), or acquire high-precision point cloud data of the target field (such as the museum, the commemorative museum and the cultural relic exhibition) through point cloud and laser equipment (a point data set of an article appearance surface obtained through a measuring instrument), further data initialize the high-precision point cloud data to obtain basic point cloud data, perform point cloud data composition through the basic point cloud data, and establish a field virtual space of the target field.
The process of the point cloud data composition may refer to building a site virtual space of the target site according to the basic point cloud data, and virtual cultural relic resources in the site virtual space (i.e., a cultural relic model built by cultural relic resources to be displayed on the target site, where the cultural relic resources may be cultural relics being displayed (i.e., entity cultural relics displayed in the target scene), or cultural relics not being displayed due to repair, lending, etc.). According to the exhibition position of the planned cultural relic resource in the target site, site exhibition position information of the virtual cultural relic resource corresponding to the cultural relic resource in the site virtual space can be determined, and the service area of the target site is calibrated (space coordinates or space areas are set) in the site virtual space through data initialized labeling, so that the site virtual space has the same three-dimensional space information as the target site. The specific function of the data initializing component may be described with reference to step S201 in the embodiment of fig. 4, and will not be described herein.
The point cloud data storage component may be used to store point cloud data (i.e., including GPU service dependent data and CPU service dependent data in fig. 6) that is relied upon by the GPU (Graphic Processing Unit, graphics processor) detection service and CPU ((Central Processing Unit, central processor)) positioning service, which may come from the data initialization component.
The AR engine component may include functional modules of pose detection service, vision algorithm, map engine, GPU detection service, CPU positioning engine, and parallax computation.
Wherein the map engine may be used to determine venue navigation routes, virtual cultural relics routes, and cultural relics navigation routes. Specific functions of the map engine may be described with reference to step S101 in the embodiment of fig. 3a, and will not be described here again.
Wherein the pose detection service can determine inertial measurement information of the user through the inertial measurement component.
The AR engine component calculates regional visual angle characteristics of the shot picture through inertial measurement information obtained by pose detection service and picture regional characteristics obtained by extracting features of the shot picture, and determines a matching relation between site display position information and shot positions according to the regional visual angle characteristics.
The visual algorithm, the GPU detection service, the CPU positioning engine and the parallax calculation function modules can be used for determining rendering information when virtual cultural relics resources are rendered and processed. The AR engine component determines a parallax angle and a parallax depth between the shooting position and the field display position information through a visual algorithm, a GPU detection service and a CPU positioning engine, and determines a display view angle and a display size of the virtual cultural relic resource on a shooting picture according to the parallax angle and the parallax depth through parallax calculation.
The specific functions of the visual algorithm, the GPU detection service, the CPU positioning engine and the parallax computation may be referred to the description of step S204 in the embodiment of fig. 4, and will not be described here again.
The AR engine component can also adjust illumination, brightness, saturation and contrast according to the photosensitive information. The photosensitive information may be obtained by performing hue, saturation and brightness analysis on a picture of a shot picture. According to the picture area characteristics of the shooting picture, the obstacle between the virtual cultural relics in the shooting picture and the shooting position can be determined, the shielding area of the obstacle is determined along the shooting visual angle range, and the virtual cultural relic resources which are not shielded by the shielding area are displayed on the target area in the shooting picture.
The resource service component can be configured to store a site virtual space for each targeted service area.
The method comprises the steps of determining the site display position information of the virtual cultural relic resource corresponding to the cultural relic resource in a site virtual space according to the position information of the cultural relic resource in a target site.
The cultural relic model may refer to virtual cultural relic resources and model information stored in a site virtual space in a resource service component. The model information may include a model size, a placement direction of the virtual cultural relic resource.
The virtual space construction component can conduct region feature recognition on a shot picture in the target field to obtain picture region features, and the region visual angle features of the user in the target field are determined through the picture region features and the inertia measurement information. And performing feature matching in the virtual space of the target field according to the regional visual angle features, and calculating to obtain the shooting position in the virtual space of the field when shooting the shooting picture.
The real environment detection may be to detect a matching relationship between the site display position information and the shooting position corresponding to each virtual cultural relic resource in the site virtual space, so as to determine whether a target area exists in the current shooting picture. The specific function of the real environment detection may be referred to the description of step S203 in the embodiment of fig. 4, and will not be described here again.
The process of detecting the cultural relics in the cultural relic model display assembly may refer to determining a virtual cultural relic resource corresponding to a shooting position according to a matching relationship between site display position information and the shooting position.
The AR environment fusion process may refer to determining an obstacle between a virtual cultural relic and a shooting position in a shooting picture according to a picture area feature of the shooting picture, determining an occlusion area of the obstacle along a shooting view angle range, and displaying a virtual cultural relic resource which is not occluded by the occlusion area on a target area in the shooting picture. For specific functions of the AR environment fusion module, please refer to the description of step S103 in the embodiment of fig. 3a, and no further description is given here.
The AR rendering may refer to determining a rendering mapping relationship of texture information in a virtual cultural relic resource according to vertex coordinate information of a cultural relic model, and rendering the virtual cultural relic resource at a target area in a shooting picture according to the rendering mapping relationship, a parallax angle and a parallax depth. For specific functions of the AR rendering service, please refer to the description of step S204 in the embodiment of fig. 4, and details are not repeated here.
The AR interaction may refer to content interaction performed when the photographing position moves, and the photographing position and the target area are redetermined. When the shooting picture continues to contain the target area, the virtual space construction component can render the virtual cultural relic resource again (can be the virtual cultural relic resource rendered last time and only rendered through the parallax angle and the parallax depth), and the display style (display view angle and display size) of the virtual cultural relic resource in the shooting picture is updated on the cultural relic offline reduction component. For specific functions of the AR interaction module, please refer to the description of step S103 in the embodiment of fig. 3a, and no further description is given here.
The relic offline reduction component can display the virtual relic resources after rendering processing on a target area of a shooting picture of the display application.
The cultural relic display may refer to that a positional relationship (or an azimuth relationship) between a shooting position (or a position where a shooting component is located) and a target area changes, and when the shooting picture continuously includes the target area, a display style of the virtual cultural relic resource in the shooting picture is updated. The specific function of the cultural relic display module may be referred to the description of step S103 in the embodiment of fig. 3a, and will not be described herein.
The virtual environment rendering module can be used for adjusting information such as illumination, brightness, saturation, contrast and the like displayed by the virtual cultural relic resources in the target area according to the photosensitive information. The photosensitive information may be obtained by performing hue, saturation and brightness analysis on a picture of a shot picture. For specific functions of the virtual environment rendering module, please refer to the description of step S103 in the embodiment of fig. 3a, and no further description is given here.
According to the method, the virtual cultural relic resource in the shooting picture changes the display style along with the movement of the user. Namely, when a user approaches, larger virtual cultural relic resources can be seen; when the user walks away, smaller virtual cultural relic resources can be seen; and when the virtual cultural relic resource is watched from different angles, the corresponding virtual cultural relic resource under the visual angle range is shot so as to simulate and realize the visual effect of the cultural relic resource during exhibition. The user can experience different distances through walking observation, the display modes and the display effects of the virtual relics under different visual angles, and the immersion sense and the atmosphere sense of the on-line relics are improved.
Referring to fig. 7, fig. 7 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present application. As shown in fig. 7, the data processing apparatus 1 may be a computer device in the embodiment corresponding to fig. 3a or fig. 4, where the computer device includes a navigation module 300, a virtual map module 400, a broadcasting guide module 500, a cultural relic guiding module 600, a cultural relic display module 700, a cultural relic update module 800, a shooting and saving module 900, a cultural relic interaction module 1000, a cultural relic collection module 1100, a site virtual space module 1200, a processing module 1300, a detection module 1400, a voice prompt module 1500, a voice cultural relic introduction module 1600, and a cultural relic location introduction module 1700. The cultural relic display module 700 comprises a model parallax unit 701 and a model rendering unit 702. The cultural relic updating module 800 includes a cultural relic size updating unit 801 and a cultural relic viewing angle updating unit 802. The processing module 1300 includes a regional view feature unit 1301 and a global image retrieval unit 1302. The detection module includes a coordinate range acquisition unit 1401 and a coordinate range matching unit 1402.
The navigation module 300 is configured to display object location information, site location information, and route navigation information between the object location information and the site location information on an electronic map in the display application in response to a navigation operation for a target site in the display application; the object position information is used for indicating the position of an object triggering the navigation operation; the site location information is used for indicating the location of the target site; the route navigation information is used to guide the object to a target site for the display relics. The specific functional implementation of the navigation module 300 may be referred to the description of step S101 in the corresponding embodiment of fig. 3a, and will not be described herein.
The virtual map module 400 is configured to display a virtual field map for a target field in a shooting picture, and display field display position information corresponding to one or more to-be-navigated cultural relic resources in the virtual field map; the site display position information is used for representing the position of a to-be-navigated cultural relic resource in a target site, and the one or more to-be-navigated cultural relic resources comprise virtual cultural relic resources; if the shooting picture also comprises the virtual cultural relic resource, the display mode of the site display position information of the virtual cultural relic resource in the virtual site map is different from the display mode of the site display position information of the residual cultural relic resource to be guided in the virtual site map, and the residual cultural relic resource to be guided comprises one or more cultural relic resources to be guided except the virtual cultural relic resource in the cultural relic resource to be guided. The specific functional implementation of the virtual map module 400 may be referred to the description of step S101 in the corresponding embodiment of fig. 3a, and will not be described herein.
The guiding and broadcasting module 500 is used for displaying a virtual cultural relic route from an object to a target area in a shooting picture and displaying guiding and broadcasting controls aiming at the virtual cultural relic route; and responding to the triggering operation of the guide control in the shooting picture, and playing the voice cultural relic introduction aiming at the virtual cultural relic resource. The specific functional implementation of the multicast module 500 may be referred to the description of step S101 and step S102 in the corresponding embodiment of fig. 3a, and will not be described herein.
The cultural relic guiding module 600 is used for responding to cultural relic guiding operation in the display application, and displaying a shooting picture aiming at a target site, shot by the shooting component in the display application; the target site is an object site triggering the cultural relic navigation operation; the specific functional implementation of the cultural relic guiding module 600 may refer to the description of step S101 in the corresponding embodiment of fig. 3a, and will not be described herein.
The cultural relic display module 700 is configured to display a virtual cultural relic resource located on a target area in a photographed picture when the photographed picture includes the target area for displaying the virtual cultural relic resource in the target area; the cultural relic type of the virtual cultural relic resource is associated with a target site, and the display position of the virtual cultural relic resource in a shooting picture is associated with the display position of a target area in the shooting picture;
the cultural relic display module 700 is specifically configured to determine, when an obstacle exists between an object and a target area, a blocking area of the obstacle to the target area along a shooting view angle of the camera assembly, and display, in the shooting picture, an unblocked virtual cultural relic resource in the virtual cultural relic resources on the target area; the virtual cultural relic resource which is not shielded is a virtual cultural relic resource except for the virtual cultural relic resource covered by the shielding area in the virtual cultural relic resource. The specific functional implementation of the cultural relic display module 700 may refer to the description of step S102 in the corresponding embodiment of fig. 3a, and will not be described herein.
A model parallax unit 701, configured to determine a parallax angle between a shooting position and field presentation position information, and a parallax depth between the shooting position and the field presentation position information, based on the shooting position of the shooting picture and the field presentation position information of the virtual cultural relics resource; the model parallax unit 701 is further configured to determine a display view angle of the virtual cultural relic resource in the photographed picture based on the parallax angle; determining the display size of the virtual cultural relic resource in a shooting picture based on the parallax depth;
the model rendering unit 702 is configured to obtain texture information of materials associated with the virtual cultural relic resource and vertex coordinate information of the cultural relic model, and render the virtual cultural relic resource at a target area in the photographed image according to a rendering mapping relationship, a parallax angle and a parallax depth between the vertex coordinate information of the cultural relic model and the texture information of the materials.
The cultural relic updating module 800 is configured to update a display view angle of the virtual cultural relic resource in the photographing picture when a positional relationship between the photographing component and the target area changes and the photographing picture continues to include the target area. The specific functional implementation of the cultural relic update module 800 may refer to the description of step S103 in the corresponding embodiment of fig. 3a, and will not be described herein.
The cultural relic size updating unit 801 is configured to change a display size of a virtual cultural relic resource in a shooting picture when a distance relationship between the shooting component and a target area changes and the shooting picture continuously includes the target area; displaying the virtual cultural relic resources in the size change in a target area all the time; the specific functional implementation of the cultural relic size updating unit 801 may refer to the description of step S103 in the corresponding embodiment of fig. 3a, and will not be described herein.
The cultural relic visual angle updating unit 802 is configured to change a display visual angle of the virtual cultural relic resource in the shooting picture when the azimuth relationship between the shooting component and the target area changes and the shooting picture continuously includes the target area; virtual cultural relic resources in the display view angle change are always displayed in the target area. The specific functional implementation of the cultural relic view angle updating unit 802 may refer to the description of step S103 in the corresponding embodiment of fig. 3a, and will not be described herein.
The shooting and saving module 900 is configured to respond to a triggering operation of a shooting control in the exhibition application when the shooting picture contains a virtual cultural relic resource, and save the cultural relic shooting picture for the virtual cultural relic resource shot by the shooting component in the exhibition application; the cultural relic shooting picture comprises a target site and virtual cultural relic resources. The specific functional implementation of the shooting saving module 900 may refer to the description of step S102 in the corresponding embodiment of fig. 3a, and will not be described herein.
The cultural relic interaction module 1000 is used for responding to the triggering operation aiming at the virtual cultural relic resources in the shooting picture, displaying a cultural relic display window in the shooting picture and displaying the virtual cultural relic resources in the cultural relic display window; and responding to the interactive operation aiming at the virtual cultural relic resources in the cultural relic display window, and performing rotation or scaling operation on the virtual cultural relic resources in the cultural relic display window. The specific functional implementation of the cultural relic interaction module 1000 may refer to the description of step S102 in the corresponding embodiment of fig. 3a, and will not be described herein.
The cultural relic collection module 1100 is configured to respond to a triggering operation of a model collection control for a virtual cultural relic resource in a shooting picture, display a cultural relic collection window in the shooting picture, and display the virtual cultural relic resource in the cultural relic collection window; responding to detailed operation of virtual cultural relic resources in the cultural relic collection window, and displaying historical information of the cultural relic for the virtual cultural relic resources; the historical information of the cultural relics is a word description of the cultural relics corresponding to the virtual cultural relic resources. The specific functional implementation of the cultural relic collection module 1100 may refer to the description of step S102 in the corresponding embodiment of fig. 3a, and will not be described herein.
The voice prompt module 1500 is configured to play a voice prompt for prompting the object to browse the virtual cultural relic resource when the photographed image does not include the target area in the target site and the distance between the position of the object in the target site and the target area is less than the distance threshold; the voice prompts include location navigation prompts that are determined based on a positional relationship between the location of the object in the target site and the target area. The specific functional implementation of the voice prompt module 1500 may refer to the description of step S101 in the corresponding embodiment of fig. 3a, and will not be described herein.
The voice cultural relic introduction module 1600 is used for playing voice cultural relic introduction aiming at the virtual cultural relic resource when the shooting picture contains the virtual cultural relic resource; the voice cultural relic introduction module 1600 is further configured to pause voice cultural relic introduction for a virtual cultural relic resource when a positional relationship between a position of an object in a target field and a target area changes and a shooting picture no longer includes the target area, and switch and play a voice cultural relic navigation for a cultural relic resource to be navigated next; the next cultural relic resource to be browsed refers to a cultural relic resource which is adjacent to the virtual cultural relic resource in the target site and is not browsed by the object. The specific functional implementation of the voice cultural relic introduction module 1600 may refer to the description of step S101 in the corresponding embodiment of fig. 3a, and will not be described herein.
The cultural relic part introduction module 1700 is used for playing the voice cultural relic introduction aiming at the virtual cultural relic resource, which is associated with the virtual unique identifier, when the photographed picture contains the virtual cultural relic resource and the proportion of the virtual unique identifier in the virtual cultural relic resource in the photographed picture is greater than or equal to the proportion threshold value; the relic part introduction module 1700 is further configured to switch the played voice relic introduction to the voice introduction content for the key part when the duration of the key part in the virtual relic resource included in the shot frame is equal to the duration threshold in the process of playing the voice relic introduction. The specific functional implementation of the relic part introduction module 1700 may refer to the description of step S101 in the corresponding embodiment of fig. 3a, and will not be described herein.
The site virtual space module 1200 is configured to obtain point cloud materials of a target site, and construct a site virtual space for the target site according to the point cloud materials; the site virtual space has the same three-dimensional space information as the target site; the specific functional implementation of the site virtual space module 1200 may be referred to the description of step S201 in the corresponding embodiment of fig. 4, and will not be described herein.
A processing module 1300, configured to determine site display position information of one or more to-be-navigated relic resources in a target site; acquiring a shooting position of a shooting picture in a field virtual space; the specific functional implementation of the processing module 1300 may refer to the descriptions of step S202 and step S203 in the corresponding embodiment of fig. 4, which are not described herein.
The regional visual angle feature unit 1301 is configured to perform regional feature recognition on a shot picture to obtain a picture regional feature, acquire inertial measurement information of an object, and determine the regional visual angle feature of the object on a target site based on the inertial measurement information and the picture regional feature; the specific functional implementation of the region view feature unit 1301 may refer to the description of step S203 in the corresponding embodiment of fig. 4, and will not be described herein.
The global image retrieval unit 1302 is configured to perform global image retrieval in the venue virtual space according to the regional view feature, and determine a spatial position matching the regional view feature retrieved in the venue virtual space as a photographing position of the photographing screen in the venue virtual space. The specific functional implementation of the global image retrieving unit 1302 may refer to the description of step S203 in the corresponding embodiment of fig. 4, and will not be described herein.
The detection module 1400 is configured to detect a matching relationship between the site display position information and the shooting position corresponding to each of the cultural relics to be navigated; if the field display position information matched with the shooting position exists, determining the to-be-navigated cultural relic resource corresponding to the field display position information matched with the field display position information as a virtual cultural relic resource, and determining that the shooting picture contains a target area for displaying the virtual cultural relic resource in the target field. The specific functional implementation of the detection module 1400 may refer to the description of step S204 in the corresponding embodiment of fig. 4, and will not be described herein.
A coordinate range obtaining unit 1401, configured to obtain a first coordinate range of the site display position information corresponding to each to-be-navigated cultural relic resource, and obtain a second coordinate range of the shooting position; the specific functional implementation of the coordinate range obtaining unit 1401 may refer to the description of step S204 in the corresponding embodiment of fig. 4, and will not be described here again.
A coordinate range matching unit 1402, configured to determine that the field display position information corresponding to the first coordinate range overlapping the second coordinate range matches the photographing position if there is the first coordinate range overlapping the second coordinate range; the target range matching unit 1402 is specifically configured to determine, as a target region, a region overlapping with a region where the first coordinate range exists in the photographed image. The specific functional implementation of the coordinate range matching unit 1402 may refer to the description of step S204 in the corresponding embodiment of fig. 4, and will not be described herein.
Further, referring to fig. 8, fig. 8 is a schematic structural diagram of a computer device according to an embodiment of the present application, where the computer device may be a terminal device or a server (i.e., a terminal device and a management server). As shown in fig. 8, the computer device 2000 may include: processor 1001, network interface 1004, and memory 1005, and in addition, the above-described computer device 2000 may further include: a user interface 1003, and at least one communication bus 1002. Wherein the communication bus 1002 is used to enable connected communication between these components. In some embodiments, the user interface 1003 may include a Display (Display), a Keyboard (Keyboard), and the optional user interface 1003 may further include a standard wired interface, a wireless interface, among others. Alternatively, the network interface 1004 may include a standard wired interface, a wireless interface (e.g., WI-FI interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory (non-volatile memory), such as at least one disk memory. Optionally, the memory 1005 may also be at least one memory device located remotely from the aforementioned processor 1001. As shown in fig. 8, an operating system, a network communication module, a user interface module, and a device control application program may be included in the memory 1005, which is one type of readable storage medium.
In the computer device 2000 illustrated in fig. 8, the network interface 1004 may provide network communication functions; while user interface 1003 is primarily used as an interface for providing input to a user; and the processor 1001 may be used to invoke a device control application stored in the memory 1005 to implement:
responding to the cultural relic navigation operation in the display application, and displaying a shooting picture aiming at a target site and shot by a shooting component in the display application; the target site is an object site triggering the cultural relic navigation operation; when the shooting picture contains a target area for displaying virtual cultural relic resources in a target site, displaying the virtual cultural relic resources positioned on the target area in the shooting picture; the cultural relic type of the virtual cultural relic resource is associated with a target site, and the display position of the virtual cultural relic resource in a shooting picture is associated with the display position of a target area in the shooting picture; when the position relation between the shooting component and the target area changes and the shooting picture continuously contains the target area, the display style of the virtual cultural relic resource in the shooting picture is updated.
It should be understood that the computer device 2000 described in the embodiment of the present application may perform the description of the data processing method in the embodiment corresponding to fig. 3a or fig. 4, and may also perform the description of the data processing apparatus 1 in the embodiment corresponding to fig. 7, which is not described herein. In addition, the description of the beneficial effects of the same method is omitted.
Furthermore, it should be noted here that: the embodiment of the present application further provides a readable storage medium, in which the computer program executed by the aforementioned data processing apparatus 1 is stored, and when the processor executes the computer program, the description of the data processing method in the embodiment corresponding to fig. 3a or fig. 4 can be executed, and therefore, a detailed description will not be given here. In addition, the description of the beneficial effects of the same method is omitted. For technical details not disclosed in the readable storage medium embodiments related to the present application, please refer to the description of the method embodiments of the present application.
In addition, it should be noted that: embodiments of the present application also provide a computer program product, which may include a computer program, which may be stored in a readable storage medium. The processor of the computer device reads the computer program from the readable storage medium, and the processor may execute the computer program, so that the computer device performs the description of the data processing method in the embodiment corresponding to fig. 3a or fig. 4, and thus, a detailed description will not be given here. In addition, the description of the beneficial effects of the same method is omitted. For technical details not disclosed in the embodiments of the computer program product according to the present application, reference is made to the description of the method embodiments of the present application.
The terms first, second and the like in the description and in the claims and drawings of embodiments of the application are used for distinguishing between different objects and not for describing a particular sequential order. Furthermore, the term "include" and any variations thereof is intended to cover a non-exclusive inclusion. For example, a process, method, apparatus, article, or device that comprises a list of steps or elements is not limited to the list of steps or modules but may, in the alternative, include other steps or modules not listed or inherent to such process, method, apparatus, article, or device.
Those skilled in the art will appreciate that implementing all or part of the above-described embodiment methods may be accomplished by way of a computer program, which may be stored on a computer readable storage medium and which, when executed, may comprise the steps of embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), or the like.
The foregoing disclosure is illustrative of the present application and is not to be construed as limiting the scope of the application, which is defined by the appended claims.
Claims (20)
1. A method of data processing, comprising:
responding to a cultural relic guiding operation in a display application, and displaying a shooting picture aiming at a target site and shot by a shooting component in the display application; the target site is an object site triggering the cultural relic navigation operation;
when the shooting picture contains a target area for displaying virtual cultural relic resources in the target site, displaying the virtual cultural relic resources positioned on the target area in the shooting picture; the cultural relic type of the virtual cultural relic resource is associated with the target site, and the display position of the virtual cultural relic resource in the shooting picture is associated with the display position of the target area in the shooting picture;
and when the position relation between the shooting component and the target area is changed and the shooting picture continuously comprises the target area, updating the display style of the virtual cultural relic resource in the shooting picture.
2. The method as recited in claim 1, further comprising:
responding to the navigation operation aiming at the target site in the display application, and displaying object position information, site position information and route navigation information between the object position information and the site position information on an electronic map in the display application; the object position information is used for indicating the position of an object triggering the navigation operation; the site location information is used for indicating the location of the target site; the route navigation information is used to guide an object to the target site for the display relics.
3. The method as recited in claim 1, further comprising:
when the shooting picture contains the virtual cultural relic resource, responding to the triggering operation of a shooting control in the display application, and storing the cultural relic shooting picture aiming at the virtual cultural relic resource, shot by the shooting component, in the display application; the cultural relic shooting picture comprises the target site and the virtual cultural relic resource.
4. The method as recited in claim 1, further comprising:
displaying a virtual cultural relic route from the object to the target area in the shooting picture, and displaying a guide control for the virtual cultural relic route;
and responding to the triggering operation of the guide control in the shooting picture, and playing the voice cultural relic introduction aiming at the virtual cultural relic resource.
5. The method as recited in claim 1, further comprising:
displaying a virtual field map aiming at the target field in the shooting picture, and displaying field display position information corresponding to one or more cultural relics to be guided respectively in the virtual field map; the site display position information is used for representing the position of one cultural relic resource to be navigated in the target site, and the one or more cultural relic resources to be navigated comprise the virtual cultural relic resource;
And if the shooting picture also comprises the virtual cultural relic resource, displaying the site display position information of the virtual cultural relic resource in the virtual site map in a different manner from displaying the site display position information of the residual cultural relic resource to be guided in the virtual site map, wherein the residual cultural relic resource to be guided comprises the cultural relic resource to be guided except the virtual cultural relic resource in the one or more cultural relic resources to be guided.
6. The method as recited in claim 1, further comprising:
responding to triggering operation aiming at the virtual cultural relic resources in the shooting picture, displaying a cultural relic display window in the shooting picture, and displaying the virtual cultural relic resources in the cultural relic display window;
responding to the interactive operation aiming at the virtual cultural relic resources in the cultural relic display window, and performing rotation or scaling operation on the virtual cultural relic resources in the cultural relic display window.
7. The method of claim 1, wherein displaying the virtual cultural relic resource on the target area in the photographed picture comprises:
when an obstacle exists between the object and the target area, determining an occlusion area of the obstacle to the target area along a shooting view angle of a shooting assembly, and displaying non-occluded virtual cultural relic resources in virtual cultural relic resources on the target area in the shooting picture; the non-occluded virtual cultural relic resource is a virtual cultural relic resource except for the virtual cultural relic resource covered by the occluded area in the virtual cultural relic resource.
8. The method as recited in claim 1, further comprising:
responding to the triggering operation of a model collection control for the virtual cultural relic resource in the shooting picture, displaying a cultural relic collection window in the shooting picture, and displaying the virtual cultural relic resource in the cultural relic collection window;
responding to detailed browsing operation of the virtual cultural relic resources in the cultural relic collection window, and displaying historical cultural relic information of the virtual cultural relic resources; the historical information of the cultural relics is a word description of the cultural relics corresponding to the virtual cultural relic resources.
9. The method of claim 1, wherein the positional relationship comprises a distance relationship and an azimuth relationship, and the display style comprises a display size and a display viewing angle; when the position relationship between the shooting component and the target area changes and the shooting picture continuously contains the target area, updating the display style of the virtual cultural relic resource in the shooting picture comprises the following steps:
when the distance relation between the shooting component and the target area is changed and the shooting picture continuously comprises the target area, changing the display size of the virtual cultural relic resource in the shooting picture; the virtual cultural relic resources in the display size change are always displayed in the target area;
When the azimuth relation between the shooting component and the target area is changed and the shooting picture continuously comprises the target area, changing the display view angle of the virtual cultural relic resource in the shooting picture; the virtual cultural relic resources in the display view angle change are always displayed in the target area.
10. The method as recited in claim 1, further comprising:
when the shooting picture does not contain a target area in the target field and the distance between the position of the object in the target field and the target area is smaller than a distance threshold value, playing a voice prompt for prompting the object to browse the virtual cultural relic resource; the voice prompts include a location navigation prompt that is determined based on a positional relationship between the location of the object in the target site and the target area.
11. The method as recited in claim 1, further comprising:
when the shooting picture contains the virtual cultural relic resource, playing a voice cultural relic introduction aiming at the virtual cultural relic resource;
when the position relation between the position of the object in the target field and the target area is changed and the shooting picture does not contain the target area any more, suspending the introduction of the voice cultural relics aiming at the virtual cultural relic resource, and switching and playing the voice cultural relic navigation of the next cultural relic resource to be navigated; the next cultural relic resource to be browsed refers to a cultural relic resource which is adjacent to the virtual cultural relic resource in the target site and is not browsed by the object.
12. The method as recited in claim 1, further comprising:
when the shooting picture contains the virtual cultural relic resource and the proportion of the virtual unique identifier in the virtual cultural relic resource in the shooting picture is larger than or equal to a proportion threshold value, playing the voice cultural relic introduction aiming at the virtual cultural relic resource and associated with the virtual unique identifier;
and when the duration of the key part in the virtual cultural relic resource is equal to a duration threshold in the process of playing the voice cultural relic introduction, switching the played voice cultural relic introduction to voice introduction contents aiming at the key part.
13. The method as recited in claim 1, further comprising:
acquiring point cloud materials of the target site, and constructing a site virtual space aiming at the target site according to the point cloud materials; the site virtual space has the same three-dimensional space information as the target site;
determining site display position information of one or more to-be-navigated relic resources in the target site;
acquiring a shooting position of the shooting picture in the field virtual space;
Detecting a matching relation between site display position information and the shooting position corresponding to each cultural relic resource to be navigated respectively;
if the field display position information matched with the shooting position exists, determining the to-be-navigated cultural relic resource corresponding to the field display position information matched with the field display position information as a virtual cultural relic resource, and determining that the shooting picture comprises a target area for displaying the virtual cultural relic resource in the target field.
14. The method of claim 13, wherein the acquiring the shooting position of the shooting picture in the venue virtual space comprises:
performing region feature recognition on the shot picture to obtain picture region features, acquiring inertial measurement information of the object, and determining region visual angle features of the object on the target site based on the inertial measurement information and the picture region features;
and carrying out global image retrieval in the field virtual space according to the regional visual angle characteristics, and determining the spatial position matched with the regional visual angle characteristics and retrieved in the field virtual space as the shooting position of the shooting picture in the field virtual space.
15. The method of claim 13, wherein the detecting the matching relationship between the location display position information and the shooting position corresponding to each to-be-navigated cultural relic resource respectively includes:
acquiring a first coordinate range of site display position information corresponding to each cultural relic resource to be navigated, and acquiring a second coordinate range of the shooting position;
if a first coordinate range which is overlapped with the second coordinate range in a region exists, determining that the site display position information corresponding to the first coordinate range which is overlapped with the second coordinate range in a region is matched with the shooting position;
the method further comprises the steps of:
and determining a region overlapping with the region in the first coordinate range in the shooting picture as a target region.
16. The method of claim 1, wherein displaying the virtual cultural relic resource on the target area in the photographed picture comprises:
determining a parallax angle between the shooting position and the site display position information and a parallax depth between the shooting position and the site display position information based on the shooting position of the shooting picture and the site display position information of the virtual cultural relics resource;
Determining a display view angle of the virtual cultural relic resource in the shooting picture based on the parallax angle;
determining the display size of the virtual cultural relic resource in the shooting picture based on the parallax depth;
and acquiring material texture information and cultural relic model vertex coordinate information associated with the virtual cultural relic resource, and rendering the virtual cultural relic resource at the target area in the shooting picture according to the rendering mapping relation between the cultural relic model vertex coordinate information and the material texture information, the parallax angle and the parallax depth.
17. A data processing apparatus, comprising:
the cultural relic guiding module is used for responding to cultural relic guiding operation in a display application, and displaying a shooting picture aiming at a target site and shot by the shooting component in the display application; the target site is an object site triggering the cultural relic navigation operation;
the cultural relic display module is used for displaying the virtual cultural relic resources positioned on the target area in the shooting picture when the shooting picture contains the target area used for displaying the virtual cultural relic resources in the target area; the cultural relic type of the virtual cultural relic resource is associated with the target site, and the display position of the virtual cultural relic resource in the shooting picture is associated with the display position of the target area in the shooting picture;
And the cultural relic updating module is used for updating the display view angle of the virtual cultural relic resource in the shooting picture when the position relation between the shooting component and the target area is changed and the shooting picture continuously comprises the target area.
18. A computer device, comprising: a processor, a memory, and a network interface;
the processor is connected to the memory, the network interface for providing data communication functions, the memory for storing a computer program, the processor for invoking the program code to cause the computer device to perform the method of any of claims 1-16.
19. A readable storage medium, characterized in that the readable storage medium has stored therein a computer program adapted to be loaded and executed by a processor to cause a computer device having the processor to perform the method of any of claims 1-16.
20. A computer program product, characterized in that the computer program product comprises a computer program stored in a readable storage medium and adapted to be read and executed by a processor to cause a computer device with the processor to perform the method of any of claims 1-16.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310203020.8A CN116974416A (en) | 2023-02-21 | 2023-02-21 | Data processing method, device, equipment and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310203020.8A CN116974416A (en) | 2023-02-21 | 2023-02-21 | Data processing method, device, equipment and readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116974416A true CN116974416A (en) | 2023-10-31 |
Family
ID=88482037
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310203020.8A Pending CN116974416A (en) | 2023-02-21 | 2023-02-21 | Data processing method, device, equipment and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116974416A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117765839A (en) * | 2023-12-25 | 2024-03-26 | 广东保伦电子股份有限公司 | Indoor intelligent navigation method, device and storage medium |
-
2023
- 2023-02-21 CN CN202310203020.8A patent/CN116974416A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117765839A (en) * | 2023-12-25 | 2024-03-26 | 广东保伦电子股份有限公司 | Indoor intelligent navigation method, device and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR102417645B1 (en) | AR scene image processing method, device, electronic device and storage medium | |
| CN105393284B (en) | Space engraving based on human body data | |
| CN105027033B (en) | Method, device and computer-readable media for selecting Augmented Reality object | |
| CN105279795B (en) | Augmented reality system based on 3D marker | |
| US20170193679A1 (en) | Information processing apparatus and information processing method | |
| US20150049086A1 (en) | 3D Space Content Visualization System | |
| KR20140082610A (en) | Method and apaaratus for augmented exhibition contents in portable terminal | |
| CN111833458B (en) | Image display method and device, equipment and computer readable storage medium | |
| CN112954292B (en) | Digital museum navigation system and method based on augmented reality | |
| CN112330819B (en) | Interaction method and device based on virtual article and storage medium | |
| CN110168615A (en) | Information processing equipment, information processing method and program | |
| US20180239514A1 (en) | Interactive 3d map with vibrant street view | |
| US20210038975A1 (en) | Calibration to be used in an augmented reality method and system | |
| US11823334B2 (en) | Efficient capture and delivery of walkable and interactive virtual reality or 360 degree video | |
| US11423625B2 (en) | Augmented reality scene image processing method and apparatus, electronic device and storage medium | |
| CN111640192A (en) | Scene image processing method and device, AR device and storage medium | |
| CN116974416A (en) | Data processing method, device, equipment and readable storage medium | |
| CN114358822A (en) | Advertisement display method, device, medium and equipment | |
| CN112947756A (en) | Content navigation method, device, system, computer equipment and storage medium | |
| CN108235764B (en) | Information processing method and device, cloud processing equipment and computer program product | |
| CN115345927A (en) | Exhibit guide method and related device, mobile terminal and storage medium | |
| WO2021029164A1 (en) | Image processing device, image processing method, and program | |
| KR102443016B1 (en) | Apparatus and method for implementing advertising using augmented technology | |
| CN114299263A (en) | Display method and device for augmented reality AR scene | |
| KR20100124571A (en) | Apparatus and method for guiding information using augmented reality |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| REG | Reference to a national code |
Ref country code: HK Ref legal event code: DE Ref document number: 40098087 Country of ref document: HK |