CN111638794A - Display control method and device for virtual cultural relics - Google Patents
Display control method and device for virtual cultural relics Download PDFInfo
- Publication number
- CN111638794A CN111638794A CN202010502261.9A CN202010502261A CN111638794A CN 111638794 A CN111638794 A CN 111638794A CN 202010502261 A CN202010502261 A CN 202010502261A CN 111638794 A CN111638794 A CN 111638794A
- Authority
- CN
- China
- Prior art keywords
- cultural relic
- limb action
- virtual cultural
- target user
- similarity
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
Abstract
The method comprises the steps of obtaining limb action characteristics of a target user and stored limb action matching characteristics corresponding to a plurality of candidate virtual cultural relic materials; respectively determining the similarity between the limb action matching characteristics corresponding to the candidate virtual cultural relic materials and the limb action characteristics of the target user; and selecting a target virtual cultural relic material from the candidate virtual cultural relic materials for displaying based on the determined similarity. According to the embodiment of the method and the device, the similarity between the limb action characteristics of the target user and the limb action matching characteristics corresponding to the candidate virtual cultural relic materials can be determined, the target virtual cultural relic materials are selected to be displayed based on the similarity, and the virtual cultural relics which the user needs to visit are flexibly and conveniently displayed or switched.
Description
Technical Field
The disclosure relates to the technical field of computers, in particular to a display control method and device for virtual cultural relics.
Background
Three-dimensional models are polygonal representations of objects, typically displayed by a computer or other video device. The displayed object may be a real-world entity or a fictional object. Anything that exists in physical nature can be represented by a three-dimensional model. At present, the three-dimensional modeling technology is rapidly developed and can be widely applied in various industries. For example, in a scene of an exhibition hall, it is inconvenient to show an entity of a historical relic, a three-dimensional model of the historical relic can be usually made, the three-dimensional model is used to replace the historical relic of the entity, and a virtual relic is displayed through a display device.
At present, in a similar scene such as an exhibition hall, one display device usually displays only one virtual cultural relic, and after a preset time interval elapses or a control instruction is received, the display device can switch the displayed virtual cultural relics in sequence. However, the control device of the exhibition hall usually only performs simple control operation, and cannot accurately locate the virtual cultural relic that the user wants to watch, which results in higher operation difficulty for the user and lower efficiency of display control.
Disclosure of Invention
The embodiment of the disclosure at least provides a display control method and device for virtual cultural relics, which are used for rapidly and accurately displaying the virtual cultural relics required by a user.
In a first aspect, an embodiment of the present disclosure provides a method for controlling display of a virtual cultural relic, including:
acquiring limb action characteristics of a target user and limb action matching characteristics corresponding to a plurality of stored candidate virtual cultural relic materials;
respectively determining the similarity between the limb action matching characteristics corresponding to the candidate virtual cultural relic materials and the limb action characteristics of the target user;
and selecting a target virtual cultural relic material from the candidate virtual cultural relic materials for displaying based on the determined similarity.
By adopting the method for controlling the display of the virtual cultural relics, the similarity between the limb action characteristics of the target user and the limb action matching characteristics corresponding to the candidate virtual cultural relic materials can be determined, and then the target virtual cultural relic materials can be selected for display based on the similarity, so that the user can flexibly and conveniently select the displayed virtual cultural relic materials based on the limb actions, the display control mode is not only flexible and convenient, but also the control operation requirements on the control equipment are reduced.
In an optional embodiment, acquiring the limb action characteristics of the target user includes:
continuously acquiring multi-frame whole-body images of a target user;
and extracting the limb action characteristics of the target user based on the multi-frame whole-body image.
In an optional implementation manner, extracting the limb action feature of the target user based on the multiple frames of whole-body images includes:
extracting position information of a skeleton key point aiming at each frame of the whole-body image;
determining pose information of each human body key skeleton corresponding to each frame of whole-body image based on the position information of the skeleton key points in each frame of whole-body image;
and determining the limb action characteristics of the target user based on the pose information of each human body key skeleton corresponding to the multiple frames of whole-body images respectively.
The position information of the key points of the bones can be extracted according to the whole-body image of the target user, the position information of key bones of the human body is determined based on the extracted position information, the body motion characteristics of the target user are determined according to the pose information, and the accuracy of the recognized body motion of the user can be improved.
In an optional implementation manner, extracting the limb action feature of the target user based on the multiple frames of whole-body images includes:
inputting the multi-frame whole-body image into a trained limb action characteristic extraction model to obtain the limb action characteristic of the target user; the limb action characteristic extraction model is obtained by training based on multi-frame sample images marked with limb action characteristics.
In an optional implementation manner, determining similarity between the limb motion matching features corresponding to the multiple candidate virtual cultural relic materials and the limb motion features of the target user respectively includes:
and calculating cosine similarity between a first feature vector representing the limb action matching feature and a second feature vector representing the limb action feature of the target user aiming at each candidate virtual cultural relic material.
In an alternative embodiment, selecting a target virtual cultural relic material from the plurality of candidate virtual cultural relic materials for presentation based on the determined similarity comprises:
selecting a corresponding target virtual cultural relic material with the highest similarity from the plurality of candidate virtual cultural relic materials for displaying; alternatively, the first and second electrodes may be,
and selecting a preset number of target virtual cultural relic materials from the candidate virtual cultural relic materials to be displayed in turn according to the sequence of the similarity from high to low.
In a second aspect, an embodiment of the present disclosure further provides a display control apparatus for a virtual cultural relic, including:
the acquisition module is used for acquiring the limb action characteristics of the target user and the stored limb action matching characteristics corresponding to the candidate virtual cultural relic materials;
the determining module is used for respectively determining the similarity between the limb action matching characteristics corresponding to the candidate virtual cultural relic materials and the limb action characteristics of the target user;
and the display module is used for selecting a target virtual cultural relic material from the candidate virtual cultural relic materials to display based on the determined similarity.
In an optional implementation, the obtaining module includes:
the acquisition unit is used for continuously acquiring multi-frame whole-body images of a target user;
and the extraction unit is used for extracting the limb action characteristics of the target user based on the multi-frame whole body image.
In an optional implementation manner, the extraction unit is specifically configured to:
extracting position information of a skeleton key point aiming at each frame of the whole-body image;
determining pose information of each human body key skeleton corresponding to each frame of whole-body image based on the position information of the skeleton key points in each frame of whole-body image;
and determining the limb action characteristics of the target user based on the pose information of each human body key skeleton corresponding to the multiple frames of whole-body images respectively.
In an optional implementation manner, the extraction unit is specifically configured to:
inputting the multi-frame whole-body image into a trained limb action characteristic extraction model to obtain the limb action characteristic of the target user; the limb action characteristic extraction model is obtained by training based on multi-frame sample images marked with limb action characteristics.
In an optional implementation manner, the determining module is specifically configured to:
and calculating cosine similarity between a first feature vector representing the limb action matching feature and a second feature vector representing the limb action feature of the target user aiming at each candidate virtual cultural relic material.
In an optional implementation manner, the display module is specifically configured to:
selecting a corresponding target virtual cultural relic material with the highest similarity from the plurality of candidate virtual cultural relic materials for displaying; alternatively, the first and second electrodes may be,
and selecting a preset number of target virtual cultural relic materials from the candidate virtual cultural relic materials to be displayed in turn according to the sequence of the similarity from high to low.
In a third aspect, an embodiment of the present disclosure further provides a computer device, including: a processor and a memory connected to each other, the memory storing machine readable instructions executable by the processor, the machine readable instructions being executed by the processor when the computer device runs to implement the method for controlling display of a virtual cultural relic in the first aspect described above, or any one of the possible implementation manners of the first aspect.
In a fourth aspect, the disclosed embodiments further provide a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and the computer program is executed by a processor to perform the steps of the method for controlling display of a virtual cultural relic in the first aspect, or any one of the possible implementation manners of the first aspect.
In order to make the aforementioned objects, features and advantages of the present disclosure more comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for use in the embodiments will be briefly described below, and the drawings herein incorporated in and forming a part of the specification illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the technical solutions of the present disclosure. It is appreciated that the following drawings depict only certain embodiments of the disclosure and are therefore not to be considered limiting of its scope, for those skilled in the art will be able to derive additional related drawings therefrom without the benefit of the inventive faculty.
Fig. 1 is a flowchart illustrating a display control method for a virtual cultural relic according to an embodiment of the present disclosure;
fig. 2 is a flowchart illustrating another method for controlling display of a virtual cultural relic according to an embodiment of the present disclosure;
fig. 3 is a schematic diagram illustrating a whole-body image in another display control method for a virtual cultural relic provided by the embodiment of the present disclosure;
fig. 4 is a schematic diagram illustrating a target virtual cultural relic displayed in another display control method for a virtual cultural relic provided by the embodiment of the present disclosure;
fig. 5 is a schematic diagram illustrating a display control apparatus for a virtual cultural relic according to an embodiment of the present disclosure;
fig. 6 is a schematic diagram illustrating an acquisition module in a display control apparatus for a virtual cultural relic according to an embodiment of the present disclosure;
fig. 7 shows a schematic diagram of a computer device provided by an embodiment of the present disclosure.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions of the embodiments of the present disclosure will be described clearly and completely with reference to the drawings in the embodiments of the present disclosure, and it is obvious that the described embodiments are only a part of the embodiments of the present disclosure, not all of the embodiments. The components of the embodiments of the present disclosure, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present disclosure, presented in the figures, is not intended to limit the scope of the claimed disclosure, but is merely representative of selected embodiments of the disclosure. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the disclosure without making creative efforts, shall fall within the protection scope of the disclosure.
Research shows that in the scenes of exhibition halls and the like, the virtual cultural relics can be displayed through the display equipment, and usually, the virtual cultural relics to be watched can be selected through a mouse, a keyboard, a remote controller, a touch panel and other controllers. However, in the mode of selecting the virtual cultural relics through the controller, the controller usually only has a simple control function before control, and cannot quickly complete functions such as searching the virtual cultural relics, so that a user needs to perform complicated operations to check the needed virtual cultural relics, the operation difficulty of the user is high, and the efficiency of performing display control is low.
Based on the research, the present disclosure provides a display control method and device for virtual cultural relics, which can quickly and accurately display the virtual cultural relics required by a user without using a controller by determining the similarity between the limb action features of the target user and the limb action matching features corresponding to candidate virtual cultural relic materials and then selecting the target virtual cultural relic materials for display based on the similarity.
The above-mentioned drawbacks are the results of the inventor after practical and careful study, and therefore, the discovery process of the above-mentioned problems and the solutions proposed by the present disclosure to the above-mentioned problems should be the contribution of the inventor in the process of the present disclosure.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.
To facilitate understanding of the present embodiment, first, a detailed description is given of a display control method for a virtual cultural relic disclosed in the embodiments of the present disclosure, and an execution subject of the display control method for a virtual cultural relic provided in the embodiments of the present disclosure is generally a computer device with certain computing capability, where the computer device includes: a terminal device, which may be a User Equipment (UE), a mobile device, a User terminal, a cellular phone, a cordless phone, a Personal Digital Assistant (PDA), a handheld device, a computing device, a vehicle mounted device, a wearable device, or a server or other processing device. In some possible implementations, the display control method of the virtual cultural relic can be implemented by the processor calling the computer readable instructions stored in the memory.
The following describes a method for controlling display of a virtual cultural relic according to an embodiment of the present disclosure, taking an execution subject as a terminal device as an example. The terminal device may be provided with a presentation device capable of presenting a graphical user interface and a photographing device capable of photographing an image.
Referring to fig. 1, a flowchart of a method for controlling display of a virtual cultural relic provided by the embodiment of the present disclosure is shown, where the method includes steps S101 to S103, where:
s101: and acquiring the limb action characteristics of the target user and the stored limb action matching characteristics corresponding to the multiple candidate virtual cultural relic materials.
In the step, the terminal device can detect the target detection area through the photographing device, the photographing device can record a video in the target detection area, the photographing device can continuously record continuous multi-frame whole body images of the target user under the condition that the target user enters the target detection area, and then the body action characteristics of the target user are determined according to the whole body images.
The limb action characteristics can be characteristics containing information such as coordinates or motion tracks of all limb parts of the target user in a preset coordinate system. The limb motion matching features are of the same content type as the limb motion features. The limb action matching characteristics can be determined according to the shape, characteristics and the like of the candidate virtual cultural relic materials corresponding to the limb action matching characteristics.
Here, the virtual cultural relic material may include presentation material such as a three-dimensional stereoscopic model, a planar image, a descriptive text, and the like of the virtual cultural relic.
S102: and respectively determining the similarity between the limb action matching characteristics corresponding to the candidate virtual cultural relic materials and the limb action characteristics of the target user.
In this step, the limb motion characteristics of the target user may be compared with the limb motion matching characteristics corresponding to the plurality of candidate virtual cultural relic materials, respectively, to obtain the similarity.
S103: and selecting a target virtual cultural relic material from the candidate virtual cultural relic materials for displaying based on the determined similarity.
In the step, after the similarity between the limb action characteristics and the plurality of limb action matching characteristics is obtained, the virtual cultural relic materials corresponding to the limb action matching characteristics with the highest similarity can be directly used as matched targets and displayed in the display equipment; and the virtual cultural relic materials with the similarity greater than the preset similarity can be used as matched targets and displayed in the display equipment in turn or together according to the preset sequence.
The display device can be arranged right in front of the target detection area, and a user in the target detection area can observe the virtual cultural relics displayed on the display device.
According to the method for controlling display of the virtual cultural relics, provided by the embodiment of the disclosure, the similarity between the limb action characteristics of the target user and the limb action matching characteristics corresponding to the candidate virtual cultural relic materials can be determined, and then the target virtual cultural relic materials are selected to be displayed based on the similarity, so that the virtual cultural relics required by the user can be displayed quickly and accurately without using a controller.
Referring to fig. 2, a flowchart of another method for controlling display of a virtual cultural relic provided by the embodiment of the present disclosure is shown, where the method includes steps S201 to S204, where:
s201: continuously acquiring multi-frame whole-body images of a target user.
In this step, the terminal device may detect the target detection area through the photographing device, and when the target user is located in the target detection area, the photographing device may photograph the target user to obtain a continuous multi-frame whole body image, where the whole body image may include all the limbs of the target user.
The whole-body image may be a normal color image or a depth image. Depth images (depth), also known as range images, refer to images that take as pixel values the distance (depth) from an image grabber to each point in a scene, which directly reflects the geometry of the visible surface of the scene. The depth image can be calculated into point cloud data through coordinate conversion, and the point cloud data with regular and necessary information can also be inversely calculated into depth image data. In the image frames provided by the depth data stream, each pixel represents the distance (in millimeters) to the plane of the camera from the object closest to the plane at that particular (x, y) coordinate in the field of view of the depth sensor. In the case where the whole-body image is a depth image, the photographing apparatus may be a depth camera.
S202: and extracting the limb action characteristics of the target user based on the multi-frame whole-body image.
In the step, the obtained multiple frames of whole-body images can be processed, so that the image information of each key point in the whole-body images is converted into information which can be processed by a computer, the feature information of each key point in each frame of whole-body images is extracted, and the feature information of one key point in each frame of whole-body images is spliced or combined to obtain the limb action features of the target user. The limb motion characteristic may characterize a trajectory, speed, or other characteristic of the user's limb motion.
S203: and respectively determining the similarity between the limb action matching characteristics corresponding to the candidate virtual cultural relic materials and the limb action characteristics of the target user.
S204: and selecting a target virtual cultural relic material from the candidate virtual cultural relic materials for displaying based on the determined similarity.
The principle and explanation of S203 and S204 are the same as S102 and S103, and are not repeated here.
In a possible implementation manner, extracting the limb action feature of the target user based on the plurality of frames of whole-body images includes:
extracting position information of a skeleton key point aiming at each frame of the whole-body image;
determining pose information of each human body key skeleton corresponding to each frame of whole-body image based on the position information of the skeleton key points in each frame of whole-body image;
and determining the limb action characteristics of the target user based on the pose information of each human body key skeleton corresponding to the multiple frames of whole-body images respectively.
In this step, the whole-body image may be processed first to obtain a person silhouette image, where the person silhouette image includes the outline of the limbs of the target user. After the figure silhouette image is obtained, skeleton key points in the figure silhouette can be identified according to inflection points of the outline, and then position information of the skeleton key points is extracted.
The number of the skeleton key points is fixed, the same number of the skeleton key points exist in each frame of image, and in different images, each skeleton key point corresponds to one another; the location information may be coordinates of the bone key points in a preset coordinate system. In the case where the whole-body image is a depth image, the position information may be three-dimensional coordinates of the bone coordinate point in a preset three-dimensional coordinate system.
After the position information of the bone key points is determined, the pose information of the human body key bones corresponding to the bone key points can be determined according to the position information of the bone key points. Each human critical bone may correspond to at least two skeletal keypoints. The pose information of the key human skeleton may include position information and inclination angle information of the key human skeleton in a preset coordinate system.
After the pose information is determined, motion information such as the motion direction, the motion track, the motion speed and the like of each human key skeleton can be determined according to the pose information of each human key skeleton in each frame of whole body image, the motion information of each human key skeleton is integrated together, and the limb motion characteristics of the target user in multiple frames are extracted.
In a possible implementation manner, extracting the limb action feature of the target user based on the plurality of frames of whole-body images includes:
inputting the multi-frame whole-body image into a trained limb action characteristic extraction model to obtain the limb action characteristic of the target user; the limb action characteristic extraction model is obtained by training based on multi-frame sample images marked with limb action characteristics.
In the step, multiple frames of whole body images can be directly input into the trained limb action characteristic extraction model, and the limb action characteristic extraction model can directly process and calculate the whole body images and output the limb action characteristics of the target user. The limb action characteristic extraction model can be obtained by training the multi-frame sample image marked with the limb action characteristics, and the model can learn the relationship between various information in the whole body image and the marked limb action characteristics.
In one possible implementation manner, the determining the similarity between the limb action matching features corresponding to the candidate virtual cultural relic materials and the limb action features of the target user respectively includes:
and calculating cosine similarity between a first feature vector representing the limb action matching feature and a second feature vector representing the limb action feature of the target user aiming at each candidate virtual cultural relic material.
In this step, the limb motion matching features and the limb motion features may be vectorized to obtain a plurality of first feature vectors representing the limb motion matching features and a plurality of second feature vectors representing the limb motion features of the target user, and then the cosine similarity between each first feature vector and each second feature vector is calculated.
In one possible embodiment, based on the determined similarity, selecting a target virtual cultural relic material from the plurality of candidate virtual cultural relic materials for presentation comprises:
selecting a corresponding target virtual cultural relic material with the highest similarity from the plurality of candidate virtual cultural relic materials for displaying; alternatively, the first and second electrodes may be,
and selecting a preset number of target virtual cultural relic materials from the candidate virtual cultural relic materials to be displayed in turn according to the sequence of the similarity from high to low.
In this step, the virtual cultural relic materials with the highest similarity can be selected as the target virtual cultural relic materials, and a preset number of virtual cultural relic materials before the highest similarity can also be selected as the target virtual cultural relic materials. When the number of the target virtual cultural relic materials is 1, the target virtual cultural relic materials can be continuously displayed on the display equipment; when the number of the target virtual cultural relic materials is multiple, the target virtual cultural relic materials can be presented in turn.
Further, the virtual cultural relic materials with the similarity higher than the preset similarity can be used as the target virtual cultural relic materials.
Referring to fig. 3 and 4, fig. 3 is a schematic diagram of a whole-body image in another display control method for a virtual cultural relic provided by the embodiment of the disclosure, and fig. 4 is a schematic diagram of a target virtual cultural relic displayed in another display control method for a virtual cultural relic provided by the embodiment of the disclosure.
In fig. 3, the limb of the target user is in the jumping state, after the limb action feature in the jumping state is acquired, the similarity between the limb action feature and the plurality of limb action matching features is calculated, and the virtual cultural relic material corresponding to the limb matching feature with the highest similarity is the "horse-trotter", so that the display device in fig. 4 can display the virtual cultural relic material of the "horse-trotter".
It will be understood by those skilled in the art that in the method of the present invention, the order of writing the steps does not imply a strict order of execution and any limitations on the implementation, and the specific order of execution of the steps should be determined by their function and possible inherent logic.
Based on the same inventive concept, the embodiment of the present disclosure further provides a display control device for a virtual cultural relic corresponding to the display control method for a virtual cultural relic, and since the principle of solving the problem of the device in the embodiment of the present disclosure is similar to that of the display control method for a virtual cultural relic in the embodiment of the present disclosure, the implementation of the device can refer to the implementation of the method, and repeated details are omitted.
Referring to fig. 5, a schematic diagram of a display control apparatus for a virtual cultural relic according to an embodiment of the present disclosure is shown, the apparatus includes: an acquisition module 510, a determination module 520, and a display module 530; wherein the content of the first and second substances,
an obtaining module 510, configured to obtain a limb motion feature of a target user and a stored limb motion matching feature corresponding to multiple candidate virtual cultural relic materials;
a determining module 520, configured to determine similarity between the limb motion matching features corresponding to the multiple candidate virtual cultural relic materials and the limb motion features of the target user respectively;
a displaying module 530, configured to select a target virtual cultural relic material from the multiple candidate virtual cultural relic materials for displaying based on the determined similarity.
In a possible implementation manner, the determining module 520 is specifically configured to:
and calculating cosine similarity between a first feature vector representing the limb action matching feature and a second feature vector representing the limb action feature of the target user aiming at each candidate virtual cultural relic material.
In a possible implementation manner, the display module 530 is specifically configured to:
selecting a corresponding target virtual cultural relic material with the highest similarity from the plurality of candidate virtual cultural relic materials for displaying; alternatively, the first and second electrodes may be,
and selecting a preset number of target virtual cultural relic materials from the candidate virtual cultural relic materials to be displayed in turn according to the sequence of the similarity from high to low.
According to the embodiment of the method and the device, the similarity between the limb action characteristics of the target user and the limb action matching characteristics corresponding to the candidate virtual cultural relic materials is determined, and then the target virtual cultural relic materials are selected to be displayed based on the similarity, so that the virtual cultural relics required by the user can be displayed quickly and accurately without using a controller.
Referring to fig. 6, a schematic diagram of an obtaining module in a display control apparatus for a virtual cultural relic provided in an embodiment of the present disclosure is shown, where the obtaining module includes: an acquisition unit 600, an extraction unit 620; wherein the content of the first and second substances,
an acquiring unit 610, configured to continuously acquire multiple frames of whole-body images of a target user;
an extracting unit 620, configured to extract, based on the multiple frames of whole-body images, a limb movement feature of the target user.
In a possible implementation manner, the extracting unit 610 is specifically configured to:
extracting position information of a skeleton key point aiming at each frame of the whole-body image;
determining pose information of each human body key skeleton corresponding to each frame of whole-body image based on the position information of the skeleton key points in each frame of whole-body image;
and determining the limb action characteristics of the target user based on the pose information of each human body key skeleton corresponding to the multiple frames of whole-body images respectively.
In a possible implementation manner, the extracting unit 620 is specifically configured to:
inputting the multi-frame whole-body image into a trained limb action characteristic extraction model to obtain the limb action characteristic of the target user; the limb action characteristic extraction model is obtained by training based on multi-frame sample images marked with limb action characteristics.
The description of the processing flow of each module in the device and the interaction flow between the modules may refer to the related description in the above method embodiments, and will not be described in detail here.
The embodiment of the present disclosure further provides a computer device 10, as shown in fig. 7, which is a schematic structural diagram of the computer device 10 provided in the embodiment of the present disclosure, and includes:
a processor 11 and a memory 12; the memory 12 stores machine-readable instructions executable by the processor 11, which when executed by a computer device are executed by the processor to perform the steps of:
acquiring limb action characteristics of a target user and limb action matching characteristics corresponding to a plurality of stored candidate virtual cultural relic materials;
respectively determining the similarity between the limb action matching characteristics corresponding to the candidate virtual cultural relic materials and the limb action characteristics of the target user;
and selecting a target virtual cultural relic material from the candidate virtual cultural relic materials for displaying based on the determined similarity.
For the specific execution process of the instruction, reference may be made to the steps of the method for controlling display of a virtual cultural relic in the embodiment of the present disclosure, which are not described herein again.
The embodiment of the present disclosure further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method for controlling display of a virtual cultural relic described in the above method embodiments are executed. The storage medium may be a volatile or non-volatile computer-readable storage medium.
The computer program product of the display control method for the virtual cultural relics provided by the embodiment of the present disclosure includes a computer readable storage medium storing a program code, where instructions included in the program code may be used to execute the steps of the display control method for the virtual cultural relics described in the above method embodiments, which may be referred to in the above method embodiments specifically, and are not described herein again.
The embodiments of the present disclosure also provide a computer program, which when executed by a processor implements any one of the methods of the foregoing embodiments. The computer program product may be embodied in hardware, software or a combination thereof. In an alternative embodiment, the computer program product is embodied in a computer storage medium, and in another alternative embodiment, the computer program product is embodied in a Software product, such as a Software Development Kit (SDK), or the like.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the system and the apparatus described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, apparatus, and method may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions when actually implemented, and for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some communication interfaces, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present disclosure may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such understanding, the technical solution of the present disclosure may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present disclosure. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
Finally, it should be noted that: the above-mentioned embodiments are merely specific embodiments of the present disclosure, which are used for illustrating the technical solutions of the present disclosure and not for limiting the same, and the scope of the present disclosure is not limited thereto, and although the present disclosure is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive of the technical solutions described in the foregoing embodiments or equivalent technical features thereof within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the embodiments of the present disclosure, and should be construed as being included therein. Therefore, the protection scope of the present disclosure shall be subject to the protection scope of the claims.
Claims (10)
1. A display control method of a virtual cultural relic is characterized by comprising the following steps:
acquiring limb action characteristics of a target user and limb action matching characteristics corresponding to a plurality of stored candidate virtual cultural relic materials;
respectively determining the similarity between the limb action matching characteristics corresponding to the candidate virtual cultural relic materials and the limb action characteristics of the target user;
and selecting a target virtual cultural relic material from the candidate virtual cultural relic materials for displaying based on the determined similarity.
2. The method of claim 1, wherein obtaining the limb movement characteristics of the target user comprises:
continuously acquiring multi-frame whole-body images of a target user;
and extracting the limb action characteristics of the target user based on the multi-frame whole-body image.
3. The method according to claim 2, wherein extracting the limb action feature of the target user based on the plurality of frames of whole-body images comprises:
extracting position information of a skeleton key point aiming at each frame of the whole-body image;
determining pose information of each human body key skeleton corresponding to each frame of whole-body image based on the position information of the skeleton key points in each frame of whole-body image;
and determining the limb action characteristics of the target user based on the pose information of each human body key skeleton corresponding to the multiple frames of whole-body images respectively.
4. The method according to claim 2, wherein extracting the limb action feature of the target user based on the plurality of frames of whole-body images comprises:
inputting the multi-frame whole-body image into a trained limb action characteristic extraction model to obtain the limb action characteristic of the target user; the limb action characteristic extraction model is obtained by training based on multi-frame sample images marked with limb action characteristics.
5. The method of claim 1, wherein determining similarity between the limb motion matching features corresponding to the candidate virtual cultural relic materials and the limb motion features of the target user respectively comprises:
and calculating cosine similarity between a first feature vector representing the limb action matching feature and a second feature vector representing the limb action feature of the target user aiming at each candidate virtual cultural relic material.
6. The method of claim 1, wherein selecting a target virtual cultural relic material from the plurality of candidate virtual cultural relic materials for presentation based on the determined similarity comprises:
selecting a corresponding target virtual cultural relic material with the highest similarity from the plurality of candidate virtual cultural relic materials for displaying; alternatively, the first and second electrodes may be,
and selecting a preset number of target virtual cultural relic materials from the candidate virtual cultural relic materials to be displayed in turn according to the sequence of the similarity from high to low.
7. A display control device for a virtual cultural relic, comprising:
the acquisition module is used for acquiring the limb action characteristics of the target user and the stored limb action matching characteristics corresponding to the candidate virtual cultural relic materials;
the determining module is used for respectively determining the similarity between the limb action matching characteristics corresponding to the candidate virtual cultural relic materials and the limb action characteristics of the target user;
and the display module is used for selecting a target virtual cultural relic material from the candidate virtual cultural relic materials to display based on the determined similarity.
8. The apparatus of claim 7, wherein the obtaining module comprises:
the acquisition unit is used for continuously acquiring multi-frame whole-body images of a target user;
and the extraction unit is used for extracting the limb action characteristics of the target user based on the multi-frame whole body image.
9. An electronic device, comprising: a processor, a memory storing machine readable instructions executable by the processor, the processor for executing the machine readable instructions stored in the memory, the processor performing the steps of the method of controlling display of a virtual cultural relic according to any one of claims 1 to 6 when the machine readable instructions are executed by the processor.
10. A computer-readable storage medium, characterized in that a computer program is stored thereon, which, when executed by an electronic device, causes the electronic device to perform the steps of the method for controlling display of a virtual cultural relic according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010502261.9A CN111638794A (en) | 2020-06-04 | 2020-06-04 | Display control method and device for virtual cultural relics |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010502261.9A CN111638794A (en) | 2020-06-04 | 2020-06-04 | Display control method and device for virtual cultural relics |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111638794A true CN111638794A (en) | 2020-09-08 |
Family
ID=72328578
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010502261.9A Pending CN111638794A (en) | 2020-06-04 | 2020-06-04 | Display control method and device for virtual cultural relics |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111638794A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116152383A (en) * | 2023-03-06 | 2023-05-23 | 深圳优立全息科技有限公司 | Voxel model, image generation method, device and storage medium |
| CN116681869A (en) * | 2023-06-21 | 2023-09-01 | 西安交通大学城市学院 | Cultural relic 3D display processing method based on virtual reality application |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108614638A (en) * | 2018-04-23 | 2018-10-02 | 太平洋未来科技（深圳）有限公司 | AR imaging methods and device |
| CN109191548A (en) * | 2018-08-28 | 2019-01-11 | 百度在线网络技术（北京）有限公司 | Animation method, device, equipment and storage medium |
| CN109453517A (en) * | 2018-10-16 | 2019-03-12 | Oppo广东移动通信有限公司 | Virtual role control method and device, storage medium, mobile terminal |
| CN110245638A (en) * | 2019-06-20 | 2019-09-17 | 北京百度网讯科技有限公司 | Video generation method and device |
| CN110298326A (en) * | 2019-07-03 | 2019-10-01 | 北京字节跳动网络技术有限公司 | A kind of image processing method and device, storage medium and terminal |
| CN110390705A (en) * | 2018-04-16 | 2019-10-29 | 北京搜狗科技发展有限公司 | A kind of method and device generating virtual image |
| CN110688018A (en) * | 2019-11-05 | 2020-01-14 | 广东虚拟现实科技有限公司 | Virtual picture control method and device, terminal equipment and storage medium |
| CN110827378A (en) * | 2019-10-31 | 2020-02-21 | 北京字节跳动网络技术有限公司 | Virtual image generation method, device, terminal and storage medium |
-
2020
- 2020-06-04 CN CN202010502261.9A patent/CN111638794A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110390705A (en) * | 2018-04-16 | 2019-10-29 | 北京搜狗科技发展有限公司 | A kind of method and device generating virtual image |
| CN108614638A (en) * | 2018-04-23 | 2018-10-02 | 太平洋未来科技（深圳）有限公司 | AR imaging methods and device |
| CN109191548A (en) * | 2018-08-28 | 2019-01-11 | 百度在线网络技术（北京）有限公司 | Animation method, device, equipment and storage medium |
| CN109453517A (en) * | 2018-10-16 | 2019-03-12 | Oppo广东移动通信有限公司 | Virtual role control method and device, storage medium, mobile terminal |
| CN110245638A (en) * | 2019-06-20 | 2019-09-17 | 北京百度网讯科技有限公司 | Video generation method and device |
| CN110298326A (en) * | 2019-07-03 | 2019-10-01 | 北京字节跳动网络技术有限公司 | A kind of image processing method and device, storage medium and terminal |
| CN110827378A (en) * | 2019-10-31 | 2020-02-21 | 北京字节跳动网络技术有限公司 | Virtual image generation method, device, terminal and storage medium |
| CN110688018A (en) * | 2019-11-05 | 2020-01-14 | 广东虚拟现实科技有限公司 | Virtual picture control method and device, terminal equipment and storage medium |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116152383A (en) * | 2023-03-06 | 2023-05-23 | 深圳优立全息科技有限公司 | Voxel model, image generation method, device and storage medium |
| CN116152383B (en) * | 2023-03-06 | 2023-08-11 | 深圳优立全息科技有限公司 | Voxel model, image generation method, device and storage medium |
| CN116681869A (en) * | 2023-06-21 | 2023-09-01 | 西安交通大学城市学院 | Cultural relic 3D display processing method based on virtual reality application |
| CN116681869B (en) * | 2023-06-21 | 2023-12-19 | 西安交通大学城市学院 | Cultural relic 3D display processing method based on virtual reality application |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106355153B (en) | A kind of virtual objects display methods, device and system based on augmented reality | |
| JP5453246B2 (en) | Camera-based user input for compact devices | |
| CN112148197A (en) | Augmented reality AR interaction method and device, electronic equipment and storage medium | |
| EP2903256B1 (en) | Image processing device, image processing method and program | |
| US20150241984A1 (en) | Methods and Devices for Natural Human Interfaces and for Man Machine and Machine to Machine Activities | |
| US11094079B2 (en) | Determining a pose of an object from RGB-D images | |
| CN111638797A (en) | Display control method and device | |
| CN112729327A (en) | Navigation method, navigation device, computer equipment and storage medium | |
| CN111698646B (en) | Positioning method and device | |
| CN109949900B (en) | Three-dimensional pulse wave display method and device, computer equipment and storage medium | |
| CN111652971A (en) | Display control method and device | |
| CN112905014A (en) | Interaction method and device in AR scene, electronic equipment and storage medium | |
| CN111638794A (en) | Display control method and device for virtual cultural relics | |
| CN111651051A (en) | Virtual sand table display method and device | |
| CN111625100A (en) | Method and device for presenting picture content, computer equipment and storage medium | |
| CN112882576A (en) | AR interaction method and device, electronic equipment and storage medium | |
| CN113359986A (en) | Augmented reality data display method and device, electronic equipment and storage medium | |
| CN111651052A (en) | Virtual sand table display method and device, electronic equipment and storage medium | |
| CN112991555B (en) | Data display method, device, equipment and storage medium | |
| KR20190048506A (en) | Method and apparatus for providing virtual room | |
| CN113359983A (en) | Augmented reality data presentation method and device, electronic equipment and storage medium | |
| CN111640203A (en) | Image processing method and device | |
| CN111640206A (en) | Dynamic control method and device | |
| CN112365607A (en) | Augmented reality AR interaction method, device, equipment and storage medium | |
| CN111639615A (en) | Trigger control method and device for virtual building |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |