CN118200663A - Interactive video playing method combined with digital three-dimensional model display - Google Patents

Abstract

The invention discloses an interactive video playing method and system combined with digital three-dimensional model display, comprising the following steps: analyzing a section of video to be played, and selecting one or more objects on a video frame to mark the video frame; acquiring real objects corresponding to the marked one or more objects; acquiring a digital three-dimensional model of the real object and storing the digital three-dimensional model into a three-dimensional model information database; simultaneously inputting background information related to the physical entity into a three-dimensional model information database, and enabling the background information to be in one-to-one correspondence with the digitized three-dimensional model of the physical entity; and establishing an interactive response function and mechanism, so that a user can call out the digitized three-dimensional model of one or more objects corresponding to the object or objects from the three-dimensional model information database and observe and enjoy the digitized three-dimensional model when watching the video and clicking the marked one or more objects.

Description

Interactive video playing method combined with digital three-dimensional model display

Technical Field

The invention belongs to the field of interactive video playing, and particularly relates to a method for enabling a user to know articles displayed in a video, particularly the form, detail, size and background information of the articles in a more comprehensive and deeper mode by clicking and importing a digital three-dimensional model.

Background

Video technology has been an ideal way to display and introduce items since birth. The disadvantage is that the video transmission has long been a "unidirectional" problem, so that the functions and effects of the video display and introduction of the objects cannot be further exerted.

By "unidirectional" is meant that the presentation and introduction of items in the video is all framed and edited at the discretion of the producer of the video, which results in users often not being able to see or carefully watch the content of interest, but passively looking at the content of no interest.

Recently, the advent of interactive video has provided a new solution to the problem of "unidirectional" video playback

The invention patent 'interactive video playing method and system' (CN 108769808B) provides a method for obtaining a static image which is most matched with a dynamic image by clicking a region where a certain object in the dynamic image is located when the video is played, so that a user can know more relevant information of the object described by the static image.

However, the invention has the defects that the method for acquiring the most matched static image is troublesome to implement and has low precision, because the method firstly uses a square to frame the interaction area between a user and the video when the video is played, and then matches the dynamic image in the square range with the most approximate static image. Since there is a certain error in matching of images, it is difficult to avoid jumping out of image information completely inconsistent with the moving image in the video after the user clicks on the moving image. Obviously, the method is more accurate, reliable and efficient by directly clicking on the image of the object in the video and linking the image with related information outside the video.

The invention patent (CN 107995516B) provides a method and a device for displaying articles in an interactive video, namely, in the playing process of the target video, when a target image is detected in a video frame of the target video, a hot spot icon of the target image is displayed in a region corresponding to the target image on a floating layer of the target video, and after the hot spot icon is controlled to track a motion track of the target image in the target video for a period of time, the hot spot icon of the target image is displayed in a preset region in the floating layer of the target video. The dynamic display of the hotspot icons can attract users to click the hotspot icons, and when a user selects an instruction of the hotspot icons, an information floating layer of a target image is displayed in a playing interface of the target video, so that better video interaction experience is brought to the users. "

The advantage emphasized by this patent is, in turn, that is, because it draws the attention of the user through the appearance and movement of "hot icons" on the screen, one by one, corresponding to the image of the item appearing in the video, and in this way induces the user's action of clicking on the icon. Some people with little experience in video viewing know that the appearance of such a "hot spot icon" and its subsequent movement in the video frame will to some extent disrupt the playback of the video, pulling the user's attention entirely from viewing the video over the decision to consider whether to click on the "hot spot icon". Moreover, once a plurality of different objects appear in the process of playing the video, the plurality of different objects will cause the appearance of a plurality of 'hot spot icons' and their movement, which will inevitably make the user dizziness and affect his wish to continue watching the video.

Whether interactive video or interactive video, the main body of the video is the video. The video related to the object is generally shot and made, and the core purpose of the video is to introduce aspects of the object. Therefore, the key point of the interactive video is not to transfer the attention of the user from the video, but to make up for the short board of the traditional video playing mode, so that the effect of introducing articles is better exerted.

Obviously, the short board with the largest traditional video is limited by subjective favorites of photographers and editors of the video, and can not fully meet the cognitive demands of users on factors such as the appearance, the size, the details and the like of the object. Therefore, the key point of the production of the interactive video or the interactive video is to put on how to add reasonable and efficient external resources to the video, and on the premise of not affecting the fluency and the ornamental effect of the video, the user can easily and pleasurably expand the cognition of the objects appearing in the video in a more direct, natural, more convenient, fluent and clear way. Therefore, combining the presentation of a digitized three-dimensional model with video playback is an ideal solution.

Existing digital three-dimensional scanning techniques and modeling techniques have been able to obtain extremely accurate spatial coordinates and texture information of an object surface. In some high-precision three-dimensional scanning processes, the distance measurement sampling point of the original object surface can be smaller than 0.01 millimeter. Such high-precision shape information can be fully considered as the only attribute that an article is hard to replicate. Moreover, through the support of related software, a user can arbitrarily rotate, move and zoom the digitized three-dimensional model on the screen, and even can measure the size of the model by using a size tool, so that the user can observe the object entity represented by the digitized three-dimensional model as if the user were personally on the scene and completely from the subjective view of the user.

The innovation of the invention is that users are completely given the initiative of knowing the articles to themselves, so that the users can watch and enjoy the digital three-dimensional model of the articles appearing in the video at will while watching the video. As long as users are interested in a certain article when watching the video, the users can click the article in the video picture at any time, and the digital three-dimensional model corresponding to the article one by one can be immediately displayed on the screen and the users can play and know according to own will, so that the problem of unidirectionality in the process of displaying and introducing the article in the traditional video is perfectly solved.

Disclosure of Invention

In order to solve the defect that a user can only know the appearance of an object in a video from the subjective view angle of a video photographer when watching the video, the invention provides an interactive video playing method and system combining digital three-dimensional model display with the video, wherein the system comprises the following specific components:

The system comprises a central processing unit, a video pre-analysis information repository, a video database, a three-dimensional model information database, a system information input port, an external information input port and a screen.

The central processing unit is the core of the whole system, has multiple functions and comprises:

A video analysis function, comprising: analyzing the video for interactive playing, and labeling the articles and parts thereof to be interacted in the video;

an interactive response function comprising: receiving an interaction instruction from a user and completing an interaction instruction task of the user;

a video play function comprising: playing the video and the video clip according to the requirement of the user, comprising: play, pause, fast forward and slow forward, fast backward and slow backward, skip play and close;

A digitized three-dimensional model presentation function comprising: moving, rotating, zooming, animation, sizing, component decomposition and component assembly of the three-dimensional model;

A retrieval function comprising: retrieving video and video clips related to the corresponding item according to the digitized three-dimensional model, retrieving similar models and similar items through the digitized three-dimensional model; the retrieval of external information of the system is realized through characters and pictures;

External resource function includes: and acquiring video, pictures, characters, voice and digital three-dimensional model information outside the system.

A video pre-analysis information repository, the functions of which include: the method comprises the steps of sorting and collecting video information analyzed by a central processing unit, enabling articles and parts appearing in the video to be in one-to-one correspondence with physical articles and parts in reality, enabling articles and parts appearing in the video to be in one-to-one correspondence with a high-fidelity digital three-dimensional model, and enabling articles and parts appearing in the video to be in one-to-one correspondence with background information.

The video database comprises the following functions: video to be played in the interactive video playback is stored.

The three-dimensional model information database comprises the following functions: storing the high fidelity digitized three dimensional model of the item appearing in the video along with its components, storing the background information of the item appearing in the video along with its components.

The system information input port comprises the following functions: input of the digitized three-dimensional model and input of background information corresponding to the digitized three-dimensional model one by one.

An external information input port whose functions include: accessing information resources outside the system, comprising: video, picture, text, speech, digitized three-dimensional model information.

A screen, the functions of which include an interface for a user to participate in an interactive video playing actual activity, the types of which include: computer screen, portable and panel computer screen, cell-phone screen, multi-screen concatenation screen, customization screen includes: strip screens, square screens, round screens and annular screens with different sizes which are customized according to the requirements; the types of the components and the combinations thereof include: single screen, double screen, multi-screen combinations; the interaction modes comprise: touch screen click, mouse click, remote control click.

An interactive video playing method combining digital three-dimensional model presentation with video, the method comprising:

Analyzing a section of video to be played, and selecting one or more objects on a video frame to mark the video frame;

techniques that need to be employed for this include:

The image segmentation technology is used for dividing an image into a plurality of mutually disjoint areas and then extracting a plurality of target objects which need to be mutually separated from other objects from the image;

The image matting technology aims to accurately and reliably extract edge details of an image foreground target, train and learn a large amount of data, and further extract high-level semantic features and low-level texture details of a complex image;

Video Object Segmentation (VOS) techniques, which function to identify and segment target instances from a video sequence, after a segmentation result of a first frame of a video is given, an algorithm needs to infer the segmentation results of the remaining frames;

a space-time memory network (STM) technique that can construct a memory bank for each object in the video, and match each query frame with the memory bank for "memory reading";

the Space-time correspondence network (Space-Time Correspondence Networks, STCN) technology is a network structure of direct image-to-image correspondence, which learns all object relationships except the marked object;

video matting techniques to accurately extract foreground objects from a sequence of video screens.

Preferably, real objects corresponding to the marked one or more objects are acquired.

Preferably, the digitalized three-dimensional scanning of the real object is obtained and stored in a three-dimensional model information database;

The method for acquiring the digital three-dimensional model comprises the following steps:

scanning by a three-dimensional scanner, modeling a sequence photo, and performing three-dimensional modeling by modeling software.

Preferably, the method for simultaneously inputting the background information related to the physical entity in the three-dimensional model information database comprises the following steps:

the special name of the object, the manufacturer and provider of the object, the function and function of the object, the component combination of the object, the material quality, the weight and the size of the object, the application scene of the object and the use method of the object.

Preferably, the background information is in one-to-one correspondence with the digitized three-dimensional model of the real object.

Preferably, the relationship between the marked object and the digital three-dimensional model of the corresponding object is established, and at the same time, an interactive function and a mechanism are established, so that when a user watches a video and clicks the image of the marked object or objects, the digital three-dimensional model corresponding to the marked object can be called out from the three-dimensional model information database and observed and enjoyed, and the method comprises the following steps:

randomly moving, rotating, scaling, measuring, disassembling and combining the digital three-dimensional model on a screen;

Panoramic viewing of the internal construction of the three-dimensional model;

viewing an animated presentation associated with the digitized three-dimensional model;

consulting background information associated with the model;

The method for realizing interactive video playing by clicking the video and digitizing the three-dimensional model by the user comprises the following steps:

Clicking a touch screen, clicking a mouse and clicking a remote control;

techniques that need to be employed for this include:

a three-dimensional model presentation aspect, comprising:

Open GL, a cross-language, cross-platform application programming for rendering 2D, 3D vector graphics, for drawing three-dimensional scenes from simple graphics to relatively complex;

Direct3D is a set of 3D drawing programming interface used for rendering graphics in a three-dimensional graphics application program with higher performance requirements;

The technology combines HTML5, javaScript and OpenGL ES, and provides hardware 3D accelerated rendering for HTML5 Canvas by adding one JavaScript binding of the OpenGL ES, and finally the three-dimensional model is smoothly displayed on a two-dimensional screen;

A three-dimensional rendering engine technique, comprising: the system comprises a scene management module, a message management module, a resource management module, a clock management module and a primitive management module;

the scene management module is mainly used for organizing and managing all elements in a scene, and related functions mainly comprise creating a camera, creating a scene node, creating an entity and creating a movable object;

The message management module is mainly used for coordinating the interrelationship of each module in the rendering engine, when a user operates a mouse or a keyboard, the message management module sends a message to an operation system message buffer queue, and meanwhile, a related function is constructed by acquiring related messages, so that the coordination related module processes the received message;

the resource management module is mainly used for managing information such as resource files, configuration files and the like in the running process of the system;

The clock management module is mainly used for counting and controlling the frame rate by setting a clock;

the primitive management module is mainly used for drawing complex graphics through management of basic primitives;

A video playback aspect, comprising:

Video coding techniques for coding and compressing a series of frame images and audio, comprising: encoding video images, encoding audio and packaging audio and video;

The video decoding technology aims at decoding the coded video screen packaging format and the coded format, and restoring the video into a series of images and audios so as to render and output;

A video player management technique comprising:

the play state management technology is used for enabling the player to better manage various complex conditions when the decoded video screen is played, avoiding the occurrence of the condition of disordered player state, and needing to carry out state management on the player so as to facilitate the interactive operation between the user and the player;

video buffering and segmentation management technology is used for improving video watching smoothness and saving bandwidth resources of users;

Network state monitoring management aims to avoid the loss of a large amount of traffic of a user under the condition of unknowing, and prompts the user of the network conditions used by the user in different view modes by setting network monitoring;

The touch gesture management technology is used for carrying out touch gesture management on the player so as to increase the convenience of interaction between the video and the user;

the screen view management technology can enable a user to randomly select the functions of small screen viewing, large screen viewing, full screen viewing, multi-view screen viewing, random movement of the window position of the view screen and the like when watching the view screen, and the requirements of individuation and diversity when watching the view screen are met by designing the diversity display function of the view;

gravity sensing and screen lock management techniques for satisfying user viewing screen requirements in different poses, comprising: designing the gravity sensing and screen locking functions of the player, including: gravity sensing is achieved by monitoring the change of the offset angle of the equipment; the screen viewing mode is rotated into a horizontal screen mode, a vertical screen mode and a reverse horizontal screen mode through the change of the offset angle of the equipment; screen locking may be achieved by turning off listening.

In practical applications of the present technology, a screen capable of implementing a digital three-dimensional model presentation in combination with video in an interactive video play mode includes:

Computer screen, portable and panel computer screen, cell-phone screen, multi-screen concatenation screen, customization screen includes: strip screens, square screens, round screens and annular screens with different sizes which are customized according to the requirements.

If desired, one or more videos may appear simultaneously on the same screen or a combination of screens, along with one or more digitized three-dimensional models, including:

the playing of different videos is put in the same picture;

the playing of different videos is put in different pictures;

The observation and appreciation of different digital three-dimensional models are put in the same picture;

The observation and appreciation of different digital three-dimensional models are put in different pictures;

The observation and appreciation of the digital three-dimensional model are put in the same picture as the video playing;

The observation and appreciation of the digitized three-dimensional model are placed in a different picture than the video play.

According to the requirement, the playing state of the video when the user enjoys, looks and rubs the digital three-dimensional model comprises:

Pause, continue playing, video close.

If the user needs to click more gently to the object of interest during the implementation of the technical scheme of the invention, the video can be slowly played and paused so that the user can click the object in the video in the process of playing the video at a relatively slow speed or aiming at a still picture.

If the user needs to review one or more fragments of the object represented by the digital three-dimensional model in the video, the video can be searched through the digital three-dimensional model, and the one or more video fragments of the object are called out and selected and clicked for playback by the user.

If a user initially wants to know what one or more items are present in the video to be played that are engaged in the interactive play, the method of solving the problem comprises:

clicking and opening a special window for displaying the digital three-dimensional model before playing the video, wherein all the digital three-dimensional models in the window correspond to the objects or parts thereof which are about to appear in the video one by one;

if necessary, the user may observe and enjoy the part or all of the digitized three-dimensional model before playing the video, including:

Randomly moving, rotating, zooming, measuring, disassembling and combining the three-dimensional model on a screen;

a panoramic viewing model internal structure;

animation presentation associated with a digitized three-dimensional model;

The background information associated with the model is consulted.

Drawings

Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. In the drawings:

FIG. 1 illustrates a flow chart of an interactive video playback method incorporating a digitized three-dimensional model presentation;

Fig. 2 shows a system overall operation diagram of an interactive video playing method incorporating a digitized three-dimensional model presentation.

Detailed Description

Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present invention may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

"One hundred Hammerett in one hundred eyes". This means that people tend to see the same thing from different angles, which is in fact a difficulty that video creators are faced with. It is well known that video is a measure of its spread in units of time, and if all aesthetic preferences are met for a good, a video of an item will become less lengthy and uninteresting. Therefore, the creator of the video can only shoot the object from the best angle he/she understands, while editing the video with the best viewing effect he/she sees. In this way, it is inevitable that users who want to enjoy a certain object in the video more are hesitant.

In the past, besides allowing the user to search the database after watching the video, to see whether other videos or photographic pictures introducing the same object exist, no better solution to the problem exists.

In recent years, the appearance of a high-precision and high-fidelity digital three-dimensional model provides people with the opportunity to enjoy everything on the spot without going out of the home, and also brings new opportunities for more comprehensive and thorough video object display efforts. If a user is interested in an item in a video while watching it, they simply click on the item in the video, and a digitized three-dimensional model that can reproduce the entity of the item in all directions with high fidelity will appear on the same screen on which the video is played. At this time, the user can move, rotate, scale and measure the three-dimensional model according to own preference, and even can obtain a series of background information related to the object through expansion, so that the requirement of the user for further and deep understanding of the object can be completely met.

Meanwhile, due to the existence of the functions, the strategy of shooting and making of the video can be properly adjusted, a video creator can not tangle whether the shot object has the appearance and the details of the object or not, but can focus on how to show the advantages of the object, and even if the shot object does not need to be aligned to the object. Sometimes groups of atmosphere shots depicting the use of an item are instead more capable of exhibiting the charm of the item.

The technical difficulty faced by the user is to perfectly integrate the video playing technology and the digital three-dimensional model displaying technology into the same screen interface, and simultaneously, the user can control and drive the video playing and the digital three-dimensional model displaying with the maximum degree of freedom, so that the user can know the displayed objects in the interactive video to the greatest extent, and the key point of the specific embodiment of the invention is that.

The method of the invention comprises two main processes: (A) Preprocessing a video and preparing a digital three-dimensional model; (B) And a stage of video playing and user interaction in the process of watching video. The following describes the two processes:

A. pretreatment of video and preparation stage of digital three-dimensional model

A1. Image analysis is carried out on a section of video to be played, and one or more articles appearing in the video are selected to be used as interaction objects when a user watches the video;

A2. Labeling the selected articles so as to be clicked by a user and make interactive response when the video is played;

A3. Acquiring a physical object identical to a selected object in a video;

A4. acquiring a digital three-dimensional model of the obtained physical object;

A5. storing the obtained digital three-dimensional model into a three-dimensional model information database;

A6. Collecting background information related to an item selected in the video, comprising: the special name of the object, the manufacturer and provider of the object, the function and function of the object, the component combination of the object, the material quality, the weight and the size of the object, the application scene of the object and the use method of the object;

A7. storing the background information into a three-dimensional model information base, and corresponding the background information to the described articles and the digital three-dimensional model obtained by the articles one by one;

A8. And establishing an interactive response function and mechanism in video playing.

B. Stage of video playing and user interaction in video watching process

B1. The user plays the video;

B2. clicking on an item in a video when a user sees the item in the video that he is interested in and has a need for further knowledge of his shape, appearance, details, size, and background information;

If the user needs to click more gracefully to the item of interest, the video can be slowly played, paused, so that the user clicks the item in the video during the process of playing the video at a slower speed or aiming at a still picture;

B3. And calling out the digital three-dimensional models corresponding to the clicked objects one by one from the database and displaying the digital three-dimensional models on a screen. The user-selectable locations for placement of the digitized three-dimensional model include: in the same frame as the video, in another frame separate from the video;

when the digitized three-dimensional model is called out, the user can autonomously select the playing mode of the video, which comprises the following steps: pausing, continuing playing and closing the video;

B4. No matter what play mode the video is in, no matter what position the digital three-dimensional model is in the screen, the user can watch and enjoy the digital three-dimensional model at will, including:

randomly moving, rotating, scaling, measuring, disassembling and combining the digital three-dimensional model on a screen;

Panoramic viewing of the internal construction of the three-dimensional model;

viewing an animated presentation associated with the digitized three-dimensional model;

consulting background information associated with the model;

B5. if the user needs to review, preview and repeatedly watch the video clips related to the articles which the user enjoys, searching and calling all the video clips related to the articles through the digital three-dimensional model of the articles, and selecting the clips which the user is interested in for playing and watching;

the playing mode of the newly called video clip comprises the following steps:

replacing the previously played main video, and playing in a video frame of the previously played video;

Playing in a picture-in-picture mode in a video frame of a previously played main video;

playing in a new frame of view that is different from the previously played main video;

when a new video clip is played, the playing state of the main video includes: pause, continue playing, and close;

B6. when a user finds a new object of interest in the process of playing the video, the user can continuously click on the object in the video and call out a new digital three-dimensional model for viewing and appreciation;

B7. If the user suddenly thinks that the object which is interested but not clicked exists in the video which is already played in the process of watching the video, the user can play back the video to search the object which is interested, and call out a new digital three-dimensional model for viewing and appreciation;

B8. If the user wants to quickly find the interested object in the video watching process, the user can quickly forward the video or skip the video to find the interested object, and call out a new digital three-dimensional model for viewing and appreciation;

B9. if the user clicks and calls more than 2 digitized three-dimensional models in the process of interactively watching the video, the method for processing the digitized three-dimensional models comprises the following steps:

Closing the display of the part of the digitized three-dimensional model;

displaying all the clicked and called digital three-dimensional models on a screen;

Temporarily shortening or hiding part or all of the digital three-dimensional model, and recovering all display functions when the model needs to be continuously enjoyed and observed;

All the displayed digital three-dimensional models are closed;

B10. If necessary, the user can uniformly place all the digitalized three-dimensional models which are called from the database in the same window for selection, and when the three-dimensional models need to be selected for appreciation, the window is opened, and one or more three-dimensional models are clicked; hiding the window when the selection is finished or the selection is not needed;

B11. if necessary, the user searches the digital three-dimensional model of the article similar to the article in the video in the external database through the called digital three-dimensional model to observe, compare and enjoy.

The interactive video playing method combined with the digital three-dimensional model display implemented by applying the inventive concept of the present invention can have various application scenarios, and 2 of the following specific descriptions are provided:

Example 1

Five new products are newly introduced by an electric appliance company, and in order to better play the effect of video popularization, the company adopts the interactive video playing method combined with the digital three-dimensional model display

S1, shooting a video, wherein a housewife introduces five new products one by one in a manner of teaching, displaying and demonstration in the video;

s2, analyzing the shot video, and determining articles which can participate in click interaction of a user in the video, namely five electric products and parts thereof;

S3, acquiring the high-fidelity digital three-dimensional models of the five electric products and the parts thereof which need to participate in click interaction of the user, and storing the high-fidelity digital three-dimensional models into a digital three-dimensional model database;

S4, simultaneously inputting background information related to the five electric products and parts thereof into a three-dimensional model database;

s5, five electric products and parts thereof stored in the digital three-dimensional model database are in one-to-one correspondence with the products and parts thereof in the video, wherein the background information of the five electric products and the parts thereof comprises the background information;

S6, according to the technical scheme provided by the invention, an interactive video is manufactured around the five electric products and the parts thereof;

s7, the user starts to play the video;

S8, when the user sees that the host starts introducing to the third electric product, the user generates a thicker interest in the product, but because the video cannot give the full view of the product, the user pauses the video and clicks the product image on the video, and the digital three-dimensional model corresponding to the product one by one is displayed on the screen;

S9, the user moves the digital three-dimensional model into a view frame different from the played video to watch and enjoy, and the video is still paused;

s10, the user performs actions such as movement, rotation, scaling and the like on the digital three-dimensional model, and measures the size of the model corresponding to a product entity by using a measuring tool, so that the appearance and details of a third electrical product are deeply known;

s11, the user clicks and refers to the background information of the third electric appliance, wherein the method comprises the following steps: the functions, materials, weight, size, related application scenes and using methods of the product;

S12, after the user fully knows the third electric appliance, still placing the third digital three-dimensional model in an independent view frame on the screen, and then continuing to play the video;

S13, when a user sees a fifth electrical product in the video, finding that the electrical product is similar to the first electrical product in the video, curiously clicking the fifth electrical product in the video, and displaying a digital three-dimensional model corresponding to the electrical products on a screen;

S14, the user plays back the video quickly, and when the user finds out the first electric appliance product in the video, the user clicks the first electric appliance picture in the video, and then a digital three-dimensional model corresponding to the first electric appliance product one by one is displayed on the screen;

S15, a user drags two digital three-dimensional models respectively representing a first electric product and a fifth electric product into the same view frame for viewing, appreciation and comparison, wherein the method comprises the following steps of:

moving, rotating, zooming, decomposing a digital three-dimensional model, observing the inside of the component and the model, measuring the size, performing drilling assembly animation and reading background information;

S16, in the process of viewing and enjoying the digital three-dimensional model, the user feels that all video fragments related to the first electrical product and the fifth electrical product in the video are necessary to be seen once again, then a special button attached to the edge of the digital three-dimensional model is clicked, all video fragments related to the first electrical product and the fifth electrical product in the whole video are listed on the screen, and then the user selects the fragments of interest to see for several times;

s17, after appreciation and comparison of the two digital three-dimensional models, the user finishes watching the rest video;

S18, then, all other windows displayed on the screen are hidden by the user, the video is seen again in full screen form, and the knowledge of the five products introduced in the video is further enhanced.

Example 2

An exercise equipment company decides to make an interactive video for better guiding its clients to install an indoor horizontal bar on which it is sold, by performing the following steps:

S1, shooting a video, wherein a body-building coach systematically introduces a way of installing the horizontal bar step by step in a way of teaching, displaying and demonstrating;

s2, analyzing the photographed video, and determining articles which can participate in click interaction of a user in the video, namely all parts and fasteners of the indoor horizontal bar;

s3, acquiring high-fidelity digital three-dimensional models of all parts and fasteners of the indoor horizontal bar which need to participate in user interaction, and storing the high-fidelity digital three-dimensional models into a digital three-dimensional model database;

s4, simultaneously inputting background information related to all parts and fasteners of the indoor horizontal bar into a three-dimensional model database;

s5, all parts and fasteners of the indoor horizontal bar stored in the digital three-dimensional model database are in one-to-one correspondence with all parts and fasteners of the indoor horizontal bar in the video, wherein the background information of the parts and fasteners comprises the background information of the parts and fasteners;

S6, according to the technical scheme provided by the invention, an interactive video is manufactured around how the indoor horizontal bar is installed;

s7, starting application of the interactive video by the user;

S8, clicking a button of a digital three-dimensional model overview before a user plays the video, and immediately displaying a window on the screen, wherein the digital three-dimensional model of a component and a fastener of the horizontal bar which are about to appear in the video is displayed in the window in sequence;

s9, the user familiarizes the digitized three-dimensional model in the view frame one time, and the specific method comprises the following steps: rotation, movement, scaling, size measurement, combination of adjacent parts and structure animation;

S10, the user compares the digital three-dimensional model in the window with the horizontal bar component and the fastener in the hand of the user, so that the user has a preliminary impression on the forthcoming image in the real object and the video and the digital three-dimensional model possibly called out immediately;

S11, temporarily hiding windows for displaying all the digitized three-dimensional models in a total mode by a user so as to play the video in a full screen mode;

s12, the user starts to play the video;

s13, when a user sees that a third component is installed, a user suddenly and somewhat knows which object the component is specifically, so that the user clicks the component in the video, and a digital three-dimensional model of the component appears on the screen immediately;

s14, in order to facilitate deep research on the digital three-dimensional model of the component, a user pauses a video in playing;

S15, a user carefully researches the digital three-dimensional model of the component, and simultaneously finds a corresponding component in the entity component;

s16, observing the composition relation and the installation mode of the component and the adjacent component by using an animation tool through a digital three-dimensional model of the component;

S17, temporarily hiding the picture frame for displaying the digital three-dimensional model by a user, and continuing watching the video;

s18, when the user just gets to see the tail sound, suddenly perceiving that the second component is not well understood, and then returning to the video picture of the second component in the early stage of the video through skip play and fast-rewinding modes;

S19, the user pauses the video and clicks a second component on the still picture;

s20, immediately displaying a digital three-dimensional model of the second component on the screen;

S21, the user searches all video clips related to the second component in the video through the auxiliary button click while carefully observing the digital three-dimensional model of the second component;

S22, a user understands the action and the installation method of the second component by watching each video clip;

s23, temporarily hiding the digital three-dimensional model by a user, and continuing watching the video;

s24, the user starts watching the video from the middle section of the video in a skip broadcasting and fast forward mode;

S25, after the user finishes watching the video, the user completely understands the installation method of the indoor horizontal bar;

S26, the user installs the indoor horizontal bars part by part while playing the video, and finally the installation task is completed in a very short time.

It should be noted that:

The required structure for constructing such a system arrangement is apparent from the above description. In addition, the present invention is not directed to any particular programming language. It will be appreciated that the teachings of the present invention described herein may be implemented in a variety of programming languages, and the above description of specific languages is provided for disclosure of enablement and best mode of the present invention.

In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.

Similarly, it should be appreciated that in the above description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be construed as reflecting the intention that: i.e., the claimed invention requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this invention.

Those skilled in the art will appreciate that modules in the apparatus of the embodiments may be adaptively changed and disposed in one or more apparatuses different from the embodiments. The modules or units or components of the embodiments may be combined into one module or unit or component and, furthermore, they may be divided into a plurality of sub-modules or sub-units or sub-components. Any combination of all features disclosed in this specification, and all processes or units of any method or apparatus so disclosed, may be employed, except that at least some of such features and/or processes or units are mutually exclusive. Each feature disclosed in this specification may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.

Furthermore, those skilled in the art will appreciate that while some embodiments described herein include some features but not others included in other embodiments, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.

Various component embodiments of the invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that some or all of the functions of some or all of the components in the creation means of a virtual machine according to an embodiment of the present invention may be implemented in practice using a microprocessor or Digital Signal Processor (DSP). The present invention can also be implemented as an apparatus or device program (e.g., a computer program and a computer program product) for performing a portion or all of the methods described herein. Such a program embodying the present invention may be stored on a computer readable medium, or may have the form of one or more signals. Such signals may be downloaded from an internet website, provided on a carrier signal, or provided in any other form.

It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware.

The present invention is not limited to the above-mentioned embodiments, and any changes or substitutions that can be easily understood by those skilled in the art within the technical scope of the present invention are intended to be included in the scope of the present invention.

Claims (5)

1. An interactive video playing method combined with digital three-dimensional model display, which is characterized by comprising the following steps:

Analyzing a section of video to be played, and selecting one or more objects on a video frame to mark the video frame;

Acquiring real objects corresponding to the marked one or more objects;

Acquiring a digital three-dimensional model of the real object and storing the digital three-dimensional model into a three-dimensional model information database;

simultaneously inputting background information related to the real object into a three-dimensional model information database, wherein the method comprises the following steps of:

The special name of the object, the manufacturer and provider of the object, the function and function of the object, the component combination of the object, the material quality, the weight and the size of the object, the application scene of the object and the use method of the object;

the background information is in one-to-one correspondence with the digital three-dimensional model of the real object;

The interactive response function and mechanism are established, so that when a user watches videos and clicks the images of the marked one or more objects, the digitalized three-dimensional model of one or more objects corresponding to the video can be called out from the three-dimensional model information database, and the user looks and enjoys the digitalized three-dimensional model, and the interactive response function and mechanism comprise the following steps:

randomly moving, rotating, scaling, measuring, disassembling and combining the digital three-dimensional model on a screen;

Panoramic viewing of the internal construction of the three-dimensional model;

viewing an animated presentation associated with the digitized three-dimensional model;

The background information associated with the model is consulted.

2. The method of claim 1, wherein the method for obtaining the digitized three-dimensional model comprises:

Three-dimensional scanning, video or sequential image modeling, three-dimensional modeling.

3. The method according to claim 1, characterized in that it comprises:

One or more videos may appear simultaneously in the same screen or in a combined screen, along with one or more digitized three-dimensional models, including:

the playing of different videos is put in the same picture;

the playing of different videos is put in different pictures;

The observation and appreciation of different digital three-dimensional models are put in the same picture;

The observation and appreciation of different digital three-dimensional models are put in different pictures;

The observation and appreciation of the digital three-dimensional model are put in the same picture as the video playing;

The observation and appreciation of the digitized three-dimensional model are placed in a different picture than the video play.

4. The method of claim 1, wherein the implementation of viewing and appreciation of the video and digitized three-dimensional model comprises:

The method is implemented on-line through the Internet and off-line on a local computer.

5. An interactive video playback system for presentation in conjunction with a digital three-dimensional model, comprising: the interactive video playing system combined with the digital three-dimensional model display consists of a central processing unit, a video pre-analysis information repository, a video database, a three-dimensional model information database, a system information input port, an external information input port and a screen;

the central processing unit is the core of the whole system, has multiple functions and comprises:

A video analysis function, comprising: analyzing the video for interactive playing, and labeling the articles and parts thereof to be interacted in the video;

an interactive response function comprising: receiving an interaction instruction from a user and completing an interaction instruction task of the user;

a video play function comprising: playing the video and the video clip according to the requirement of the user, comprising: play, pause, fast forward and slow forward, fast backward and slow backward, skip play and close;

A digitized three-dimensional model presentation function comprising: moving, rotating, zooming, animation, sizing, component decomposition and component assembly of the three-dimensional model;

A retrieval function comprising: searching related videos and video fragments according to the digital three-dimensional model, searching similar three-dimensional models and similar articles through the digital three-dimensional model, and searching external information of the system through characters and pictures;

external resource function includes: acquiring video, pictures, characters, voice and digital three-dimensional model information outside the system;

A video pre-analysis information repository, the functions of which include: sorting and collecting video information analyzed by a central processing unit, enabling articles and parts thereof to be in one-to-one correspondence with physical articles and parts thereof in reality, enabling articles and parts thereof to be in one-to-one correspondence with a high-fidelity digital three-dimensional model, and enabling articles and parts thereof to be in one-to-one correspondence with background information;

The video database comprises the following functions: storing video to be played in the interactive video playing;

The three-dimensional model information database comprises the following functions: storing a high-fidelity digitized three-dimensional model in one-to-one correspondence with the items appearing in the video and the components thereof, and storing background information of the items appearing in the video and the components thereof;

the system information input port comprises the following functions: input of a digital three-dimensional model and input of background information corresponding to the digital three-dimensional model one by one;

An external information input port whose functions include: accessing information resources outside the system, comprising: video, picture, text, voice and digital three-dimensional model information;

A screen, the functions of which include an interface for a user to participate in an interactive video playing actual activity, the types of which include: computer screen, portable and panel computer screen, cell-phone screen, multi-screen concatenation screen, customization screen, include: strip screens, square screens, round screens and annular screens with different sizes which are customized according to the requirements; the combination mode comprises the following steps: single screen, double screen, multi-screen combinations; the interaction modes comprise: touch screen click, mouse click, remote control click.