CN113691779B - Method for enhancing presentation mode in AR system - Google Patents

Method for enhancing presentation mode in AR system Download PDF

Info

Publication number
CN113691779B
CN113691779B CN202110984031.5A CN202110984031A CN113691779B CN 113691779 B CN113691779 B CN 113691779B CN 202110984031 A CN202110984031 A CN 202110984031A CN 113691779 B CN113691779 B CN 113691779B
Authority
CN
China
Prior art keywords
video
view
scene
certain
presentation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110984031.5A
Other languages
Chinese (zh)
Other versions
CN113691779A (en
Inventor
冯志全
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
University of Jinan
Original Assignee
University of Jinan
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by University of Jinan filed Critical University of Jinan
Priority to CN202110984031.5A priority Critical patent/CN113691779B/en
Publication of CN113691779A publication Critical patent/CN113691779A/en
Application granted granted Critical
Publication of CN113691779B publication Critical patent/CN113691779B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00Television systems
    • H04N7/18Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/65Transmission of management data between client and server
    • H04N21/658Transmission by the client directed to the server
    • H04N21/6587Control parameters, e.g. trick play commands, viewpoint selection
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60Control of cameras or camera modules
    • H04N23/69Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Studio Devices (AREA)

Abstract

The invention provides a method for enhancing a presentation mode in an AR system, which comprises the following steps: s1: shooting at least one complete scene presentation video by using a camera, wherein for any operation object in each video, the video comprises two or more operation visual angles; s2: constructing a video library by utilizing the videos in the step S1; s3: the operator enters a scene to operate, and a view angle conversion request is sent out at any time point; s4: after receiving the view angle conversion request, the system acquires video clips meeting the view angle conversion request from a video library; s5: the system covers the acquired video clips on the original video stream; s6: and when the coverage state exits, restoring the scene presentation of the original video stream. On the premise of not increasing the number of cameras, an operator can perceive information of the interactive scene from different visual angles, and the perception effect which can be achieved only by moving the cameras is achieved.

Description

Method for enhancing presentation mode in AR system
Technical Field
The invention relates to the technical field of AR (augmented reality), in particular to a method for enhancing a presentation mode in an AR system.
Background
In an AR (augmented reality) system, the view angle of an operator is derived from a picture photographed by a camera, and in the case that the camera is fixed, the view angle of the operator is fixed, and in order to see different view angles, two methods are available at present:
one is to increase the number of cameras, and as the pictures shot by different cameras are different, the purpose of switching the visual angles can be achieved by switching the cameras;
the other is to construct a track, so that the camera can move along the track, and the aim of switching the visual angle is fulfilled from a fixed visual angle to a moving visual angle.
For both approaches, adding a camera or a track significantly increases the hardware cost of the system, and not only does this require the development of specialized software for the hardware to match the functions of the hardware, further increasing the cost of the AR system.
Disclosure of Invention
In order to solve the above problems, the present application provides a method for enhancing a presentation mode in an AR system, in which an operator can perceive information of an interactive scene from different view angles without increasing the number of cameras, so as to achieve a perception effect that can be achieved only by moving the cameras, and even for the same experimental phenomenon, the operator can have different operation experiences.
The invention discloses the following technical scheme:
the embodiment of the invention provides a method for enhancing a presentation mode in an AR system, which comprises the following steps:
s1: shooting at least one complete scene presentation video by using a camera, wherein for any operation object in each video, the video comprises two or more operation visual angles;
s2: constructing a video library by utilizing the videos in the step S1;
s3: the operator enters a scene to operate, and a view angle conversion request is sent out at any time point;
s4: after receiving the view angle conversion request, the system acquires video clips meeting the view angle conversion request from a video library;
s5: the system covers the acquired video clips on the original video stream;
s6: and when the coverage state exits, restoring the scene presentation of the original video stream.
Further, in step S1, the method for shooting the scene to present video includes, but is not limited to:
simulating a plurality of visual angles by using a plurality of fixed cameras to shoot; or (b)
Simulating a motion visual angle by using a single motion camera to shoot; or (b)
And simulating a plurality of movement visual angles by using a plurality of movement cameras to shoot.
Further, in step S2, specific methods for constructing the video library include, but are not limited to:
directly storing the shot complete video to form a video library; or (b)
And splitting the shot complete video into a plurality of video frequency bands, and storing to form a video library.
Further, in step S3, the method for issuing the view angle conversion request includes, but is not limited to:
sending a visual angle conversion request by utilizing a voice input mode; or (b)
And sending out a view angle conversion request in a scene interaction mode.
Further, the view angle transformation request content at least comprises one of an operation object, an operation time and an operation view angle.
Further, in step S4, when the video library is composed of complete videos, the system intercepts video clips conforming to the view angle transformation request from the complete videos; when the video library is composed of video clips, the system selects video clips from the video clips that meet the view transformation request.
Further, in step S5, after the overlay is completed, the original video stream still exists, but the operator can only see the overlaid video clip.
Further, in step S6, the manner of exiting the coverage state includes, but is not limited to:
the video clip obtained in the step S4 is played completely; or (b)
The operator initiates a new view angle conversion request; or (b)
The operator initiates a request to end the coverage status.
The invention has the beneficial effects that:
compared with the prior art, the AR system needs to be improved, so that the visual angle switching method is realized in the operation process, the scene is shot only before the operation starts, after the operation starts, an operator can perceive information of an interactive scene from different visual angles on the premise of not changing the AR system architecture, the perception effect which can be achieved by a mobile camera is achieved, and even for the same experimental phenomenon, the operator can have different operation experiences.
Drawings
FIG. 1 is a flow chart of an embodiment of the method of the present invention.
Detailed Description
In order to make the technical problems, technical solutions and advantages to be solved more apparent, the following detailed description will be given with reference to the accompanying drawings and specific embodiments. It will be appreciated by those of skill in the art that the following specific embodiments or implementations are provided as a series of preferred arrangements of the present invention for further explanation of the specific disclosure, and that the arrangements may be used in conjunction or association with each other, unless it is specifically contemplated that some or some of the specific embodiments or implementations may not be associated or used with other embodiments or implementations. Meanwhile, the following specific examples or embodiments are merely provided as an optimized arrangement, and are not to be construed as limiting the scope of the present invention.
As shown in fig. 1, an embodiment of the present invention provides a method for enhancing a presentation manner in an AR system, where the method includes:
s1: and shooting at least one complete scene presentation video by using the camera.
When a scene is presented, two or more operation views are required to be included in a captured video for any one of operation objects in the scene. In general, the more the operation view angle of the operation object is, the better the enhanced effect is finally presented.
When the shooting scene presents video, a single camera can be adopted for shooting, and a plurality of cameras can be adopted for shooting.
If a single camera is used for shooting, the camera which moves when the camera needs to be used can achieve the effect of visual angle switching, and one shooting scheme which can be realized is as follows:
and constructing a spherical space, requiring the spherical space to completely cover the operation scene, setting a longitude orbit and a latitude orbit along the surface of the spherical space, enabling a camera to move along the longitude orbit and the latitude orbit, and traversing the surface of the whole spherical space, so that shooting of scene presentation can be realized.
It should be noted that the spherical space, the longitude track, and the latitude track are merely for convenience of description, and the frame of the entity is not required to be built, and the camera may move on the route represented by the longitude track and the latitude track.
In practical operation, the space may be square, trapezoid, cone, etc. and may be even one irregular space, or may be not constructed, and the design of the camera motion path may not have specific requirement, and only the same operation object may be operated in multiple views.
If multiple cameras are utilized, the cameras may be stationary or moving.
When utilizing a plurality of fixed cameras to shoot, compare in prior art, not just leading a plurality of fixed cameras to the preparation stage from the operation process, the purpose of this application is the video library of production, after the video library produced, can infinitely many times use, reaches the purpose of "shooting once, permanently uses", and prior art shoots in real time and switches the visual angle, and AR system's software and hardware development cost and maintenance cost are far above this application.
When shooting is performed by using a plurality of moving cameras, the shooting principle is similar to that of a single moving camera, but compared with the single moving camera, the plurality of moving cameras can present a plurality of visual angles to the same operation object at the same time point.
S2: and constructing a video library by utilizing the videos in the step S1.
When the video library is constructed, the shot complete video can be directly stored to form the video library, or the shot complete video can be stored to form the video library after being split into a plurality of video frequency bands.
It should be noted that when the complete video is split, the complete video may be split into a plurality of time units in an equally-divided manner, or may be split according to different main operation objects in a picture, for example, the main operation object in the video segment 1 is a, the duration is 2 minutes, the main operation object in the video segment 2 is B, and the duration is 1 minute.
S3: the operator enters the scene to operate, and issues a viewing angle conversion request at any time point.
After the operator enters the scene to operate, what the operator sees is the original video stream, namely the real-time picture, and at any time point when the original video stream exists, the operator can send out a viewing angle conversion request, and the viewing angle conversion request content at least comprises one of an operation object, an operation time and an operation viewing angle.
If the view transformation request content only has an operation object, the view transformation request content represents an unlimited time and an unlimited view, and if the view transformation request content is a video picture containing the operation object, the request is a request which is intended by us, and in fact, the whole process of a certain operation object is changed.
If the view transformation request content has only an operation time, it indicates that a certain point in time or a certain period of time is requested, and all operation objects and all video clips of the operation views in the whole scene, in fact, a certain point in time or a certain period of time, the real-time state of the whole scene.
If the view transformation request content has only an operation view, it indicates that a complete video of all operation objects in the entire scene is requested under the view, and in fact, the entire process of the entire scene is changed under a certain view.
If the view angle conversion request content contains an operation object and operation time, a certain time point or a certain time period is requested, and the state of the operation object under different view angles is determined.
If the view transformation request content contains an operation object and an operation view, the whole process of the operation object is changed under a certain view.
If the view transformation request content contains operation time and operation view, the request is a certain time point or a certain time period, and the real-time state of the whole scene under a certain view is indicated.
If the view transformation request content includes an operation object, an operation time, and an operation view, only one screen, i.e., a certain time point or a certain time period, is actually used, and the real-time state of a certain operation object at a certain view is determined.
For the manner of sending the view angle conversion request, the view angle conversion request can be sent by using a voice input manner or by using a gesture and other scene interaction manner.
S4: after receiving the view angle conversion request, the system acquires a video clip meeting the view angle conversion request from the video library in combination with the analysis of the actual requirement of the operator in the step S3.
It should be noted that when the video library is composed of complete videos, the system intercepts video clips conforming to the view angle transformation request from the complete videos; when the video library is composed of video clips, the system selects video clips from the video clips that meet the view transformation request.
S5: the system overlays the acquired video clip over the original video stream. After the overlay is completed, the original video stream and the overlaid video clip coexist, and the operator can only see the overlaid video clip.
S6: and when the coverage state exits, restoring the scene presentation of the original video stream.
The manner in which the coverage state exits includes, but is not limited to: finishing the video clip playing; the operator initiates a new view angle conversion request; the operator initiates a request to end the coverage status.
It should be noted that, after the operator initiates a new view angle conversion request, the scene presentation of the original video stream is resumed and the new video clip meeting the requirements is obtained simultaneously, but because the scene presentation operation of the original video stream is resumed is simpler, the sequence on the picture presentation is as follows: the scene presentation of the original video stream is restored first and then overlaid as a new view transform request picture.
Based on the above principle, a specific embodiment is provided to represent more possibilities of the present application:
1) Shooting experimental process videos of a complete experiment from N different viewpoints simultaneously by adopting a plurality of cameras to form a video library;
2) Equally dividing each video into M identical time units, wherein the time interval of each unit is delta, and forming a node;
3) The original video stream is decomposed according to delta to form nodes;
4) An operator can input the system through voice, gestures and the like at any time to tell the system the viewpoint position to be observed;
5) The system finds a node S with the same length as the current video node in the database;
6) Covering the original video stream with a node S;
7) The operator ends the overlay presentation with voice and gesture instructions to resume the scene presentation of the original video stream.
Specific examples that may occur when the above principles are applied to experimental operation based on AR systems are as follows:
example 1
In a virtual-real fusion AR chemistry experiment system, there is a fixed camera to acquire the real-time scene of the operator's operation, which is fused with a computer-generated virtual experiment device (e.g., virtual beaker) to form an AR experiment system. When the operator starts to ingest the chemical, the system automatically imports a pre-recorded video, which 360 degrees presents the process of ingesting the chemical as if the operator were operating by himself.
Example 2
In the sodium water reaction AR experiment, a fixed camera is adopted to acquire scene information. When an operator puts the virtual sodium block into the virtual beaker, the phenomenon of the sodium water chemical reaction process is a key phenomenon which needs to be observed by the operator. Therefore, in the experimental process of observing the sodium water reaction, a video is randomly covered, and an operator observes several key experimental phenomena such as floating, swimming, ringing, melting, red and the like from different angles such as overlooking, encircling and the like in an all-around manner. Since the enhanced video overlaid by each experiment is not necessarily the same, the phenomena and experience observed by the operator in each experiment are not necessarily the same.
Example 3
In the limestone and dilute hydrochloric acid experiment, after the dilute hydrochloric acid is poured into the limestone, in order to clearly observe the key phenomenon that 'limestone disappears and bubbles' are generated, an experimenter can select a conical flask where the limestone is positioned on a screen by using a 'pointing gesture', then the experimenter makes a 'rotating gesture', after the system recognizes the gesture, the conical flask automatically and slowly rotates along a central shaft, and the experimenter can observe the experimental phenomenon from different angles; after an experimenter makes an 'enlarged gesture', the conical flask is enlarged on a screen, and the experimenter can observe experimental phenomena more clearly.
It should be noted that the above-described embodiments will enable those skilled in the art to more fully understand the specific structure of the present invention, but do not limit the invention in any way. Thus, while the specification and drawings and examples have been described in detail, it will be understood by those skilled in the art that the invention may be modified or equivalents; all technical solutions and modifications thereof which do not depart from the spirit and scope of the invention are included in the protection scope of the invention.

Claims (6)

1. A method for enhancing a presentation in an AR system, the method comprising:
s1: shooting at least one complete scene presentation video by using a camera, wherein for any operation object in each video, the video comprises two or more operation visual angles;
when shooting is carried out by utilizing a plurality of moving cameras, the plurality of moving cameras present a plurality of visual angles to the same operation object at the same time point, wherein the shooting scheme of each camera is as follows:
constructing a spherical space, requiring the spherical space to completely cover an operation scene, setting a longitude orbit and a latitude orbit along the surface of the spherical space, enabling a camera to move along the longitude orbit and the latitude orbit, traversing the surface of the whole spherical space, and thus realizing shooting of scene presentation;
s2: constructing a video library by utilizing the videos in the step S1;
s3: the operator enters a scene to operate, and a view angle conversion request is sent out at any time point;
after an operator enters a scene to operate, the operator sees an original video stream, namely a real-time picture, and can send a visual angle conversion request at any time point when the original video stream exists;
if the view angle transformation request content only has operation objects, the whole process change of a certain operation object is presented;
if the view angle transformation request content only has operation time, which indicates that a certain time point or time period is requested, all operation objects and video clips of all operation views in the whole scene are presented, and the real-time state of the whole scene is presented at a certain time point or time period;
if the view transformation request content only has an operation view, which indicates that the complete video of all operation objects in the whole scene is requested under the view, the whole process change of the whole scene under a certain view is presented;
if the view angle conversion request content contains an operation object and operation time, indicating that a certain time point or a certain time period is requested, and determining the state of the operation object under different view angles;
if the view transformation request content contains an operation object and an operation view, indicating that the whole process of a certain operation object is changed under a certain view;
if the view transformation request content contains operation time and operation view, indicating that a certain time point or a certain time period is requested, and the real-time state of the whole scene under a certain view;
if the view transformation request content comprises an operation object, operation time and an operation view, indicating that a certain time point or a certain time period is requested, and the real-time state of a certain operation object under a certain view;
s4: after receiving the view angle conversion request, the system acquires video clips meeting the view angle conversion request from a video library;
s5: the system covers the acquired video clips on the original video stream;
s6: and when the coverage state exits, restoring the scene presentation of the original video stream.
2. The method for enhancing a presentation style in an AR system according to claim 1, wherein in step S2, the specific method for constructing a video library includes:
directly storing the shot complete video to form a video library; or (b)
And splitting the shot complete video into a plurality of video frequency bands, and storing to form a video library.
3. The method for enhancing a presentation style in an AR system according to claim 1, wherein in step S3, the method for issuing a view angle conversion request includes:
and sending out a view angle conversion request in a scene interaction mode.
4. The method for enhancing a presentation style in an AR system according to claim 2, wherein in step S4, when the video library is composed of complete videos, the system intercepts video clips conforming to the view angle transformation request from the complete videos; when the video library is composed of video clips, the system selects video clips from the video clips that meet the view transformation request.
5. The method for enhancing a presentation in an AR system according to claim 1, wherein in step S5, after the overlay is completed, the original video stream and the overlaid video clip coexist, and the operator can only see the overlaid video clip.
6. The method for enhancing a presentation manner in an AR system according to claim 1, wherein in step S6, the manner in which the coverage state exits includes:
the video clip obtained in the step S4 is played completely; or (b)
The operator initiates a new view angle conversion request; or (b)
The operator initiates a request to end the coverage status.
CN202110984031.5A 2021-08-25 2021-08-25 Method for enhancing presentation mode in AR system Active CN113691779B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110984031.5A CN113691779B (en) 2021-08-25 2021-08-25 Method for enhancing presentation mode in AR system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110984031.5A CN113691779B (en) 2021-08-25 2021-08-25 Method for enhancing presentation mode in AR system

Publications (2)

Publication Number Publication Date
CN113691779A CN113691779A (en) 2021-11-23
CN113691779B true CN113691779B (en) 2024-04-05

Family

ID=78582971

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110984031.5A Active CN113691779B (en) 2021-08-25 2021-08-25 Method for enhancing presentation mode in AR system

Country Status (1)

Country Link
CN (1) CN113691779B (en)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106358036A (en) * 2016-08-31 2017-01-25 杭州当虹科技有限公司 Method for watching virtual reality video at preset view angle
CN107396085A (en) * 2017-08-24 2017-11-24 三星电子(中国)研发中心 A kind of processing method and system of full multi-view video image
CN107633241A (en) * 2017-10-23 2018-01-26 三星电子(中国)研发中心 A kind of method and apparatus of panoramic video automatic marking and tracking object
CN108882018A (en) * 2017-05-09 2018-11-23 合信息技术(北京)有限公司 Video playing, data offering method, client and server in virtual scene
CN110531865A (en) * 2019-09-20 2019-12-03 深圳市凯达尔科技实业有限公司 Actual situation scene operation platform and application method based on 5G and MR technology
CN111158469A (en) * 2019-12-12 2020-05-15 广东虚拟现实科技有限公司 Visual angle switching method and device, terminal equipment and storage medium

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106358036A (en) * 2016-08-31 2017-01-25 杭州当虹科技有限公司 Method for watching virtual reality video at preset view angle
CN108882018A (en) * 2017-05-09 2018-11-23 合信息技术(北京)有限公司 Video playing, data offering method, client and server in virtual scene
CN107396085A (en) * 2017-08-24 2017-11-24 三星电子(中国)研发中心 A kind of processing method and system of full multi-view video image
CN107633241A (en) * 2017-10-23 2018-01-26 三星电子(中国)研发中心 A kind of method and apparatus of panoramic video automatic marking and tracking object
CN110531865A (en) * 2019-09-20 2019-12-03 深圳市凯达尔科技实业有限公司 Actual situation scene operation platform and application method based on 5G and MR technology
CN111158469A (en) * 2019-12-12 2020-05-15 广东虚拟现实科技有限公司 Visual angle switching method and device, terminal equipment and storage medium

Also Published As

Publication number Publication date
CN113691779A (en) 2021-11-23

Similar Documents

Publication Publication Date Title
CN106803966B (en) Multi-user network live broadcast method and device and electronic equipment thereof
EP3238445B1 (en) Interactive binocular video display
CN112585978B (en) Generating a composite video stream for display in VR
EP3691280B1 (en) Video transmission method, server, vr playback terminal and computer-readable storage medium
CN105657438A (en) Method and apparatus for processing panoramic live video resource
CN112312111A (en) Virtual image display method and device, electronic equipment and storage medium
CN108900857A (en) A kind of multi-visual angle video stream treating method and apparatus
WO2017113577A1 (en) Method for playing game scene in real-time and relevant apparatus and system
CN113115110B (en) Video synthesis method and device, storage medium and electronic equipment
CN107734244B (en) Panorama movie playback method and playing device
US20040169676A1 (en) Efficient method for creating a visual telepresence for large numbers of simultaneous users
CN101742096A (en) Multi-viewing-angle interactive TV system and method
US11703942B2 (en) System and method for interactive 360 video playback based on user location
KR20180001550A (en) Personalized shopping mall system using virtual camera
CN108093300B (en) Animation capture management system
Bolshakov et al. Building a system architecture for displaying data in a complex of output devices
CN113691779B (en) Method for enhancing presentation mode in AR system
EP3621309A1 (en) Transmission system for multi-channel image, control method therefor, and multi-channel image playback method and apparatus
WO2018234622A1 (en) A method for detecting events-of-interest
CN112019921A (en) Body motion data processing method applied to virtual studio
Foote et al. One-man-band: A touch screen interface for producing live multi-camera sports broadcasts
CN112995692B (en) Interactive data processing method, device, equipment and medium
JP2017183816A (en) Moving image distribution server, moving image output device, moving image distribution system, and moving image distribution method
CN112019922A (en) Facial expression data processing method applied to virtual studio
WO2024084943A1 (en) Information processing device, information processing method, and program

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant