CN114339134A - Remote online conference system based on Internet and VR technology - Google Patents

Remote online conference system based on Internet and VR technology Download PDF

Info

Publication number
CN114339134A
CN114339134A CN202210250143.2A CN202210250143A CN114339134A CN 114339134 A CN114339134 A CN 114339134A CN 202210250143 A CN202210250143 A CN 202210250143A CN 114339134 A CN114339134 A CN 114339134A
Authority
CN
China
Prior art keywords
display window
file
file display
area
internet
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202210250143.2A
Other languages
Chinese (zh)
Other versions
CN114339134B (en
Inventor
程华山
彭莎莎
汪红刚
黄芸
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhongruiyun Software Shenzhen Co ltd
Original Assignee
Zhongruiyun Software Shenzhen Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhongruiyun Software Shenzhen Co ltd filed Critical Zhongruiyun Software Shenzhen Co ltd
Priority to CN202210250143.2A priority Critical patent/CN114339134B/en
Publication of CN114339134A publication Critical patent/CN114339134A/en
Application granted granted Critical
Publication of CN114339134B publication Critical patent/CN114339134B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Information Transfer Between Computers (AREA)

Abstract

The invention provides a remote online conference system based on Internet and VR technology, which relates to the technical field of VR online conferences.

Description

Remote online conference system based on Internet and VR technology
Technical Field
The invention relates to the technical field of VR online conferences, in particular to a remote online conference system based on the Internet and VR technology.
Background
VR generally refers to virtual reality. Virtual Reality (abbreviated as VR) is a new practical technology developed in the 20 th century. Virtual reality technology encompasses computer, electronic information, simulation technology, the basic implementation of which is that a computer simulates a virtual environment to give a person a sense of environmental immersion. In recent years, due to the influence of epidemic situations, more and more large companies begin to select a form of a teleconference to realize communication among employees, and compared with the traditional online conference, the teleconference using the VR technology can collect all employees working in different places into a virtual space, and communicates and interacts with other colleagues in real time, so that the conference experience effect is more impulsive, and the conference is more vivid.
However, the existing remote online conference system based on the VR technology does not perform targeted optimization for the usage scene of the online conference, and generally performs global rendering on the picture in the field, so that the processing performance requirement on the VR terminal is high, the frame rate and the picture quality are reduced, and the usage experience of the user is further affected.
Disclosure of Invention
Technical problem to be solved
Aiming at the defects of the prior art, the invention provides a remote online conference system based on the Internet and VR technology, which solves the problems that the prior art is not purposefully optimized and the processing performance requirement of a VR terminal is high.
(II) technical scheme
In order to achieve the purpose, the invention is realized by the following technical scheme:
a remote online conference system based on internet and VR technology, comprising:
a pattern recognition module for recognizing a current conference pattern, the conference pattern comprising: a file display mode and a chat mode;
the windowing display module is used for displaying the files in a windowing mode when the files are opened;
the file display window identification module is used for identifying the file display window when the file display window is in a file display mode to obtain the position information of the file display window;
the visual field locking control module is used for closing the gesture detection of the VR terminal after identifying the file display window so as to lock the visual field; the system is also used for starting the gesture detection of the VR terminal and canceling the view locking after the system is in the chat mode;
the window position adjusting module is used for jacking the file display window and placing the file display window in the center of the visual field after the visual field is locked;
the fixation point area adjusting module is used for calculating a fixation point area based on the position information of the file display window;
and the rendering control module is used for rendering the view picture based on the fixation point area.
Further, the identifying the current conference mode includes:
when a conference host opens a file, the file is identified as a file display mode, otherwise, the file is identified as a chat mode;
and the file includes a picture, text, or video.
Further, the gesture detection for closing the VR terminal includes: deactivating a controller key and a head-display gyroscope of the VR terminal;
the gesture detection of the VR-enabled terminal includes: launch controller button and the head at VR terminal and show the gyroscope.
Further, the file display window comprises a window identification area, a position identification area and a content display area; and the number of the position identification areas is four, and the position identification areas are respectively positioned at four corner points of the content display area.
Further, the identifying the file display window to obtain the position information of the file display window includes:
selecting a file display window from a plurality of windows based on the window identification area;
based on the four position recognition areas, four corner positions of the content display area are determined as position information of the file display window.
Further, the calculating a gazing point region based on the position information of the file display window includes:
and taking the length from the center of the visual field to any corner point as the radius of the fixation point area to obtain a circular fixation point area.
Further, rendering the view picture based on the gazing point area includes:
taking the visual field picture outside the fixation point area as other areas;
rendering the fixation point area according to a% of resolution, and rendering other areas according to b% of resolution;
and a is greater than b.
Further, a =100 and b = 20.
(III) advantageous effects
The invention applies the existing annotation point rendering technology to a VR online conference system, performs targeted optimization aiming at the specific use scene of the online conference, automatically adjusts the size of the gaze point area in real time based on the size of the file display window, reduces the performance requirement of a VR terminal, simultaneously can ensure the definition of a content display area, and avoids the problems that the picture is fuzzy and the content cannot be seen clearly due to lower rendering resolution outside the gaze point area when a user watches windowed files.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a system block diagram of an embodiment of the present invention;
FIG. 2 is a diagram illustrating a conventional point-of-regard rendering technique;
FIG. 3 is a diagram of a file display window according to an embodiment of the present invention;
fig. 4 is a schematic diagram of a gaze point region and other regions calculated by the embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention are clearly and completely described, and it is obvious that the described embodiments are a part of the embodiments of the present invention, but not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The embodiment of the application provides a remote online conference system based on the Internet and the VR technology, and solves the problems that the prior art is not pertinently optimized and the processing performance requirement of the VR terminal is high.
In order to better understand the technical solution, the technical solution will be described in detail with reference to the drawings and the specific embodiments.
The embodiment of the invention provides a remote online conference system based on the internet and VR technology, referring to fig. 1, the system comprises:
a pattern recognition module for recognizing a current conference pattern, the conference pattern comprising: a file display mode and a chat mode;
the windowing display module is used for displaying the files in a windowing mode when the files are opened;
the file display window identification module is used for identifying the file display window when the file display window is in a file display mode to obtain the position information of the file display window;
the visual field locking control module is used for closing the gesture detection of the VR terminal after identifying the file display window so as to lock the visual field; the system is also used for starting the gesture detection of the VR terminal and canceling the view locking after the system is in the chat mode;
the window position adjusting module is used for jacking the file display window and placing the file display window in the center of the visual field after the visual field is locked;
the fixation point area adjusting module is used for calculating a fixation point area based on the position information of the file display window;
and the rendering control module is used for rendering the view picture based on the fixation point area.
The following describes the implementation process of the embodiment of the present invention in detail:
point of regard rendering techniques: in the process of viewing objects, the eyes of people cannot notice all details, and only the vicinity of the intermediate vision focus is clear, so that the processing performance is not necessarily divided on the whole screen, only partial processing which is really noticed by people needs to be processed preferentially, the rendering burden of a VR terminal can be reduced, and the requirement of VR on the processing performance is reduced. Therefore, as shown in fig. 2, by identifying the gazing point (inner circle) of the visual field picture of the user in VR, the definition of the VR screen is dynamically adjusted, that is, the gazing point region is clearest, and the afterglow part reduces the definition. Therefore, the point of regard rendering scheme is beneficial to reducing hardware pressure and improving VR visual definition. According to data of head companies in the industry, the point-of-regard rendering can improve the display effect by 57% at most.
Considering the main 2 usage scenarios of online conferencing, i.e. multiple people talking with a virtual doll or discussing a certain document, it is preferred to identify the current conference mode.
And S1, identifying the current conference mode.
Specifically, the conference mode includes: a file presentation mode and a chat mode.
When an online conference is carried out, a plurality of participants usually exist, each participant uses a VR terminal to join the conference, the conference can comprise a speaker and a plurality of listeners, the speaker and the listeners can be freely switched according to user requirements, the speaker has the authority of opening files, the authority is correspondingly changed during switching, the format of the files is not limited, and the files can be pictures, texts, videos and the like. Therefore, when the speaker opens the file in the conference, the conference can be identified as a file display mode; otherwise, a chat mode is identified by default. And after the file is closed, the default chat mode is recovered.
If the file presentation mode is recognized at a certain point of the conference, indicating that the user has opened a document, S2 is performed.
S2, when the file is opened, the file is displayed in a window.
Specifically, after a file is displayed in a window, a file display window is suspended on a plane at a certain position in a virtual space, and when a user looks at the plane, the display plane is perpendicular to the sight of the user.
And S3, identifying the file display window to obtain the position information of the file display window.
Specifically, once the user views the plane where the file display window is located, that is, the file display window is located in the virtual field of view of the user, the position of the file display window can be identified by using the existing target identification algorithm.
For example: as shown in fig. 3, the file display window includes a window identification area, a position identification area, and a content display area; and the number of the position identification areas is four, and the position identification areas are respectively positioned at four corner points of the content display area.
And the window identification area is used for distinguishing the file display window from other display windows, such as a conference setting window and the like. The window identification area may contain text information, specifically, a file name of the file, and other windows may be distinguished from the file display window according to text contents of the file.
The position identification area can be a specific preset pattern or a character, and the position identification areas at four corners can be identified through a target identification algorithm. The size and position information of the whole content display area can be known through the position identification areas at the four corners as the position of the file display window.
S4, after the file display window is recognized, closing the gesture detection of the VR terminal to lock the visual field; and after the terminal is in the chat mode, starting gesture detection of the VR terminal, and canceling view locking.
Specifically, the visual field is locked, that is, the user rotates or triggers the key anyway, and the visual angle of the user in the virtual space is kept fixed.
In order to avoid influencing the effect of subsequent viewpoint rendering due to the change of the user posture, the view angle of the user in the virtual space needs to be locked after the file display window is identified. Because most of the existing VR terminals can be conditioned on the visual angle through a head-display gyroscope or a controller key, the gesture detection function of the equipment needs to be turned off when the terminal is in a file display mode. Thus: the gesture detection of closing the VR terminal includes: deactivating a controller key and a head-display gyroscope of the VR terminal; the gesture detection of the VR-enabled terminal includes: launch controller button and the head at VR terminal and show the gyroscope. This also does not affect the positioning of the VR terminal to the controller.
And S5, after the visual field is locked, the file display window is placed at the top and at the center of the visual field.
Specifically, if a file is opened, a plurality of windows exist, and in order to match the position of the subsequent gaze point region, the file display window needs to be placed on top and placed in the center of the view after the view is locked.
S6, the gazing point region is calculated based on the position information of the file display window.
The current gaze point rendering technology usually needs to cooperate with an eye tracking module or manually set a rendering ratio in advance to render a view picture. However, most of the existing VR devices are not equipped with an eye tracking module, and cannot adapt to the current mainstream configuration level, and the preset rendering ratio may cause the content display area to exceed the gazing point area, so that the resolution of the exceeded portion is low, and the content cannot be seen clearly. Therefore, in the present application, as shown in fig. 4, a circular gaze point region (+ hatched region) is obtained by using the length d from the center of the field of view (region where the outer circle is located) to any one corner point as the radius of the gaze point region. Therefore, the content display area can be ensured to be completely positioned in the fixation point area, and the picture of the content display area is further ensured to be clear.
And S7, rendering the view picture based on the fixation point area.
Specifically, as shown in fig. 4, the view field screen other than the gazing point region is set as another region (diagonally shaded region);
rendering the fixation point area according to a% of resolution, and rendering other areas according to b% of resolution; and a is greater than b. For example, a =100 and b = 20. In this way, the resolution of the other region is lower than that of the gazing point region, and the demand for processing performance can be reduced.
In summary, compared with the prior art, the method has the following beneficial effects:
the invention applies the existing annotation point rendering technology to a VR online conference system, performs targeted optimization aiming at the specific use scene of the online conference, automatically adjusts the size of the gaze point area in real time based on the size of the file display window, reduces the performance requirement of a VR terminal, simultaneously can ensure the definition of a content display area, and avoids the problems that the picture is fuzzy and the content cannot be seen clearly due to lower rendering resolution outside the gaze point area when a user watches windowed files.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims (8)

1. A remote online conference system based on internet and VR technology, the system comprising:
a pattern recognition module for recognizing a current conference pattern, the conference pattern comprising: a file display mode and a chat mode;
the windowing display module is used for displaying the files in a windowing mode when the files are opened;
the file display window identification module is used for identifying the file display window when the file display window is in a file display mode to obtain the position information of the file display window;
the visual field locking control module is used for closing the gesture detection of the VR terminal after identifying the file display window so as to lock the visual field; the system is also used for starting the gesture detection of the VR terminal and canceling the view locking after the system is in the chat mode;
the window position adjusting module is used for jacking the file display window and placing the file display window in the center of the visual field after the visual field is locked;
the fixation point area adjusting module is used for calculating a fixation point area based on the position information of the file display window;
and the rendering control module is used for rendering the view picture based on the fixation point area.
2. The internet and VR technology based remote online conferencing system of claim 1, wherein the identifying a current conferencing mode comprises:
when a conference host opens a file, the file is identified as a file display mode, otherwise, the file is identified as a chat mode;
and the file includes a picture, text, or video.
3. The internet and VR technology based remote online conference system of claim 1, wherein the gesture detection to turn off the VR terminal includes: deactivating a controller key and a head-display gyroscope of the VR terminal;
the gesture detection of the VR-enabled terminal includes: launch controller button and the head at VR terminal and show the gyroscope.
4. The internet and VR technology based remote online conference system of claim 1, wherein the file display window includes a window identification area, a location identification area, and a content display area; and the number of the position identification areas is four, and the position identification areas are respectively positioned at four corner points of the content display area.
5. The system of claim 4, wherein the identifying the file display window to obtain the location information of the file display window comprises:
selecting a file display window from a plurality of windows based on the window identification area;
based on the four position recognition areas, four corner positions of the content display area are determined as position information of the file display window.
6. The internet and VR technology based remote online conference system of claim 5, wherein the calculating of the gaze point area based on the location information of the file display window comprises:
and taking the length from the center of the visual field to any corner point as the radius of the fixation point area to obtain a circular fixation point area.
7. The internet and VR technology based remote online conference system of claim 1, wherein the rendering of the view based on the gaze point area comprises:
taking the visual field picture outside the fixation point area as other areas;
rendering the fixation point area according to a% of resolution, and rendering other areas according to b% of resolution;
and a is greater than b.
8. An internet and VR technology based remote online conferencing system as in claim 7 where a =100 and b = 20.
CN202210250143.2A 2022-03-15 2022-03-15 Remote online conference system based on Internet and VR technology Active CN114339134B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210250143.2A CN114339134B (en) 2022-03-15 2022-03-15 Remote online conference system based on Internet and VR technology

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210250143.2A CN114339134B (en) 2022-03-15 2022-03-15 Remote online conference system based on Internet and VR technology

Publications (2)

Publication Number Publication Date
CN114339134A true CN114339134A (en) 2022-04-12
CN114339134B CN114339134B (en) 2022-06-21

Family

ID=81033881

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210250143.2A Active CN114339134B (en) 2022-03-15 2022-03-15 Remote online conference system based on Internet and VR technology

Country Status (1)

Country Link
CN (1) CN114339134B (en)

Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105786360A (en) * 2014-12-23 2016-07-20 珠海金山办公软件有限公司 Method and device for demonstrating PPT file with mobile terminal
WO2019046597A1 (en) * 2017-08-31 2019-03-07 Apple Inc. Systems, methods, and graphical user interfaces for interacting with augmented and virtual reality environments
US20190098255A1 (en) * 2017-09-22 2019-03-28 Faro Technologies, Inc. Collaborative virtual reality online meeting platform
CN109698949A (en) * 2017-10-20 2019-04-30 腾讯科技(深圳)有限公司 Method for processing video frequency, device and system based on virtual reality scenario
CN109766011A (en) * 2019-01-16 2019-05-17 北京七鑫易维信息技术有限公司 A kind of image rendering method and device
US20190243445A1 (en) * 2018-02-06 2019-08-08 Gree, Inc. Application processing system, method of processing application and storage medium storing program for processing application
CN110401810A (en) * 2019-06-28 2019-11-01 广东虚拟现实科技有限公司 Processing method, device, system, electronic equipment and the storage medium of virtual screen
CN110427502A (en) * 2019-06-28 2019-11-08 广东虚拟现实科技有限公司 Display methods, device, electronic equipment and the storage medium of virtual content
CN111064919A (en) * 2019-12-09 2020-04-24 零空间(北京)科技有限公司 VR (virtual reality) teleconference method and device
CN111881807A (en) * 2020-07-23 2020-11-03 深圳市凯达尔科技实业有限公司 VR conference control system and method based on face modeling and expression tracking
US20210058612A1 (en) * 2019-08-21 2021-02-25 Beijing Boe Optoelectronics Technology Co., Ltd. Virtual reality display method, device, system and storage medium
US20210209856A1 (en) * 2018-06-01 2021-07-08 Immersal Oy Solution for generating virtual reality representation
CN113504830A (en) * 2021-07-22 2021-10-15 优奈柯恩(北京)科技有限公司 Display method and device for head-mounted display equipment

Patent Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105786360A (en) * 2014-12-23 2016-07-20 珠海金山办公软件有限公司 Method and device for demonstrating PPT file with mobile terminal
WO2019046597A1 (en) * 2017-08-31 2019-03-07 Apple Inc. Systems, methods, and graphical user interfaces for interacting with augmented and virtual reality environments
US20190098255A1 (en) * 2017-09-22 2019-03-28 Faro Technologies, Inc. Collaborative virtual reality online meeting platform
CN109698949A (en) * 2017-10-20 2019-04-30 腾讯科技(深圳)有限公司 Method for processing video frequency, device and system based on virtual reality scenario
US20190243445A1 (en) * 2018-02-06 2019-08-08 Gree, Inc. Application processing system, method of processing application and storage medium storing program for processing application
US20210209856A1 (en) * 2018-06-01 2021-07-08 Immersal Oy Solution for generating virtual reality representation
CN109766011A (en) * 2019-01-16 2019-05-17 北京七鑫易维信息技术有限公司 A kind of image rendering method and device
CN110401810A (en) * 2019-06-28 2019-11-01 广东虚拟现实科技有限公司 Processing method, device, system, electronic equipment and the storage medium of virtual screen
CN110427502A (en) * 2019-06-28 2019-11-08 广东虚拟现实科技有限公司 Display methods, device, electronic equipment and the storage medium of virtual content
US20210058612A1 (en) * 2019-08-21 2021-02-25 Beijing Boe Optoelectronics Technology Co., Ltd. Virtual reality display method, device, system and storage medium
CN111064919A (en) * 2019-12-09 2020-04-24 零空间(北京)科技有限公司 VR (virtual reality) teleconference method and device
CN111881807A (en) * 2020-07-23 2020-11-03 深圳市凯达尔科技实业有限公司 VR conference control system and method based on face modeling and expression tracking
CN113504830A (en) * 2021-07-22 2021-10-15 优奈柯恩(北京)科技有限公司 Display method and device for head-mounted display equipment

Also Published As

Publication number Publication date
CN114339134B (en) 2022-06-21

Similar Documents

Publication Publication Date Title
US10554921B1 (en) Gaze-correct video conferencing systems and methods
US20170301120A1 (en) Augmented display of information in a device view of a display screen
US10013805B2 (en) Control of enhanced communication between remote participants using augmented and virtual reality
JP2022528294A (en) Video background subtraction method using depth
US8514262B2 (en) Information processing apparatus and network conference system
US7916165B2 (en) Systems and method for enhancing teleconferencing collaboration
CN105474303B (en) Information processing unit, information processing method and program
CN111654715B (en) Live video processing method and device, electronic equipment and storage medium
US11662879B2 (en) Electronic nameplate display method and apparatus in video conference
US20240179272A1 (en) Virtual image video call method, terminal device, and storage medium
US20200304713A1 (en) Intelligent Video Presentation System
CN107682638A (en) Generation, the method and device of display panoramic picture
US20180270428A1 (en) Electronic information board system, image processing device, and image processing method
US20200195518A1 (en) Computer system, remote control notifacation method and program
CN109168071A (en) Image processing method and device
CN111242704A (en) Method and electronic equipment for superposing live character images in real scene
JP2009246408A (en) Interaction device, image processing module, image processing method, and program
CN109934150B (en) Conference participation degree identification method, device, server and storage medium
CN114339134B (en) Remote online conference system based on Internet and VR technology
CN105931204B (en) Picture restoring method and system
JP2000137789A (en) Image processor
CN109885172A (en) A kind of object interaction display method and system based on augmented reality AR
CN114615556B (en) Virtual live broadcast enhanced interaction method and device, electronic equipment and storage medium
US11972505B2 (en) Augmented image overlay on external panel
CN113766167A (en) Panoramic video conference enhancement method, system and network equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
TA01 Transfer of patent application right

Effective date of registration: 20220517

Address after: 518000 Guangdong, Shenzhen, Nanshan District Nanshan street, Guangdong Hainan street, business road, Yihai Plaza, room 1406, East.

Applicant after: Shenzhen Yi Pu Shi Shang You Technology Co.,Ltd.

Address before: 518000 03c31, 16 / F, building AB, Nanfang Securities Building, No. 2016 and 2018, Jianshe Road, Jiabei community, Nanhu street, Luohu District, Shenzhen, Guangdong

Applicant before: Zhongruiyun software (Shenzhen) Co.,Ltd.

TA01 Transfer of patent application right
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right

Effective date of registration: 20230712

Address after: 518000 03c31, 16 / F, building AB, Nanfang Securities Building, No. 2016 and 2018, Jianshe Road, Jiabei community, Nanhu street, Luohu District, Shenzhen, Guangdong

Patentee after: Zhongruiyun software (Shenzhen) Co.,Ltd.

Address before: 518000 Guangdong, Shenzhen, Nanshan District Nanshan street, Guangdong Hainan street, business road, Yihai Plaza, room 1406, East.

Patentee before: Shenzhen Yi Pu Shi Shang You Technology Co.,Ltd.

TR01 Transfer of patent right