WO2023051739A1 - Procédé et dispositif d'affichage d'image, support d'enregistrement et dispositif électronique - Google Patents

Procédé et dispositif d'affichage d'image, support d'enregistrement et dispositif électronique Download PDF

Info

Publication number
WO2023051739A1
WO2023051739A1 PCT/CN2022/122882 CN2022122882W WO2023051739A1 WO 2023051739 A1 WO2023051739 A1 WO 2023051739A1 CN 2022122882 W CN2022122882 W CN 2022122882W WO 2023051739 A1 WO2023051739 A1 WO 2023051739A1
Authority
WO
WIPO (PCT)
Prior art keywords
target
image
client
identification
content
Prior art date
Application number
PCT/CN2022/122882
Other languages
English (en)
Chinese (zh)
Inventor
王宙
Original Assignee
中兴通讯股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中兴通讯股份有限公司 filed Critical 中兴通讯股份有限公司
Publication of WO2023051739A1 publication Critical patent/WO2023051739A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H04N5/265—Mixing

Definitions

  • the present disclosure relates to the communication field, and in particular, to an image display method and device, a storage medium, and an electronic device.
  • AR/VR equipment With the rapid development of society, AR/VR equipment has been further developed, and more and more people pay attention to and use it.
  • the current AR applications usually use the application program of the AR equipment to extract the video collected by the camera. Drawing virtual objects after frame, recognition, and calculation will lead to excessive calculation pressure on AR devices, which makes the hardware requirements of AR devices higher, and because AR devices are used alone, the screen cannot be sent to the remote end for sharing and collaboration.
  • the traditional augmented reality AR device implements the augmented reality function independently, which makes the hardware requirements of the AR device too high, and no effective solution has been proposed yet.
  • Embodiments of the present disclosure provide an image display method and device, a storage medium, and an electronic device, so as to at least solve the problem that traditional augmented reality AR devices implement augmented reality functions independently, which makes the hardware requirements of the AR devices too high.
  • an image display method including: sending the collected target image to a target server; acquiring the identification of the first target object sent by the target server after processing the target image and a target location, wherein the first target object is located in the target image; sending the identification of the first target object to the content management platform, and obtaining the target content sent by the content management platform according to the identification; According to the target position of the first target object, the target content is superimposed on the target image to obtain an augmented reality AR image.
  • an image display method including: acquiring the target image sent by the target client; processing the target image to obtain the identification and target position of the first target object, wherein, The first target object is located in the target image; sending the identification of the first target object and the target position to the target client to instruct the target client to The logo and target location are identified in the augmented reality AR image.
  • an image display method including: acquiring the identifier of the first target object sent by the target client; determining the target content according to the identifier of the first target object; The content is sent to the target client to instruct the target client to determine an augmented reality AR image according to the target content.
  • an image display system including: a target server configured to acquire the target image collected by the target client, and process the target image to obtain the first The identification and target position of the target object are sent to the target client, wherein the first target object is located in the target image; the content management platform is configured to obtain the first target object sent by the target client identification, and send the target content determined according to the identification of the first target object to the target client; the target client is configured to send the target content according to the target location of the first target object superimposed on the target image to obtain an augmented reality AR image.
  • an image display device including: a first sending module configured to send the collected target image to a target server; a first obtaining module configured to obtain the The target server processes the target image and sends the identification and target position of the first target object, wherein the first target object is located in the target image; the second acquisition module is configured to take the first target object The identification of the target object is sent to the content management platform, and the target content sent by the content management platform according to the identification is obtained; the first display module is configured to display the target content according to the target position of the first target object superimposed on the target image to obtain an augmented reality AR image.
  • an image display device including: a third acquisition module configured to acquire the target image sent by the target client; a processing module configured to process the target image , to obtain the identification and target position of the first target object, wherein the first target object is located in the target image; the second display module is configured to send the identification of the first target object and the target position to the target client, so as to instruct the target client to determine an augmented reality AR image according to the identifier of the first target object and the target position.
  • an image display device which includes: a fourth acquisition module configured to acquire the identifier of the first target object sent by the target client; a determination module configured to The identification of the first target object determines the target content; the third display module is configured to send the target content to the target client to instruct the target client to determine the augmented reality AR image according to the target content .
  • a computer-readable storage medium where a computer program is stored in the computer-readable storage medium, wherein the computer program is configured to execute the above-mentioned image display method when running .
  • an electronic device including a memory, a processor, and a computer program stored on the memory and operable on the processor, wherein the processor executes the above image through the computer program Show method.
  • the target client on the AR device sends the target image collected by the AR device to the target server, and obtains the identification and target position of the first target object sent by the target server after processing the target image, so that the target client
  • the terminal can obtain the target content sent by the content management platform according to the identifier of the first target object, and superimpose the target content on the target image according to the target position of the first target object to obtain an augmented reality AR image.
  • the traditional augmented reality AR device realizes the augmented reality function independently, which makes the hardware requirements of the AR device too high and the like.
  • the AR device communicates with the target server and the content management platform, and then cooperates to realize the augmented reality function of the AR device, which reduces the hardware requirements of the AR device.
  • FIG. 1 is a block diagram of the hardware structure of a computer terminal of an image display method according to an embodiment of the present disclosure
  • FIG. 2 is a flowchart (1) of an image display method according to an embodiment of the present disclosure
  • FIG. 3 is a flowchart (2) of an image display method according to an embodiment of the present disclosure.
  • FIG. 4 is a flowchart (3) of an image display method according to an embodiment of the present disclosure.
  • FIG. 5 is a frame diagram of an image display method according to an embodiment of the present disclosure.
  • FIG. 6 is a timing diagram of an image display method according to an embodiment of the disclosure.
  • FIG. 7 is a flowchart of content overlay according to an embodiment of the disclosure.
  • Fig. 8 is a schematic diagram of content superposition according to an embodiment of the present disclosure.
  • FIG. 9 is an overall flowchart of an image display method according to an embodiment of the present disclosure.
  • FIG. 10 is a block diagram of an image display system according to an embodiment of the present disclosure.
  • FIG. 11 is a frame diagram (1) of an image display device according to an embodiment of the present disclosure.
  • FIG. 12 is a frame diagram (2) of an image display device according to an embodiment of the present disclosure.
  • FIG. 13 is a frame diagram (3) of an image display device according to an embodiment of the present disclosure.
  • FIG. 1 is a block diagram of a hardware structure of a computer terminal according to an image display method according to an embodiment of the present disclosure.
  • the computer terminal can include one or more (only one is shown in Figure 1) processor 102 (processor 102 can include but not limited to microprocessor (Microprocessor Unit, MPU for short) or programmable logic A device (Programmable logic device, PLD for short)) and a memory 104 for storing data.
  • processor 102 can include but not limited to microprocessor (Microprocessor Unit, MPU for short) or programmable logic A device (Programmable logic device, PLD for short)) and a memory 104 for storing data.
  • the above-mentioned computer terminal may also include a transmission device 106 and an input and output device 108 for communication functions.
  • FIG. 1 is only for illustration, and it does not limit the structure of the above computer terminal.
  • the computer terminal may also include more or less components than those shown in FIG. 1 , or have a different configuration with functions equivalent to those shown in FIG. 1 or more functions than those shown in FIG. 1 .
  • the memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as computer programs corresponding to the image display method in the embodiments of the present disclosure, and the processor 102 executes various functions by running the computer programs stored in the memory 104.
  • a functional application and data processing that is, to realize the above-mentioned method.
  • the memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory.
  • the memory 104 may further include a memory that is remotely located relative to the processor 102, and these remote memories may be connected to a computer terminal through a network. Examples of the aforementioned networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.
  • the transmission device 106 is used to receive or transmit data via a network.
  • the specific example of the above-mentioned network may include a wireless network provided by the communication provider of the computer terminal.
  • the transmission device 106 includes a network interface controller (NIC for short), which can be connected to other network devices through a base station so as to communicate with the Internet.
  • the transmission device 106 may be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet in a wireless manner.
  • RF Radio Frequency
  • FIG. 2 is a flow chart (1) of an image display method according to an embodiment of the present disclosure. The process includes the following steps:
  • Step S202 sending the collected target image to the target server
  • Step S204 acquiring the identification and target position of the first target object sent by the target server after processing the target image, wherein the first target object is located in the target image;
  • the first target object is an object in the target image or a two-dimensional code.
  • Step S206 sending the identifier of the first target object to the content management platform, and obtaining the target content sent by the content management platform according to the identifier;
  • Step S208 according to the target position of the first target object, superimpose the target content on the target image to obtain an augmented reality (Augmented Reality, AR) image.
  • augmented reality Augmented Reality, AR
  • the target client may display the AR image through the AR device.
  • AR devices include: AR glasses, AR helmets, etc.
  • the target client in the embodiment of the present application includes: a video conference client; the target server includes: a video conference server.
  • the target client on the AR device sends the target image collected by the AR device to the target server, and obtains the identification and target position of the first target object sent by the target server after processing the target image, so that the target client
  • the terminal can obtain the target content sent by the content management platform according to the identifier of the first target object, and superimpose the target content on the target image according to the target position of the first target object to obtain an augmented reality AR image.
  • the traditional augmented reality AR device realizes the augmented reality function independently, which makes the hardware requirements of the AR device too high and the like.
  • the AR device communicates with the target server and the content management platform, and then cooperates to realize the augmented reality function of the AR device, which reduces the hardware requirements of the AR device.
  • the aforementioned sending of the collected target image to the target server may be achieved in the following manner: the target area is captured by the image capture device on the AR device to obtain the target image; the target image is transmitted through the real-time transmission protocol (Real-time Transport protocol, referred to as RTP) code stream form and send to the target server.
  • RTP Real-time Transport protocol
  • the AR device will collect images of the identifiable target area through an image acquisition device such as a camera, and then the AR device will send the target image to the video conference server through the video conference client installed by itself.
  • the video conference The client needs to convert the target image into a real-time transport protocol RTP code stream, and send the RTP code stream to the video conference server.
  • step S208 is realized by the following methods: obtaining the preset first position of the target content in the target image; The first target object overlaps; if the target content does not overlap with the first target object, the target content is superimposed on the first position in the target image to obtain the AR image; in the case where the target content overlaps with the first target object, adjusting the preset position of the target content in the target image from the first position to a second position, wherein The target content at the second position does not overlap with the target position of the first target object at the target position; superimposing the target content at the second position in the target image, Obtain the AR image.
  • the video conferencing client obtains the identifier of the first target object sent by the video conferencing server and the target position of the first target object in the target image, it then obtains the corresponding identifier of the first target object from the content management platform. Since the user will initially set the target content at the initial position, the video conferencing client needs to obtain the preset first position of the target content in the target image, and then need to determine whether the target content at the first position is consistent with the target position The first target object overlaps; if there is no overlap, you can directly superimpose the target content on the first position of the target image to obtain an AR image.
  • the size of the target content may not change, only the vertex coordinates of the target content change, or the target content may The vertex coordinates of and the size of the target content changed at the same time.
  • the AR device can send the superimposed AR image to the video conference server through its own video conference client so that multiple video conferencing clients on other AR devices obtain AR images from the video conferencing server, and then other users can see the screens in the AR device in this embodiment through other AR devices, realizing the sharing of screens. It should be noted that multiple video conference clients participate in the target video conference through the video conference server.
  • the AR screen can be shared, the user using the AR device can mark the AR image, and then other users can also see the mark. Specifically, it can be realized through a data conference.
  • the data Conference client and data conference server are installed on the AR device.
  • the video conference client can communicate with the video conference server, and the data conference client can communicate with the data conference client.
  • the server communicates, and the video conferencing client and the data conferencing client can also communicate.
  • the video conference client on the AR device invokes the data conference client on the AR device to obtain the first transparent layer sent by the data conference client; and sends the second transparent layer to the data conference client to indicate
  • the data conferencing client sends the second transparent layer to the data conferencing server for storage, wherein the second transparent layer is a transparent layer obtained by marking the AR image on the first transparent layer , the second transparent layer is used to be overlaid and displayed on the AR image.
  • the user using the AR device wants to perform some marking operations on the screen, he can call the data conference client of the AR device, and the data conference client sends a first transparent layer.
  • the first transparent layer does not have marking information, and then The video conferencing client can superimpose the first transparent layer on the AR screen, and the user can make some marks on the first transparent layer.
  • the marked first transparent layer is defined as the second transparent layer, and then the video conference client sends the second transparent layer to the data conference client, so that the data conference client can send the second transparent layer Stored in the data conferencing server.
  • Other AR devices can obtain the second transparent layer from the data conference server through their own data conference client, send the second transparent layer to their own video conference client, and then superimpose the second transparent layer on the AR screen. Make other users can also see the mark, realize the sharing of mark.
  • FIG. 3 is According to the flow chart (2) of the image display method of the embodiment of the present disclosure, the process includes the following steps:
  • Step S302 acquiring the target image sent by the target client
  • Step S304 processing the target image to obtain the identification and target position of the first target object, wherein the first target object is located in the target image;
  • Step S306 sending the identification of the first target object and the target position to the target client, so as to instruct the target client to determine an augmented reality AR image according to the identification of the first target object and the target position.
  • the target client in the embodiment of the present application includes: a video conference client; the target server includes: a video conference server.
  • the target server acquires the target image sent by the target client, processes the target image, and sends the processed first target object identifier and target position to the target client to instruct the target client to The object's identification and target location determine the augmented reality AR image.
  • the traditional augmented reality AR device realizes the augmented reality function independently, which makes the hardware requirements of the AR device too high and the like.
  • the AR device communicates with the target server, and then cooperates to realize the augmented reality function of the AR device, which reduces the hardware requirements of the AR device.
  • the multi-point control unit MCU sends the target image to the identification and positioning module of the target server in the following manner; the identification and positioning module performs identification and positioning processing on the target image, The identifier of the first target object and the target position are obtained.
  • the target server is used as a video conference server for description below, and the video conference server is composed of a multi-point control unit MCU, an identification and positioning module, and a video conference management system.
  • the video conference server sends the target image to the identification and positioning module in the video conference server through the multi-point control unit (MulTI-point Control Unit, referred to as MCU), and then performs identification and positioning processing on the target image through the identification and positioning module. , to obtain the identifier of the first target object and the target position.
  • MCU multi-point control unit
  • the video conferencing server sends the target image to the identification and positioning module through the MCU, it needs to send an access request to the identification and positioning module through the MCU; Send the Internet Protocol IP address and port number to the MCU.
  • the target client when it sends the target image to the video conference server, it will send a Session Initiation Protocol (SIP) control command to the video conference management module of the video conference server, and then the video conference management module Instructing the MCU of the video conference server to send an access request to the identification and positioning module of the video conference server, and the identification and positioning module of the video conference server will send the Internet protocol IP address and port number to the MCU according to the request.
  • SIP Session Initiation Protocol
  • the video conferencing server sends the target image to the identification and positioning module through the MCU, which can be achieved in the following manner: using the IP address, the port number and the User Datagram Protocol (User Datagram Protocol, referred to as UDP) by the MCU to
  • UDP User Datagram Protocol
  • the target image is sent to the identification and positioning module in the form of a real-time transport protocol (RTP) code stream.
  • RTP real-time transport protocol
  • the MCU of the video conference server will send the target image to the identification and positioning module of the video conference server in the form of the real-time transport protocol RTP code stream according to the IP address, port number and user datagram protocol sent by the identification and positioning module.
  • the recognition and positioning module of the video conferencing server processes the target image to obtain the identification and target position of the first target object, which can be realized in the following manner: the recognition and positioning module acquires For the frame image, the identification and positioning module performs identification and positioning processing on the frame image to obtain the identification of the first target object and the target position.
  • the identification and positioning module performs identification and positioning processing on the frame image to obtain the identification of the first target object, which may be achieved in the following manner: the identification and positioning module indicates that the first target object is When the target object is a two-dimensional code, the recognition and positioning module analyzes the two-dimensional code to obtain the identification of the first target object; the recognition and positioning module indicates that the first target object is an object when the result of the frame image recognition is In this case, the identification and positioning module determines the identifier of the first target object from a preset object identifier set. It should be noted that the object identifier set includes one or more object identifiers.
  • the video conferencing server can send the identification of the first target object and the target position of the first target object to the target client in the following manner: the video conference server first sends the identification and target position of the first target object through the identification and positioning module To its own multipoint control unit MCU, and then the MCU of the video conference server sends the identifier of the first target object and the target location to the video conference client through the video conference management module.
  • the video conference server includes an MCU, an identification and location module, and a video conference management module.
  • the video conferencing server can not only determine the identity and target position of the first target object in the target image through the target image, but also realize screen sharing between different AR devices.
  • the target client includes the first video conferencing client
  • the AR image is sent to the second video conferencing client, wherein the multiple video conferencing clients participate in the target video conference through the video conferencing server, and the multiple video conferencing clients include the The first video conference client and the second video conference client.
  • FIG. 4 is a video conference server according to the present disclosure.
  • the flow chart (3) of the image display method of the embodiment, the process includes the following steps:
  • Step S402 acquiring the identifier of the first target object sent by the target client;
  • Step S404 determining the target content according to the identifier of the first target object
  • Step S406 sending the target content to the target client to instruct the target client to determine an augmented reality AR image according to the target content.
  • the target client in the embodiment of the present application includes: a video conference client.
  • the content management platform has correspondences between identifiers of different first objects and target content, and the correspondences are preset by the second target object.
  • the content management platform obtains the identifier of the first target object sent by the target client, determines the target content according to the identifier of the first target object, and sends the target content to the target client, so that the target client can determine the Augmented reality AR imagery.
  • the traditional augmented reality AR device realizes the augmented reality function independently, which makes the hardware requirements of the AR device too high and the like.
  • the AR device communicates with the content management platform, and then cooperates to realize the augmented reality function of the AR device, which reduces the hardware requirements of the AR device.
  • the content management platform determines the target content according to the identifier of the first target object, which may be implemented in the following manner: determine a content list according to the identifier of the first target object, and send the content list to The target client: acquiring operation information of the second target object on the content list sent by the video conferencing client; determining target content according to the operation information.
  • the content management platform determines the content list according to the identification of the first target object, and the content list has different contents, and then the content list is sent to the video conferencing client, and the user (second target object) selects the target content from the content list , and then the content management platform sends the target content to the video conference client.
  • This disclosure provides a system that uses data conferencing, video conferencing, and AR glasses to display different related content according to specified QR codes or objects in the front camera screen of AR glasses, and at the same time, other terminals can also mark.
  • the whole system includes content management platform, video conferencing client, data conferencing client, data conferencing server, data conferencing client, AR glasses and other systems and equipment.
  • the video conference server is composed of an identification and positioning module, an MCU, and a video conference management system.
  • the video conferencing client on the AR glasses sends the picture of the front camera to the MCU through the video conferencing system.
  • the two-dimensional code or the object identifier (ObjID) of the object is returned to the MCU, and returned to the video conference client on the glasses through the video conference management system.
  • the video conference client obtains the pre-associated content (document, video , link, etc.) superimposed on the glasses screen.
  • other terminals in the venue can also mark the content seen by the glasses in real time through the marking function of the data conference.
  • AR glasses require wireless networking (such as: 5G, Wifi).
  • the video conferencing client (including the video conferencing SDK) needs to be installed on the AR glasses, and the connection between the client and the server is normal.
  • FIG. 5 is a frame diagram of an image display method according to an embodiment of the present disclosure
  • FIG. 6 is a timing diagram of an image display method according to an embodiment of the present disclosure.
  • FIG. 5 and FIG. 6 are specifically described below. :
  • the video conference client is the soft client of the video conference
  • the MCU is the multi-point control unit in the video conference
  • the video conference management system is the conference management system in the video conference
  • the video conferencing client on the AR glasses will send the image of the camera to the MCU as a video stream.
  • the video conferencing client needs to initiate a video recognition AR service, it will notify the video conference management system through a Sip message, and then notify the MCU to send
  • the identifying and positioning module initiates a video access request.
  • the identification and positioning module decodes the RTP code stream, identifies and positions the acquired frame images at a certain frequency (such as 1s), and returns the identification result.
  • the recognition and positioning module recognizes the frame image in the code stream. For example, if a two-dimensional code appears in the frame picture, the text information parsed after the two-dimensional code is scanned is: zobjid-95da65f8214211eaae330221860e9b7e. At this time, the identification and positioning module will obtain and return the ObjectID (equivalent to the identification of the first target object in the above embodiment) and the area coordinates of the two-dimensional code (equivalent to the target position of the first target object in the above embodiment).
  • the MCU After receiving the identified ObjectID and coordinates, the MCU returns them to the video conference management system, and the video conference management system returns them to the AR glasses (video conference client), the initiator of the video stream, through the Sip protocol.
  • the video conferencing client queries the associated content list on the content management platform according to the ObjectID, and displays the content list. After the user selects the specific content, he downloads the specific content from the content management platform according to the URL. Because the sequence number is included in the content description file, it can support sequential display of multiple contents.
  • the video conference client performs AR content display according to the content downloaded from the content management platform and the area coordinate information transparently transmitted from the video conference management system.
  • the video conference soft client informs the MCU through the video conference management system to stop the forwarding of the code stream.
  • the video conference client receives the four vertex coordinates (X1Y1, X1Y2, X2Y1, X2Y2) of the object sent back by the video conference server in real time and the HTTP URL of the content to be superimposed, and calculates the content to be superimposed in real time.
  • the screen size screen resolution
  • the X axis is set to X
  • the Y axis is set to Y
  • Fig. 7 is a flow chart of content overlay according to an embodiment of the present disclosure
  • Fig. 8 is a schematic diagram of content overlay according to an embodiment of the present disclosure , the specific steps in Figure 7 are as follows:
  • Step S702 obtain screen size (screen resolution);
  • Step S704 Obtain the four vertex coordinates of the identified object, for example: X1Y1, X1Y2, X2Y1, X2Y2 respectively;
  • Step S706 judge whether it is X2>X or Y2>Y, in the case of yes, execute step S708, otherwise execute step S709;
  • Step S708 a prompt pops up to adjust the position of the glasses
  • Step S709 set the initial size of the superimposed content, such as 100*50;
  • Step S710 set the minimum threshold of superimposed content, such as 80*40;
  • Step S712 Set the initial position of the superimposed content, such as the origin of the lower left corner (0,0), (0,50), (100,50), (100,0);
  • Step S714 Detect whether the superimposed content collides with the identified object, and obtain the detection result
  • Step S716 determine whether the result collides (overlaps), in the case of overlapping, execute step S720, otherwise execute step S718;
  • Step S718 the superimposed content window is displayed, and the process ends;
  • Step S720 reduce the size of the overlay window (for example: XY minus 5 pixels);
  • Step S722 compared with the minimum display threshold, in the case of greater than, execute step S724, otherwise execute step S726;
  • Step S724 detect collision, execute step S716;
  • Step S726 adjust the position of the glasses, and reacquire the vertex coordinates
  • Step S728 Acquire the initial size of the overlay content
  • Step S730 Detect collision, execute step S716.
  • FIG. 9 is an overall flowchart of an image display method according to an embodiment of the present disclosure, and the specific steps are as follows:
  • the clients on the AR glasses include video conferencing clients and data conferencing clients.
  • S2 Share the screen of the front camera of the AR glasses through the MCU of the video conferencing server;
  • the identification and positioning module extracts frames, identifies and locates the video stream transferred from the MCU;
  • the identifying and positioning module is located in the video conference server.
  • the identification and positioning module returns the identified object ObjID and coordinates to the client through the MCU;
  • the client acquires associated content (documents, pictures, videos, etc.) from the content management platform according to the ObjID;
  • S6 The client performs superimposed display according to the content and coordinates
  • the data conference client is invoked by the video conference client, so as to realize the marking of the content.
  • the above-mentioned technical solution of the embodiment of the present application adopts the technology of video conferencing and data conferencing, which not only reduces the hardware requirements of AR devices, but also solves the shortcomings of the original technology that cannot be shared, marked, and collaborated.
  • FIG. 10 is a frame diagram of an image display system according to an embodiment of the present disclosure, including:
  • the target server 12 is configured to acquire the target image collected by the target client 16, and after processing the target image, obtain the identification of the first target object and the target position and send it to the target client 16, wherein, the first target object is located in the target image;
  • the content management platform 14 is configured to acquire the identifier of the first target object sent by the target client 16, and send the target content determined according to the identifier of the first target object to the target client 16;
  • the target client 16 is configured to superimpose the target content on the target image according to the target position of the first target object to obtain an augmented reality AR image.
  • the target client 16 on the AR device sends the target image collected by the AR device to the target server 12, and obtains the identification and target position of the first target object sent by the target server 12 after processing the target image,
  • the target client 16 can obtain the target content sent by the content management platform according to the identifier of the first target object, and superimpose the target content on the target image according to the target position of the first target object to obtain an augmented reality AR image.
  • the traditional augmented reality AR device realizes the augmented reality function independently, which makes the hardware requirements of the AR device too high and the like.
  • the AR device communicates with the target server and the content management platform, and then cooperates to realize the augmented reality function of the AR device, which reduces the hardware requirements of the AR device.
  • the method according to the above embodiments can be implemented by means of software plus a necessary general-purpose hardware platform, and of course also by hardware, but in many cases the former is better implementation.
  • the technical solution of the present disclosure can be embodied in the form of a software product in essence or the part that contributes to the prior art, and the computer software product is stored in a storage medium (such as ROM/RAM, disk, CD) contains several instructions to make a terminal device (which may be a mobile phone, a computer, a server, or a network device, etc.) execute the methods of various embodiments of the present disclosure.
  • an image display device is also provided, which is used to implement the above embodiments and preferred implementation modes, and what has already been described will not be repeated.
  • the term "module” may be a combination of software and/or hardware that realizes a predetermined function.
  • the devices described in the following embodiments are preferably implemented in software, implementations in hardware, or a combination of software and hardware are also possible and contemplated.
  • FIG. 11 is a frame diagram (1) of an image display device according to an embodiment of the present disclosure, which includes:
  • the first sending module 22 is configured to send the collected target image to the target server
  • the first obtaining module 24 is configured to obtain the identification and target position of the first target object sent by the target server after processing the target image, wherein the first target object is located in the target image;
  • the second acquiring module 26 is configured to send the identifier of the first target object to the content management platform, and acquire the target content sent by the content management platform according to the identifier;
  • the first display module 28 is configured to superimpose the target content on the target image according to the target position of the first target object to obtain an augmented reality AR image.
  • the target client on the AR device sends the target image collected by the AR device to the target server, and obtains the identification and target position of the first target object sent by the target server after processing the target image, so that the target client
  • the terminal can obtain the target content sent by the content management platform according to the identifier of the first target object, and superimpose the target content on the target image according to the target position of the first target object to obtain an augmented reality AR image.
  • the traditional augmented reality AR device realizes the augmented reality function independently, which makes the hardware requirements of the AR device too high and the like.
  • the AR device communicates with the target server and the content management platform, and then cooperates to realize the augmented reality function of the AR device, which reduces the hardware requirements of the AR device.
  • the first sending module 22 is also configured to collect an image of the target area through the image collection device on the AR device to obtain the target image; and transmit the target image in the form of a real-time transport protocol (RTP) stream sent to the target server.
  • RTP real-time transport protocol
  • the first display module 28 is also configured to acquire a preset first position of the target content in the target image; The position of the first target object overlaps; in the case that the target content does not overlap with the first target object, the target content is superimposed on the first position in the target image, and the obtained the AR image; in the case where the target content overlaps with the first target object, adjusting the preset position of the target content in the target image from the first position to a second position, wherein, The target content located at the second position does not overlap with the target position of the first target object located at the target position; superimposing the target content at the second position in the target image above, get the AR image.
  • the first display module 28 is further configured to send the AR image to a video conferencing server, so that multiple video conferencing clients acquire the AR image from the video conferencing server, wherein the target The server includes the video conference server, and the multiple video conference clients participate in the target video conference through the video conference server.
  • the first display module 28 is also configured to call the data conference client corresponding to the AR device to obtain the first transparent layer sent by the data conference client; send the second transparent layer to the data
  • the conference client is configured to instruct the data conference client to send the second transparent layer to the data conference server for storage, where the second transparent layer is to perform the AR image on the first transparent layer
  • the transparent layer obtained after marking, the second transparent layer is used to be superimposed on the AR image.
  • the first display module 28 is further configured to call the data conference client corresponding to the AR device to obtain the second transparent layer sent by the data conference client, wherein the second transparent layer has a mark information, the second transparent layer is the transparent layer obtained by the data conferencing client from the data conferencing server; superimposing the second transparent layer on the AR image to obtain the marked AR image, and passing The AR device displays the marked AR image.
  • Fig. 12 is a frame diagram (2) of an image display device according to an embodiment of the disclosure, the device includes:
  • the third acquiring module 32 is configured to acquire the target image sent by the target client
  • the processing module 34 is configured to process the target image to obtain the identification and target position of the first target object, wherein the first target object is located in the target image;
  • the second display module 36 is configured to send the identification of the first target object and the target position to the target client, so as to instruct the target client to determine the augmented reality according to the identification of the first target object AR image.
  • the target server obtains the target image sent by the target client, and processes the target image, and sends the processed first target object identification and target position to the target client to instruct the target client to follow the first target
  • the object's identification and target location determine the augmented reality AR image.
  • the traditional augmented reality AR device realizes the augmented reality function independently, which makes the hardware requirements of the AR device too high and the like.
  • the AR device communicates with the target server, and then cooperates to realize the augmented reality function of the AR device, which reduces the hardware requirements of the AR device.
  • the processing module 34 is also configured to send the target image to the identification and positioning module of the target server through the multi-point control unit MCU; through the identification and positioning module, the identification and positioning process is performed on the target image to obtain the An identification of the first target object and the target location.
  • the processing module 34 is also configured to send an access request to the identification and positioning module through the MCU; when the identification and positioning module responds to the access request, send an Internet Protocol IP address through the identification and positioning module. address and port number to the MCU.
  • the processing module 34 is also configured to use the IP address, the port number, and the User Datagram Protocol UDP to send the target image to the recognition system in the form of a real-time transport protocol (RTP) stream through the MCU. positioning module.
  • RTP real-time transport protocol
  • the processing module 34 is further configured to obtain a frame image from the target image according to a preset frequency through the identification and positioning module; identify the frame image through the identification and positioning module to obtain the first An identification of the target object and the target location.
  • the processing module 34 is further configured to parse the two-dimensional code through the recognition and positioning module to obtain the second An identification of a target object; when the result of the frame image recognition indicates that the first target object is an object, the identification and positioning module is used to determine the identity of the first target object from a preset object identification set IDs, wherein the set of object IDs includes IDs of one or more objects.
  • the second display module 36 is also configured to send the identification of the first target object and the target position to the multipoint control unit MCU of the target server through the identification and positioning module;
  • the identifier of the first target object and the target location are sent to a video conference client, wherein the target client includes the video conference client.
  • the second display module 36 is further configured to acquire the AR image sent by the first video conferencing client, wherein the target client includes the first video conferencing client;
  • the AR image is sent to the second video conferencing client, wherein the multiple video conferencing clients participate in the target video conference through the video conferencing server, and the multiple video conferencing The clients include the first video conference client and the second video conference client.
  • FIG. 13 is a frame diagram (3) of an image display device according to an embodiment of the present disclosure.
  • the device includes:
  • the fourth obtaining module 42 is configured to obtain the identifier of the first target object sent by the target client;
  • a determination module 44 configured to determine target content according to the identifier of the first target object
  • the third display module 46 is configured to send the target content to the target client to instruct the target client to determine an augmented reality AR image according to the target content.
  • the content management platform obtains the identifier of the first target object sent by the target client, determines the target content according to the identifier of the first target object, and sends the target content to the target client, so that the target client can determine the Augmented reality AR imagery.
  • the traditional augmented reality AR device realizes the augmented reality function independently, which makes the hardware requirements of the AR device too high and the like.
  • the AR device communicates with the content management platform, and then cooperates to realize the augmented reality function of the AR device, which reduces the hardware requirements of the AR device.
  • the determining module 44 is further configured to determine a content list according to the identifier of the first target object, and send the content list to the target client; obtain the second target object sent by the video conferencing client The operation information of the object on the content list; determine the target content according to the operation information.
  • Embodiments of the present disclosure also provide a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps in any one of the above method embodiments when running.
  • the above-mentioned storage medium may be configured to store a computer program for performing the following steps:
  • the above-mentioned computer-readable storage medium may include but not limited to: U disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM) , mobile hard disk, magnetic disk or optical disk and other media that can store computer programs.
  • ROM read-only memory
  • RAM random access memory
  • mobile hard disk magnetic disk or optical disk and other media that can store computer programs.
  • Embodiments of the present disclosure also provide an electronic device, including a memory and a processor, where a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.
  • the above-mentioned processor may be configured to execute the following steps through a computer program:
  • the electronic device may further include a transmission device and an input and output device, wherein the transmission device is connected to the processor, and the input and output device is connected to the processor.
  • each module or each step of the above-mentioned disclosure can be realized by a general-purpose computing device, and they can be concentrated on a single computing device, or distributed in a network composed of multiple computing devices In fact, they can be implemented in program code executable by a computing device, and thus, they can be stored in a storage device to be executed by a computing device, and in some cases, can be executed in an order different from that shown here. Or described steps, or they are fabricated into individual integrated circuit modules, or multiple modules or steps among them are fabricated into a single integrated circuit module for implementation. As such, the present disclosure is not limited to any specific combination of hardware and software.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Graphics (AREA)
  • Computer Hardware Design (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Information Transfer Between Computers (AREA)
  • Processing Or Creating Images (AREA)

Abstract

La présente divulgation concerne un procédé et un dispositif d'affichage d'image, un support d'enregistrement et un dispositif électronique. Le procédé consiste à : envoyer une image cible recueillie à un serveur cible ; obtenir un identifiant et une position cible d'un premier objet cible envoyé par le serveur cible après traitement de l'image cible, le premier objet cible étant situé dans l'image cible ; envoyer l'identifiant du premier objet cible à une plateforme de gestion de contenu, et obtenir un contenu cible envoyé par la plateforme de gestion de contenu en fonction de l'identifiant ; et superposer le contenu cible sur l'image cible en fonction de la position cible du premier objet cible pour obtenir une image en réalité augmentée (RA). La solution technique décrite résout les problèmes tels que lorsqu'un dispositif de RA classique effectue de manière indépendante une fonction de réalité augmentée, ce qui conduit à des exigences matérielles trop élevées pour le dispositif de RA.
PCT/CN2022/122882 2021-09-30 2022-09-29 Procédé et dispositif d'affichage d'image, support d'enregistrement et dispositif électronique WO2023051739A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111166756.X 2021-09-30
CN202111166756.XA CN115914716A (zh) 2021-09-30 2021-09-30 图像显示方法及装置、存储介质及电子装置

Publications (1)

Publication Number Publication Date
WO2023051739A1 true WO2023051739A1 (fr) 2023-04-06

Family

ID=85729589

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/122882 WO2023051739A1 (fr) 2021-09-30 2022-09-29 Procédé et dispositif d'affichage d'image, support d'enregistrement et dispositif électronique

Country Status (2)

Country Link
CN (1) CN115914716A (fr)
WO (1) WO2023051739A1 (fr)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102843347A (zh) * 2011-06-24 2012-12-26 中兴通讯股份有限公司 实现移动增强现实业务的系统及方法、终端及服务器
US9589372B1 (en) * 2016-01-21 2017-03-07 International Business Machines Corporation Augmented reality overlays based on an optically zoomed input
CN108550190A (zh) * 2018-04-19 2018-09-18 腾讯科技（深圳）有限公司 增强现实数据处理方法、装置、计算机设备和存储介质
CN110266992A (zh) * 2019-06-24 2019-09-20 苏芯物联技术(南京)有限公司 一种基于增强现实的远程视频交互系统以及方法
CN112394809A (zh) * 2019-08-16 2021-02-23 中国移动通信有限公司研究院 一种实现增强现实的方法、装置及通信设备
WO2021173147A1 (fr) * 2020-02-28 2021-09-02 Google Llc Système et procédé pour la lecture d'un contenu de réalité augmentée déclenchée par une reconnaissance d'image

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102843347A (zh) * 2011-06-24 2012-12-26 中兴通讯股份有限公司 实现移动增强现实业务的系统及方法、终端及服务器
US9589372B1 (en) * 2016-01-21 2017-03-07 International Business Machines Corporation Augmented reality overlays based on an optically zoomed input
CN108550190A (zh) * 2018-04-19 2018-09-18 腾讯科技（深圳）有限公司 增强现实数据处理方法、装置、计算机设备和存储介质
CN110266992A (zh) * 2019-06-24 2019-09-20 苏芯物联技术(南京)有限公司 一种基于增强现实的远程视频交互系统以及方法
CN112394809A (zh) * 2019-08-16 2021-02-23 中国移动通信有限公司研究院 一种实现增强现实的方法、装置及通信设备
WO2021173147A1 (fr) * 2020-02-28 2021-09-02 Google Llc Système et procédé pour la lecture d'un contenu de réalité augmentée déclenchée par une reconnaissance d'image

Also Published As

Publication number Publication date
CN115914716A (zh) 2023-04-04

Similar Documents

Publication Publication Date Title
WO2015101126A1 (fr) Procédé et dispositif d'adaptation à un changement d'orientation d'écran pendant un appel vidéo
US10194180B2 (en) Systems and methods for transmitting video data over a network
CN108154058B (zh) 图形码展示、位置区域确定方法及装置
US20140043485A1 (en) Wireless video camera and connection methods including multiple video streams
US20130111351A1 (en) Method for remotely controlling mobile terminal and mobile terminal
CN112261353B (zh) 一种视频监控分流方法、系统以及计算机可读存储介质
US20140118475A1 (en) Information processing device, conference system and storage medium
US9007531B2 (en) Methods and apparatus for expanding a field of view in a video communication session
WO2012019517A1 (fr) Procédé, dispositif et système de traitement vidéo pour une communication vidéo
CN114610253A (zh) 一种投屏方法及设备
EP4304166A1 (fr) Procédé, appareil et système de communication à réalité augmentée
WO2018200337A1 (fr) Système et procédé de simulation de transport de lumière entre des objets virtuels et réels en réalité mixte
US9052866B2 (en) Method, apparatus and computer-readable medium for image registration and display
WO2020001610A1 (fr) Procédé et dispositif d'insertion de vidéo
WO2014183533A1 (fr) Procédé de traitement d'image, terminal utilisateur, terminal de traitement d'image et système
CN109963106B (zh) 一种视频图像处理方法、装置、存储介质及终端
US20240015264A1 (en) System for broadcasting volumetric videoconferences in 3d animated virtual environment with audio information, and procedure for operating said device
EP3261341A2 (fr) Système de génération d'images vidéo et son procédé de génération d'images vidéo
WO2023051739A1 (fr) Procédé et dispositif d'affichage d'image, support d'enregistrement et dispositif électronique
TWI583199B (zh) 視頻會議控制系統及方法
WO2013067898A1 (fr) Procédé et terminal de transmission d'informations
KR20090014465A (ko) 영상 통화중 객체 숨김 서비스 제공 방법 및 시스템
WO2020063171A1 (fr) Procédé de transmission de données, terminal, serveur et support de stockage
US20210099547A1 (en) Collaborative and edge-enhanced augmented reality systems
EP4304165A1 (fr) Procédé, appareil et système de communication à réalité augmentée

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22875120

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22875120

Country of ref document: EP

Kind code of ref document: A1