CN108243293B - Image display method and system based on virtual reality equipment - Google Patents
Image display method and system based on virtual reality equipment Download PDFInfo
- Publication number
- CN108243293B CN108243293B CN201611209453.0A CN201611209453A CN108243293B CN 108243293 B CN108243293 B CN 108243293B CN 201611209453 A CN201611209453 A CN 201611209453A CN 108243293 B CN108243293 B CN 108243293B
- Authority
- CN
- China
- Prior art keywords
- display
- module
- display unit
- image data
- virtual reality
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/04—Synchronising
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
- G06F3/1423—Digital output to display device ; Cooperation and interconnection of the display device with other functional units controlling a plurality of local displays, e.g. CRT and flat panel display
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/2624—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects for obtaining an image which is composed of whole input images, e.g. splitscreen
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Controls And Circuits For Display Device (AREA)
- Control Of Indicators Other Than Cathode Ray Tubes (AREA)
Abstract
The invention discloses an image display method and system based on virtual reality equipment, relating to the technical field of virtual reality equipment, wherein the method comprises the following steps: the virtual reality equipment comprises at least two display units and at least two processing units which are in one-to-one correspondence with the at least two display units; each display unit reads the image data of the same target image through the corresponding processing unit; the image data are converted into signals received by the virtual reality equipment and transmitted to the at least two display units for display, the virtual reality equipment can be divided into a plurality of display units, each display unit reads the image data of the same target image through the corresponding processing unit, and the function that different display units display the same content on the virtual reality equipment is achieved.
Description
Technical Field
The invention relates to the technical field of virtual reality equipment, in particular to an image display method and system based on virtual reality equipment.
Background
Head-mounted displays have long been used in virtual display and augmented display systems, in which a virtual reality device separates the left and right eyes of a person, and the left and right eyes see the same or different angle contents of the same object in a closed environment, so as to form a complete picture in the brain of the person, whereas in current conventional mobile devices, rendering of a UI is performed by combining a plurality of 2D plane images into an image buffer and then sending the combined image to a 2D display for display, if a letter "a" is drawn on the UI, then the "a" can be seen on a general display at the middle of the screen, if the UI is viewed through the virtual reality device, since the left and right eyes on the virtual reality device are separated from each other, the left and right eyes see half contents of the "a", which results in the head-mounted display device, traditional UI rendering and display modes do not meet the needs of the client.
Based on the above scenario, some developers may also meet the requirements of clients by using the GPU/CPU to perform content rearrangement of the 2D UI on the buffer in the image synthesis stage, but implementing the content rearrangement of the 2D content in the synthesis stage may increase bandwidth and reduce performance, and may increase power consumption due to the need to use a coprocessor such as the GPU or the CPU to complete the rearrangement.
Disclosure of Invention
The invention mainly aims to provide an image display method and system based on virtual reality equipment, which can realize the function of displaying the same content on different display units on the virtual reality equipment by dividing the virtual reality equipment into a plurality of display units and reading the image data of the same target image by each display unit through a corresponding processing unit.
In order to achieve the above object, the present invention provides an image display method based on a virtual reality device, including:
the virtual reality equipment comprises at least two display units and at least two processing units which are in one-to-one correspondence with the at least two display units;
each display unit reads the image data of the same target image through the corresponding processing unit;
and converting the image data into signals received by the virtual reality equipment, and transmitting the signals into the at least two display units for display.
Optionally, the processing unit comprises: an input buffer module, an image data conversion module, an output module, a control module, a rearrangement fetch module and a rearrangement display time sequence module,
the input buffer module is used for receiving the image data according to the control signal of the control module and transmitting the image data to the image data conversion module;
the image data conversion module is used for reading data from the input buffer module, converting the image data into a signal received by virtual reality equipment according to a control signal of the control module, and transmitting the signal received by the virtual reality equipment to the output module;
the output module is used for transmitting the signals received by the virtual reality equipment to the corresponding display unit for displaying;
the control module is used for controlling the input buffer module, the image data conversion module, the output module re-arrangement module and the re-arrangement display time sequence module, and sending corresponding control signals to the input buffer module, the image data conversion module, the output module re-arrangement module and the re-arrangement display time sequence module;
the rearrangement access module is used for acquiring an access address of the image data according to a control signal of the control module;
and the rearrangement display time sequence module is used for reading the signals received by the virtual reality equipment from the output module according to the display time sequence information sent by the control module.
Optionally, the number of the display units is two, and the display unit includes a first display unit and a second display unit, and correspondingly, the number of the processing units is two, and the display unit includes a first processing unit and a second processing unit, the first display unit is associated with the first processing unit in a mapping manner, and the second display unit is associated with the second processing unit in a mapping manner.
Optionally, the display unit comprises a single-screen dual-display unit or a dual-screen dual-display unit, wherein,
when the display unit is a single-screen dual-display unit, reading image data of the same target image by each display unit through the corresponding processing unit comprises:
the first display unit informs the first processing unit to refresh the target image in a transverse scanning mode;
the second display unit informs the second processing unit to refresh the target image in a transverse scanning mode;
the first processing unit and the second processing unit alternately read the same image data in the memory line by line and convert the same image data into electrical timing signals;
when the display unit is a dual-screen dual-display unit, reading image data of the same target image by each display unit through the corresponding processing unit comprises:
the first display unit informs the first processing unit to refresh the target image in a transverse scanning mode;
the second display unit informs the second processing unit to refresh the target image in a transverse scanning mode;
the first processing unit and the second processing unit read the same image data in the memory line by line simultaneously and convert the same image data into electrical timing signals.
Optionally, before converting the image data into a signal accepted by a virtual reality device, the method further includes:
adjusting the size and the position of the display unit;
scaling a size of the image data.
As another aspect of the present invention, there is provided a virtual reality device-based image display system including: a virtual reality device and an image processing apparatus, the virtual reality device and the image processing apparatus being connected by a display timing synchronization interface, wherein,
the virtual reality equipment comprises at least two display units, a virtual reality display unit and a virtual reality display unit, wherein the at least two display units are used for receiving and displaying image data sent by the image processing device;
the image processing device comprises at least two processing units, a virtual reality device and a display unit, wherein the processing units are used for reading image data of a target image from a memory, converting the image data into signals received by the virtual reality device and transmitting the signals to the at least two display units for display;
the at least two processing units correspond to the at least two display units one by one; each display unit reads image data of the same target image through a corresponding processing unit.
Optionally, the processing unit comprises: an input buffer module, an image data conversion module, an output module, a control module, a rearrangement fetch module and a rearrangement display time sequence module,
the input buffer module is used for receiving the image data according to the control signal of the control module and transmitting the image data to the image data conversion module;
the image data conversion module is used for reading data from the input buffer module, converting the image data into a signal received by virtual reality equipment according to a control signal of the control module, and transmitting the signal received by the virtual reality equipment to the output module;
the output module is used for transmitting the signals received by the virtual reality equipment to the corresponding display unit for displaying;
the control module is used for controlling the input buffer module, the image data conversion module, the output module re-arrangement module and the re-arrangement display time sequence module, and sending corresponding control signals to the input buffer module, the image data conversion module, the output module re-arrangement module and the re-arrangement display time sequence module;
the rearrangement access module is used for acquiring an access address of the image data according to a control signal of the control module;
and the rearrangement display time sequence module is used for reading the signals received by the virtual reality equipment from the output module according to the display time sequence information sent by the control module.
Optionally, the number of the display units is two, and the display unit includes a first display unit and a second display unit, and correspondingly, the number of the processing units is two, and the display unit includes a first processing unit and a second processing unit, the first display unit is associated with the first processing unit in a mapping manner, and the second display unit is associated with the second processing unit in a mapping manner.
Optionally, the display unit comprises a single-screen dual-display unit or a dual-screen dual-display unit, wherein,
when the display unit is a single-screen dual-display unit, reading image data of the same target image by each display unit through the corresponding processing unit comprises:
the first display unit informs the first processing unit to refresh the target image in a transverse scanning mode;
the second display unit informs the second processing unit to refresh the target image in a transverse scanning mode;
the first processing unit and the second processing unit alternately read the same image data in the memory line by line and convert the same image data into electrical timing signals;
when the display unit is a dual-screen dual-display unit, reading image data of the same target image by each display unit through the corresponding processing unit comprises:
the first display unit informs the first processing unit to refresh the target image in a transverse scanning mode;
the second display unit informs the second processing unit to refresh the target image in a transverse scanning mode;
the first processing unit and the second processing unit read the same image data in the memory line by line simultaneously and convert the same image data into electrical timing signals.
Optionally, the image processing apparatus further includes:
the adjusting unit is used for adjusting the size and the position of the display unit;
a scaling unit for scaling the size of the image data.
The invention provides an image display method and system based on virtual reality equipment, wherein the method comprises the following steps: the virtual reality equipment comprises at least two display units and at least two processing units which are in one-to-one correspondence with the at least two display units; each display unit reads the image data of the same target image through the corresponding processing unit; the image data are converted into signals received by the virtual reality equipment and transmitted to the at least two display units for display, the virtual reality equipment can be divided into a plurality of display units, each display unit reads the image data of the same target image through the corresponding processing unit, and the function that different display units display the same content on the virtual reality equipment is achieved.
Drawings
Fig. 1 is a flowchart of an image display method based on virtual reality equipment according to an embodiment of the present invention;
FIG. 2 is a block diagram illustrating an exemplary structure of a processing unit according to an embodiment of the present invention;
FIG. 3 is a flowchart of an image display method when the display unit is single-screen dual-display according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a virtual reality device according to an embodiment of the present invention;
FIG. 5 is a flowchart of an image display method when the display unit is dual-screen dual-display according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of another virtual reality device according to an embodiment of the present invention;
fig. 7 is a block diagram illustrating an exemplary structure of an image display system based on a virtual reality device according to a second embodiment of the present invention;
fig. 8 is a block diagram illustrating an exemplary structure of another image display system based on a virtual reality device according to a second embodiment of the present invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
In the following description, suffixes such as "module", "component", or "unit" used to denote elements are used only for facilitating the explanation of the present invention, and have no specific meaning in themselves. Thus, "module" and "component" may be used in a mixture.
Example one
As shown in fig. 1, in this embodiment, an image display method based on a virtual reality device includes:
s10, the virtual reality equipment comprises at least two display units, and at least two processing units corresponding to the at least two display units one to one are established;
s20, each display unit reads the image data of the same target image through the corresponding processing unit;
and S30, converting the image data into signals received by the virtual reality equipment, and transmitting the signals to the at least two display units for display.
In this embodiment, the virtual reality device can be divided into a plurality of display units, and each display unit reads image data of the same target image through a corresponding processing unit, so as to realize a function of displaying the same content on different display units on the virtual reality device, where the image data of the same target image refers to image data on the same memory address.
As shown in fig. 2, in the present embodiment, the processing unit includes: an input buffer module, an image data conversion module, an output module, a control module, a rearrangement fetch module and a rearrangement display time sequence module,
the input buffer module is used for receiving the image data according to the control signal of the control module and transmitting the image data to the image data conversion module; the input buffer module is also called input BUFF.
The image data conversion module is used for reading data from the input buffer module, converting the image data into a signal received by virtual reality equipment according to a control signal of the control module, and transmitting the signal received by the virtual reality equipment to the output module;
the output module is used for transmitting the signals received by the virtual reality equipment to the corresponding display unit for displaying; the output module performs data transmission in a first-in first-out mode, and is also called as output FIFO;
the control module is used for controlling the input buffer module, the image data conversion module, the output module re-arrangement module and the re-arrangement display time sequence module, and sending corresponding control signals to the input buffer module, the image data conversion module, the output module re-arrangement module and the re-arrangement display time sequence module;
the rearrangement access module is used for acquiring an access address of the image data according to a control signal of the control module;
and the rearrangement display time sequence module is used for reading the signals received by the virtual reality equipment from the output module according to the display time sequence information sent by the control module.
In this embodiment, the display units include two display units, each display unit includes a first display unit and a second display unit, and correspondingly, the processing units include a first processing unit and a second processing unit, the first display unit is associated with the first processing unit in a mapping manner, and the second display unit is associated with the second processing unit in a mapping manner.
In this embodiment, the display areas of different display units do not overlap with each other, and the target image is a 2D image.
In this embodiment, the display unit includes a single-screen dual-display unit or a dual-screen dual-display unit, where the single-screen dual-display unit refers to that two display units are on the same display screen, for example, one display screen is divided into a left display unit and a right display unit; the double-screen double-display unit refers to two display units on two different display screens, such as a glasses type virtual reality device.
As shown in fig. 3, when the display unit is a single-screen dual-display unit, the step S20 includes:
s21, the first display unit informs the first processing unit to refresh the target image in a transverse scanning mode;
s22, the second display unit informs the second processing unit to refresh the target image in a transverse scanning mode;
s23, alternately reading the same image data in the memory row by the first processing unit and the second processing unit, and converting the same image data into an electric timing signal;
for example, as shown in fig. 4, a schematic structure diagram of a virtual reality device in a single-screen dual-display mode is shown, where a display unit is divided into a first display unit (may also be referred to as a left-eye display unit) and a second display unit (may also be referred to as a right-eye display unit), a processing unit is divided into a first processing unit and a second processing unit, and the first processing unit corresponds to the first display unit and the second display unit, and the first processing unit includes: the first input buffer module, the first image data conversion module, the first output module, the first control module, the first rearrangement access module and the first rearrangement display time sequence module, the second processing unit includes: as another embodiment, the first control module and the second control module, the first rearrangement access module and the second rearrangement display time sequence module, and the first rearrangement display time sequence module and the second rearrangement access module, and the first rearrangement display time sequence module and the second rearrangement display time sequence module share one rearrangement display time sequence module, and the first rearrangement display time sequence module and the second rearrangement display time sequence module may also be separately disposed in two processing units, respectively.
As shown in fig. 4, in the single-screen dual-display mode, the data format of the frame buffer is RGB888 (each pixel is 4 Bytes), the timing of the DDR access memory interface is set to burst4, the AXI bus bit width is 64 bits, after the control module sends out the start signal, the first address sent by the rearrangement access module is Addr = the first address of the frame buffer, the second access address Addr = the first address of the frame buffer +4 (Burst 4) × 8 (8 Bytes are transmitted each time), as long as the control signal of the control module is valid (if the access is too fast, the input buffer module is full, the calculation is suspended), the rearrangement access module calculates the transmission addresses in sequence until the last Burst transmission of the first line, and the last Burst transmission of a line needs to calculate whether the Burst is complete according to the resolution, if the burst is not exactly one, a mask signal is also sent to shield the mask signal of the irrelevant byte from the memory access interface; after the first line of data of the content of the first display unit is transmitted, the re-fetching module switches the first address of the frame buffer again to continue the address of the second display unit for displaying the fetched data, in the same way as the first display unit, and when the first line of the second display unit is ended, the second line of data of the frame buffer is switched to, at this time, the Stride step size needs to be calculated according to the arrangement mode of the frame buffer in the DDR and the last burst mask to drop several data, and Addr, which is sent at this time, = the first address of the frame buffer + the resolution column 4 (each pixel occupies 4 Bytes) + Stride, and so on in sequence.
In the display unit in fig. 4, the labels 1, 3, 5, and 7 in the first display unit and 2, 4, 6, and 8 in the second display unit are used to indicate the reading timing of each row of data in the display unit, and do not mean that each row of data in the two display units is different, but the content of the first row of data (the row with timing labeled as 1) in the first display unit is identical to the content of the first row of data (the row with timing labeled as 2) in the second display unit, and the content of the data in each next row is identical.
As shown in fig. 5, when the display unit is a dual-screen dual-display unit, the step S20 includes:
s201, a first display unit informs a first processing unit to refresh a target image in a transverse scanning mode;
s202, the second display unit informs the second processing unit to refresh the target image in a transverse scanning mode;
and S203, the first processing unit and the second processing unit read the same image data in the memory line by line simultaneously and convert the same image data into electric timing signals.
As shown in fig. 6, the schematic structure diagram of a virtual reality device in a dual-screen dual-display mode is shown, in which a display unit is divided into a first display unit (also referred to as a left-eye display unit) and a second display unit (also referred to as a right-eye display unit), a processing unit is divided into a first processing unit and a second processing unit, and the first processing unit corresponds to the first display unit and the second display unit, respectively, and the first processing unit includes: the first input buffer module, the first image data conversion module, the first output module, the first control module, the first rearrangement access module and the first rearrangement display time sequence module, the second processing unit includes: as another embodiment, the first control module and the second control module, the first rearrangement access module and the second rearrangement display time sequence module, and the first rearrangement display time sequence module and the second rearrangement access module, and the first rearrangement display time sequence module and the second rearrangement display time sequence module share one rearrangement display time sequence module, and the first rearrangement display time sequence module and the second rearrangement display time sequence module may also be separately disposed in two processing units, respectively.
As shown in fig. 6, in the dual-screen dual-display mode, the first display unit and the second display unit do not need to alternately fetch data in a row unit, and can fetch data simultaneously, and the data is fetched to the tail of the last row through the first input buffer module and the second input buffer module respectively, and a mask signal is also calculated and sent to the input buffer module.
In this embodiment, as shown in fig. 4 and fig. 6, the control module is specifically configured to generate the state and control signal of calculation and data transmission in the whole display process, and an interface with software, the software configures the display resolution, whether single-screen dual display or dual-screen dual display, and rearranges the frame buffer address of the number, the data format, the storage mode, whether scaling and color space conversion calculation are needed, and the related coordination parameters synchronized with the timing sequence of the LCD, etc. through the module; the control module controls the work of all modules, monitors the empty and full states of the input BUFF and the output FIFO, adjusts the rearrangement access module and the rearrangement display time sequence module in time, and controls the calculation of the rearrangement access module according to the signals of the rearrangement display time sequence module, thereby achieving the work of coordination and order.
The rearrangement access module is specifically used for receiving a control signal of the control module, a first address of a frame buffer area, a color format (RGB 888 or YUV421 and the like), a data arrangement mode and a starting time point of conversion of a rearrangement address; the module alternately calculates the fetch address of the frame buffer area according to the first address and the color format of the frame buffer BUFF and the control signal of the control module.
The rearrangement display time sequence module is specifically used for calculating frame line blanking according to the LCD relevant configuration information and the starting signal sent by the control module, according to the horizontal and vertical time sequence characteristics of the LCD, sending a ready signal to the control module, controlling a display time sequence synchronous interface to send data to the display from the output FIFO according to the time sequence requirements of the LCD, and when a single-screen double-display mode is received and sent by the control module, the rearrangement display time sequence module needs to alternately take the data with the data volume of 1 line from 2 output FIFOs according to SYNC signals; when a double-screen double-eye mode is received from the control module, the rearrangement display timing sequence module fetches data from the corresponding output FIFO according to SYNC signals of 2 LCDs of the left-eye LCD and the right-eye LCD, and meanwhile, the display timing sequence synchronous interface is controlled to send out correct interface signals such as DE (data enable) and the like so that mistaken data receiving does not occur between different LCDs.
The access interface is used for receiving the DDR access requirement of the rearrangement access module, sending the access operation of the AXI of the DDR according to the address given by the rearrangement access module, transmitting the read data to the left and right eye input BUFFs, and simultaneously transmitting the corresponding mask information to the left and right eye input BUFFs through the AXI standard protocol.
The input buffer module (input BUFF) is specifically used for determining whether to receive data sent by the memory access interface according to a control signal of the control module, and determining which bytes data need to be written into the address of the corresponding pointer according to a mask signal. In the single-screen double-display mode, only one input BUFF receives a receipt at each time according to a signal of the control module; under the double-screen double-display mode, the left and right eye input BUFFs can simultaneously receive data sent by the memory access interface. And managing the read-write pointer of the BUFF, and sending a state signal to the control module once the FIFO is empty and full.
The output module (output FIFO) is specifically used for receiving the data of the image data conversion module, then sending the data to the display timing synchronization interface, managing the read-write pointer of the FIFO, and sending a state signal to the control module to coordinate the rearrangement display timing module and the rearrangement access module once the FIFO is empty and full.
The display time sequence synchronous interface is mainly used for timely fetching data from the output FIFO according to the requirement of the display LCD according to the control of the rearrangement display time sequence module and then forwarding the data to the external LCD display equipment according to the time sequence of the LCD. The module will send out different de (data enable) control signals to different LCDs to enable the LCDs to mask out data from another LCD that is not associated with the LCD. For single-screen double display, the module only outputs 1 group of control timing signals of the display equipment and 1 VSYNC synchronous signal to the rearrangement display timing module (a group of signals are invalid and in a high-impedance state); for dual-screen dual display, the module outputs 2 sets of control timing signals for the display devices and 2 VSYNC synchronization signals to the rearrangement display timing module.
In this embodiment, before converting the image data into a signal accepted by the virtual reality device, the method further includes:
adjusting the size and the position of the display unit;
and scaling the size of the image data, and scaling the content which is not equal to the resolution of the display unit into a certain display unit through a scaling mechanism.
In this embodiment, in step S30, the data is converted by the image data conversion module and then placed into the output FIFO, the display timing synchronization interface converts the data in the FIFO into a digital level signal and sends the digital level signal to the virtual reality device, and displays a line of image information on the display unit in a horizontal scanning manner, after reading each line of data, it is determined whether the line is changed to the last line, if so, the virtual reality device sends a VSYNC synchronization signal to the display timing synchronization interface to indicate that the display of a frame of image is completed, otherwise, the virtual reality device sends a HSYNC synchronization signal to the display timing synchronization interface, and continues to read the next line of data.
In this embodiment, the scanning mode for reading data is horizontal scanning, that is, line scanning, and as another embodiment, the scanning mode for reading data can also be vertical scanning, that is, column scanning.
In this embodiment, the signal received by the virtual reality device is a digital signal or an analog signal.
As another embodiment, the number of the display units may also be two or more, taking a single-screen dual-display unit as an example, the single screen may be divided into two left and right display units with equal size and position relationship, and when the content to be displayed is not equal to the resolution of the display unit, the content to be displayed may be zoomed into each display unit through a zoom mechanism for display.
Example two
As shown in fig. 7, in the present embodiment, an image display system based on a virtual reality device includes: a virtual reality device 10 and an image processing apparatus 20, the virtual reality device and the image processing apparatus being connected by a display timing synchronization interface, wherein,
the virtual reality equipment comprises at least two display units, a virtual reality display unit and a virtual reality display unit, wherein the at least two display units are used for receiving and displaying image data sent by the image processing device;
the image processing device comprises at least two processing units, a virtual reality device and a display unit, wherein the processing units are used for reading image data of a target image from a memory, converting the image data into signals received by the virtual reality device and transmitting the signals to the at least two display units for display;
the at least two processing units correspond to the at least two display units one by one; each display unit reads image data of the same target image through a corresponding processing unit.
As shown in fig. 2, in the present embodiment, the processing unit includes: an input buffer module, an image data conversion module, an output module, a control module, a rearrangement fetch module and a rearrangement display time sequence module,
the input buffer module is used for receiving the image data according to the control signal of the control module and transmitting the image data to the image data conversion module; the input buffer module is also called input BUFF.
The image data conversion module is used for reading data from the input buffer module, converting the image data into a signal received by virtual reality equipment according to a control signal of the control module, and transmitting the signal received by the virtual reality equipment to the output module;
the output module is used for transmitting the signals received by the virtual reality equipment to the corresponding display unit for displaying; the output module performs data transmission in a first-in first-out mode, and is also called as output FIFO;
the control module is used for controlling the input buffer module, the image data conversion module, the output module re-arrangement module and the re-arrangement display time sequence module, and sending corresponding control signals to the input buffer module, the image data conversion module, the output module re-arrangement module and the re-arrangement display time sequence module;
the rearrangement access module is used for acquiring an access address of the image data according to a control signal of the control module;
and the rearrangement display time sequence module is used for reading the signals received by the virtual reality equipment from the output module according to the display time sequence information sent by the control module.
In this embodiment, the display units include two display units, each display unit includes a first display unit and a second display unit, and correspondingly, the processing units include a first processing unit and a second processing unit, the first display unit is associated with the first processing unit in a mapping manner, and the second display unit is associated with the second processing unit in a mapping manner.
In this embodiment, the display areas of different display units do not overlap with each other, and the target image is a 2D image.
In this embodiment, the display unit includes a single-screen dual-display unit or a dual-screen dual-display unit, where the single-screen dual-display unit means that two display units are on the same display screen, and the dual-screen dual-display unit means that two display units are on two different display screens, such as a glasses-type virtual reality device.
When the display unit is a single-screen dual-display unit, reading image data of the same target image by each display unit through the corresponding processing unit comprises:
the first display unit informs the first processing unit to refresh the target image in a transverse scanning mode;
the second display unit informs the second processing unit to refresh the target image in a transverse scanning mode;
the first processing unit and the second processing unit alternately read the same image data in the memory line by line and convert the same image data into electrical timing signals;
for example, as shown in fig. 4, a schematic structure diagram of a virtual reality device in a single-screen dual-display mode is shown, where a display unit is divided into a first display unit (may also be referred to as a left-eye display unit) and a second display unit (may also be referred to as a right-eye display unit), a processing unit is divided into a first processing unit and a second processing unit, and the first processing unit corresponds to the first display unit and the second display unit, and the first processing unit includes: the first input buffer module, the first image data conversion module, the first output module, the first control module, the first rearrangement access module and the first rearrangement display time sequence module, the second processing unit includes: as another embodiment, the first control module and the second control module, the first rearrangement access module and the second rearrangement display time sequence module, and the first rearrangement display time sequence module and the second rearrangement access module, and the first rearrangement display time sequence module and the second rearrangement display time sequence module share one rearrangement display time sequence module, and the first rearrangement display time sequence module and the second rearrangement display time sequence module may also be separately disposed in two processing units, respectively.
As shown in fig. 4, in the single-screen dual-display mode, the data format of the frame buffer is RGB888 (each pixel is 4 Bytes), the timing of the DDR access memory interface is set to burst4, the AXI bus bit width is 64 bits, after the control module sends out the start signal, the first address sent by the rearrangement access module is Addr = the first address of the frame buffer, the second access address Addr = the first address of the frame buffer +4 (Burst 4) × 8 (8 Bytes are transmitted each time), as long as the control signal of the control module is valid (if the access is too fast, the input buffer module is full, the calculation is suspended), the rearrangement access module calculates the transmission addresses in sequence until the last Burst transmission of the first line, and the last Burst transmission of a line needs to calculate whether the Burst is complete according to the resolution, if not, sending a mask signal to the memory access interface for shielding the mask signal of the irrelevant byte; after the first line of data of the content of the first display unit is transmitted, the re-fetching module switches the first address of the frame buffer again to continue the address of the second display unit for displaying the fetched data, in the same way as the first display unit, and when the first line of the second display unit is ended, the second line of data of the frame buffer is switched to, at this time, the Stride step size needs to be calculated according to the arrangement mode of the frame buffer in the DDR and the last burst mask to drop several data, and Addr, which is sent at this time, = the first address of the frame buffer + the resolution column 4 (each pixel occupies 4 Bytes) + Stride, and so on in sequence.
In the display unit in fig. 4, the labels 1, 3, 5, and 7 in the first display unit and 2, 4, 6, and 8 in the second display unit are used to indicate the reading timing of each row of data in the display unit, and do not mean that each row of data in the two display units is different, but the content of the first row of data (the row with timing labeled as 1) in the first display unit is identical to the content of the first row of data (the row with timing labeled as 2) in the second display unit, and the content of the data in each next row is identical.
When the display unit is a dual-screen dual-display unit, reading image data of the same target image by each display unit through the corresponding processing unit comprises:
the first display unit informs the first processing unit to refresh the target image in a transverse scanning mode;
the second display unit informs the second processing unit to refresh the target image in a transverse scanning mode;
the first processing unit and the second processing unit read the same image data in the memory line by line simultaneously and convert the same image data into electrical timing signals.
As shown in fig. 6, the schematic structure diagram of a virtual reality device in a dual-screen dual-display mode is shown, in which a display unit is divided into a first display unit (also referred to as a left-eye display unit) and a second display unit (also referred to as a right-eye display unit), a processing unit is divided into a first processing unit and a second processing unit, and the first processing unit corresponds to the first display unit and the second display unit, respectively, and the first processing unit includes: the first input buffer module, the first image data conversion module, the first output module, the first control module, the first rearrangement access module and the first rearrangement display time sequence module, the second processing unit includes: as another embodiment, the first control module and the second control module, the first rearrangement access module and the second rearrangement display time sequence module, and the first rearrangement display time sequence module and the second rearrangement access module, and the first rearrangement display time sequence module and the second rearrangement display time sequence module share one rearrangement display time sequence module, and the first rearrangement display time sequence module and the second rearrangement display time sequence module may also be separately disposed in two processing units, respectively.
As shown in fig. 6, in the dual-screen dual-display mode, the first display unit and the second display unit do not need to alternately fetch data in a row unit, and can fetch data simultaneously, and the data is fetched to the tail of the last row through the first input buffer module and the second input buffer module respectively, and a mask signal is also calculated and sent to the input buffer module.
In this embodiment, as shown in fig. 4 and fig. 6, the control module is specifically configured to generate the state and control signal of calculation and data transmission in the whole display process, and an interface with software, the software configures the display resolution, whether single-screen dual display or dual-screen dual display, and rearranges the frame buffer address of the number, the data format, the storage mode, whether scaling and color space conversion calculation are needed, and the related coordination parameters synchronized with the timing sequence of the LCD, etc. through the module; the control module controls the work of all modules, monitors the empty and full states of the input BUFF and the output FIFO, adjusts the rearrangement access module and the rearrangement display time sequence module in time, and controls the calculation of the rearrangement access module according to the signals of the rearrangement display time sequence module, thereby achieving the work of coordination and order.
The rearrangement access module is specifically used for receiving a control signal of the control module, a first address of a frame buffer area, a color format (RGB 888 or YUV421 and the like), a data arrangement mode and a starting time point of conversion of a rearrangement address; the module alternately calculates the fetch address of the frame buffer area according to the first address and the color format of the frame buffer BUFF and the control signal of the control module.
The rearrangement display time sequence module is specifically used for calculating frame line blanking according to the LCD relevant configuration information and the starting signal sent by the control module, according to the horizontal and vertical time sequence characteristics of the LCD, sending a ready signal to the control module, controlling a display time sequence synchronous interface to send data to the display from the output FIFO according to the time sequence requirements of the LCD, and when a single-screen double-display mode is received and sent by the control module, the rearrangement display time sequence module needs to alternately take the data with the data volume of 1 line from 2 output FIFOs according to SYNC signals; when a double-screen double-eye mode is received from the control module, the rearrangement display timing sequence module fetches data from the corresponding output FIFO according to SYNC signals of 2 LCDs of the left-eye LCD and the right-eye LCD, and meanwhile, the display timing sequence synchronous interface is controlled to send out correct interface signals such as DE (data enable) and the like so that mistaken data receiving does not occur between different LCDs.
The access interface is used for receiving the DDR access requirement of the rearrangement access module, sending the access operation of the AXI of the DDR according to the address given by the rearrangement access module, transmitting the read data to the left and right eye input BUFFs, and simultaneously transmitting the corresponding mask information to the left and right eye input BUFFs through the AXI standard protocol.
The input buffer module (input BUFF) is specifically used for determining whether to receive data sent by the memory access interface according to a control signal of the control module, and determining which bytes data need to be written into the address of the corresponding pointer according to a mask signal. In the single-screen double-display mode, only one input BUFF receives a receipt at each time according to a signal of the control module; under the double-screen double-display mode, the left and right eye input BUFFs can simultaneously receive data sent by the memory access interface. And managing the read-write pointer of the BUFF, and sending a state signal to the control module once the FIFO is empty and full.
The output module (output FIFO) is specifically used for receiving the data of the image data conversion module, then sending the data to the display timing synchronization interface, managing the read-write pointer of the FIFO, and sending a state signal to the control module to coordinate the rearrangement display timing module and the rearrangement access module once the FIFO is empty and full.
The display time sequence synchronous interface is mainly used for timely fetching data from the output FIFO according to the requirement of the display LCD according to the control of the rearrangement display time sequence module and then forwarding the data to the external LCD display equipment according to the time sequence of the LCD. The module will send out different de (data enable) control signals to different LCDs to enable the LCDs to mask out data from another LCD that is not associated with the LCD. For single-screen double display, the module only outputs 1 group of control timing signals of the display equipment and 1 VSYNC synchronous signal to the rearrangement display timing module (a group of signals are invalid and in a high-impedance state); for dual-screen dual display, the module outputs 2 sets of control timing signals for the display devices and 2 VSYNC synchronization signals to the rearrangement display timing module.
As shown in fig. 8, in the present embodiment, the image processing apparatus further includes:
the adjusting unit is used for adjusting the size and the position of the display unit;
and the scaling unit is used for scaling the size of the image data and scaling the content which is not equal to the resolution of the display unit into a certain display unit through a scaling mechanism.
In this embodiment, the scanning mode for reading data is horizontal scanning, that is, line scanning, and as another embodiment, the scanning mode for reading data can also be vertical scanning, that is, column scanning.
In this embodiment, the signal received by the virtual reality device is a digital signal or an analog signal.
As another embodiment, the number of the display units may be two or more.
In this embodiment, the image rearrangement apparatus reads image data from a memory through a memory access interface; as another embodiment, the image rearrangement apparatus may also read the image data from the frame buffer through the memory access interface.
In this embodiment, the system implements a function of displaying the same content on different display units of the virtual reality device, and can directly and synchronously deliver the same display content to a plurality of displayable regions of the virtual reality device, and directly rearrange the 2D content in the display process, without additional overhead and additional access (write back to memory) operations.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (such as a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present invention.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (8)
1. An image display method based on virtual reality equipment is characterized by comprising the following steps:
the virtual reality equipment comprises at least two display units and at least two processing units which are in one-to-one correspondence with the at least two display units;
wherein each of the processing units comprises: the device comprises an input buffer module, an image data conversion module, an output module, a control module, a rearrangement access module and a rearrangement display time sequence module;
the input buffer module is used for receiving the image data according to the control signal of the control module and transmitting the image data to the image data conversion module;
the image data conversion module is used for reading data from the input buffer module, converting the image data into a signal received by virtual reality equipment according to a control signal of the control module, and transmitting the signal received by the virtual reality equipment to the output module;
the output module is used for transmitting the signals received by the virtual reality equipment to the corresponding display unit for displaying;
the control module is used for controlling the input buffer module, the image data conversion module, the output module re-arrangement module and the re-arrangement display time sequence module, and sending corresponding control signals to the input buffer module, the image data conversion module, the output module re-arrangement module and the re-arrangement display time sequence module;
the rearrangement access module is used for acquiring an access address of the image data according to a control signal of the control module;
the rearrangement display time sequence module is used for reading a signal received by the virtual reality equipment from the output module according to the display time sequence information sent by the control module;
each display unit respectively reads the same image data of the same target image through the same access address provided by the corresponding processing unit;
the processing unit converts the image data into signals received by the virtual reality equipment, and transmits the signals to the at least two display units for display, and the display content of each line of the at least two display units is completely the same.
2. The image display method based on the virtual reality device according to claim 1, wherein the number of the display units is two, and the image display method comprises a first display unit and a second display unit, and correspondingly, the number of the processing units is two, and the image display method comprises a first processing unit and a second processing unit, wherein the first display unit is associated with the first processing unit in a mapping manner, and the second display unit is associated with the second processing unit in a mapping manner.
3. The image display method based on virtual reality equipment according to claim 2, wherein the display unit comprises a single-screen dual-display unit or a dual-screen dual-display unit, wherein,
when the display unit is a single-screen dual-display unit, reading image data of the same target image by each display unit through the corresponding processing unit comprises:
the first display unit informs the first processing unit to refresh the target image in a transverse scanning mode;
the second display unit informs the second processing unit to refresh the target image in a transverse scanning mode;
the first processing unit and the second processing unit alternately read the same image data in the memory line by line and convert the same image data into electrical timing signals;
when the display unit is a dual-screen dual-display unit, reading image data of the same target image by each display unit through the corresponding processing unit comprises:
the first display unit informs the first processing unit to refresh the target image in a transverse scanning mode;
the second display unit informs the second processing unit to refresh the target image in a transverse scanning mode;
the first processing unit and the second processing unit read the same image data in the memory line by line simultaneously and convert the same image data into electrical timing signals.
4. The image display method based on the virtual reality device according to claim 1, wherein before converting the image data into the signal accepted by the virtual reality device, the method further comprises:
adjusting the size and the position of the display unit;
scaling a size of the image data.
5. An image display system based on a virtual reality device, comprising: a virtual reality device and an image processing apparatus, the virtual reality device and the image processing apparatus being connected by a display timing synchronization interface, wherein,
the virtual reality equipment comprises at least two display units, a virtual reality display unit and a virtual reality display unit, wherein the at least two display units are used for receiving and displaying image data sent by the image processing device;
the image processing device comprises at least two processing units, a virtual reality device and a display unit, wherein the processing units are used for reading image data of a target image from a memory, converting the image data into signals received by the virtual reality device and transmitting the signals to the at least two display units for display;
the at least two processing units correspond to the at least two display units one by one; each display unit reads the image data of the same target image through the corresponding processing unit; the processing unit includes: an input buffer module, an image data conversion module, an output module, a control module, a rearrangement fetch module and a rearrangement display time sequence module,
the input buffer module is used for receiving the image data according to the control signal of the control module and transmitting the image data to the image data conversion module;
the image data conversion module is used for reading data from the input buffer module, converting the image data into a signal received by virtual reality equipment according to a control signal of the control module, and transmitting the signal received by the virtual reality equipment to the output module;
the output module is used for transmitting the signals received by the virtual reality equipment to the corresponding display unit for displaying;
the control module is used for controlling the input buffer module, the image data conversion module, the output module re-arrangement module and the re-arrangement display time sequence module, and sending corresponding control signals to the input buffer module, the image data conversion module, the output module re-arrangement module and the re-arrangement display time sequence module;
the rearrangement access module is used for acquiring an access address of the image data according to a control signal of the control module;
and the rearrangement display time sequence module is used for reading the signals received by the virtual reality equipment from the output module according to the display time sequence information sent by the control module.
6. The image display system based on the virtual reality device, according to claim 5, wherein the number of the display units is two, and the image display system comprises a first display unit and a second display unit, and correspondingly, the number of the processing units is two, and the image display system comprises a first processing unit and a second processing unit, the first display unit is associated with the first processing unit in a mapping manner, and the second display unit is associated with the second processing unit in a mapping manner.
7. The virtual reality device-based image display system of claim 6, wherein the display unit comprises a single-screen dual-display unit or a dual-screen dual-display unit, wherein,
when the display unit is a single-screen dual-display unit, reading image data of the same target image by each display unit through the corresponding processing unit comprises:
the first display unit informs the first processing unit to refresh the target image in a transverse scanning mode;
the second display unit informs the second processing unit to refresh the target image in a transverse scanning mode;
the first processing unit and the second processing unit alternately read the same image data in the memory line by line and convert the same image data into electrical timing signals;
when the display unit is a dual-screen dual-display unit, reading image data of the same target image by each display unit through the corresponding processing unit comprises:
the first display unit informs the first processing unit to refresh the target image in a transverse scanning mode;
the second display unit informs the second processing unit to refresh the target image in a transverse scanning mode;
the first processing unit and the second processing unit read the same image data in the memory line by line simultaneously and convert the same image data into electrical timing signals.
8. The virtual reality device-based image display system according to claim 5, wherein the image processing apparatus further comprises:
the adjusting unit is used for adjusting the size and the position of the display unit;
a scaling unit for scaling the size of the image data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611209453.0A CN108243293B (en) | 2016-12-23 | 2016-12-23 | Image display method and system based on virtual reality equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611209453.0A CN108243293B (en) | 2016-12-23 | 2016-12-23 | Image display method and system based on virtual reality equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108243293A CN108243293A (en) | 2018-07-03 |
| CN108243293B true CN108243293B (en) | 2021-02-26 |
Family
ID=62704407
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611209453.0A Active CN108243293B (en) | 2016-12-23 | 2016-12-23 | Image display method and system based on virtual reality equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108243293B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109920519B (en) * | 2019-02-20 | 2023-05-02 | 东软医疗系统股份有限公司 | Method, device and equipment for processing image data |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR101525011B1 (en) * | 2014-10-07 | 2015-06-09 | 동국대학교 산학협력단 | tangible virtual reality display control device based on NUI, and method thereof |
| EP3062219A1 (en) * | 2015-02-25 | 2016-08-31 | BAE Systems PLC | A mixed reality system and method for displaying data therein |
| CN204989630U (en) * | 2015-09-29 | 2016-01-20 | 海南虚幻视界科技有限公司 | Detachable screen virtual reality glasses |
| CN105791810A (en) * | 2016-04-27 | 2016-07-20 | 深圳市高巨创新科技开发有限公司 | Virtual stereo display method and device |
| CN106162156B (en) * | 2016-07-26 | 2018-10-19 | 北京小鸟看看科技有限公司 | A kind of virtual reality system and its view angle regulating method and device |
-
2016
- 2016-12-23 CN CN201611209453.0A patent/CN108243293B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN108243293A (en) | 2018-07-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109891381B (en) | Dual path foveated graphics pipeline | |
| CN108449566B (en) | Video frame rate compensation by adjusting vertical blanking | |
| WO2016091082A1 (en) | Multi-screen joint display processing method and device | |
| US20070152992A1 (en) | Image data transmission apparatus and method for image display system | |
| US11217201B2 (en) | Video frame interfaces for logically-defined pixels | |
| JP2007219082A (en) | Composite reality feeling display system | |
| CN107249107B (en) | Video controller and image processing method and device | |
| CN102724433A (en) | Method and device for realizing multi-video signal image composition | |
| GB2538797B (en) | Managing display data | |
| US20200167119A1 (en) | Managing display data | |
| CN114020228A (en) | Screen display method and device | |
| CN102572463A (en) | Video signal processing device, video signal processing method, and computer program | |
| CN111988552B (en) | Image output control method and device and video processing equipment | |
| CN108243293B (en) | Image display method and system based on virtual reality equipment | |
| CN116467099B (en) | Information processing method and device based on inter-core communication, chip and electronic equipment | |
| EP3169067A1 (en) | Display processing system and method, and electronic device | |
| JP2015096920A (en) | Image processor and control method of image processing system | |
| CN108055480B (en) | Single-channel video signal input multi-window display | |
| CN109308862B (en) | Signal control method, control device, processing device and signal control equipment | |
| KR102213423B1 (en) | Apparatus and method for distributing of ultra high definition videos using scalers | |
| CN103577133B (en) | Ultra high-definition information display system and display packing thereof | |
| US20030098866A1 (en) | Apparatus and method for controlling a stereo 3D display using BITBLT operation | |
| KR100598957B1 (en) | Head mounted display for mobile terminal | |
| JP2005122119A (en) | Video interface device in system constituted of mpu and video codec | |
| CN111818380B (en) | Interactive synchronous image display method and system for micro display unit |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: Zone C, floor 1, plant 1, No.1, Keji 4th Road, Tangjiawan Town, high tech Zone, Zhuhai City, Guangdong Province 519085 Applicant after: ACTIONS TECHNOLOGY Co.,Ltd. Address before: 519085 Guangdong province Zhuhai high tech Zone Science and technology innovation Coastal Science and technology No. four Road No. 1, No. 1 workshop C area Applicant before: ACTIONS (ZHUHAI) TECHNOLOGY Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |