WO2023206098A1 - 光场数据的传输方法、光场通信设备及系统 - Google Patents

光场数据的传输方法、光场通信设备及系统 Download PDF

Info

Publication number
WO2023206098A1
WO2023206098A1 PCT/CN2022/089380 CN2022089380W WO2023206098A1 WO 2023206098 A1 WO2023206098 A1 WO 2023206098A1 CN 2022089380 W CN2022089380 W CN 2022089380W WO 2023206098 A1 WO2023206098 A1 WO 2023206098A1
Authority
WO
WIPO (PCT)
Prior art keywords
light field
camera
gaze point
communication device
point position
Prior art date
Application number
PCT/CN2022/089380
Other languages
English (en)
French (fr)
Inventor
王明东
李亚鹏
李扬冰
冯煊
马媛媛
赵方圆
Original Assignee
京东方科技集团股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 京东方科技集团股份有限公司 filed Critical 京东方科技集团股份有限公司
Priority to CN202280000932.9A priority Critical patent/CN117413512A/zh
Priority to PCT/CN2022/089380 priority patent/WO2023206098A1/zh
Publication of WO2023206098A1 publication Critical patent/WO2023206098A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30—Image reproducers
    • H04N13/366—Image reproducers using viewer tracking
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30—Image reproducers
    • H04N13/366—Image reproducers using viewer tracking
    • H04N13/383—Image reproducers using viewer tracking for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes

Definitions

  • the present application relates to the field of communication technology, and in particular to a light field data transmission method, light field communication equipment and system.
  • 3D video communication can be achieved through two light field display devices.
  • Each light field display device includes multiple cameras, and the light field data collected by the multiple cameras can be processed and then sent to another light field display device.
  • the other light field display device can then display the light field data in a 3D form.
  • This application provides a light field data transmission method, light field communication equipment and system.
  • the technical solutions are as follows:
  • an image processing model training method which is applied to a light field communication device, where the light field communication device includes a plurality of first cameras; the method includes:
  • the first gaze point position is the gaze point of the eye of the viewing object of the light field display device on the light field display screen of the light field display device.
  • the first light field data collected by the at least one target camera is processed and then transmitted to the light field display device.
  • the processed first light field data is used for light field display by the light field display device. .
  • the light field communication device stores a corresponding relationship between the gaze point position and the camera, and a plurality of different gaze point positions are recorded in the correspondence relationship, each gaze point position corresponding to at least one first camera, and The number of at least one first camera corresponding to each gaze point position is smaller than the number of the plurality of first cameras;
  • Determining at least one target camera corresponding to the first gaze point position from the plurality of first cameras includes:
  • At least one target camera corresponding to the first gaze point position is determined.
  • the light field communication device further includes: a plurality of data processing chips, each data processing chip being connected to at least one first camera;
  • the first light field data collected by the at least one target camera is processed by at least one target data processing chip among the plurality of data processing chips, and the at least one target data processing chip is in contact with the at least one target camera. connect.
  • the first gaze point position corresponds to multiple target cameras
  • the plurality of target cameras are connected to a plurality of different target data processing chips in a one-to-one correspondence.
  • each data processing chip is connected to multiple first cameras;
  • the at least one target data processing chip is controlled to process the first light field data collected by the at least one target camera, and not to process the first camera other than the at least one target camera.
  • the method before processing the first light field data collected by the at least one target camera, the method further includes:
  • the light field communication device further includes an eye tracking camera and a light field display; the method further includes:
  • the second gaze point position is obtained through the eye tracking camera.
  • the second gaze point position is the gaze point of the eye of the viewing object of the light field communication device on the light field display screen of the light field communication device. s position;
  • Receive second light field data transmitted by the light field display device the second light field data is collected by at least one second camera in the light field display device, and the at least one second camera is based on the The position of the second gaze point is determined;
  • Light field display is performed based on the second light field data.
  • the light field communication device includes: a main control circuit, a data processing circuit and a plurality of first cameras; the main control circuit is used for:
  • the first gaze point position is the gaze point of the eye of the viewing object of the light field display device on the light field display screen of the light field display device.
  • the data processing circuit is used to process the first light field data collected by the at least one target camera and then transmit it to the light field display device, and the processed first light field data is used for the The light field display device performs light field display.
  • the main control circuit stores a corresponding relationship between the gaze point position and the camera.
  • a plurality of different gaze point positions are recorded in the correspondence relationship.
  • Each gaze point position corresponds to at least one first camera, and corresponds to The number of at least one first camera is less than the number of the plurality of first cameras;
  • the main control circuit is configured to determine at least one target camera corresponding to the first gaze point position according to the corresponding relationship.
  • the data processing circuit includes a plurality of data processing chips, each data processing chip being connected to at least one first camera;
  • At least one target data processing chip among the plurality of data processing chips is connected to the at least one target camera, and the at least one target data processing chip is used to process the first light field data collected by the at least one target camera. deal with.
  • the first gaze point position corresponds to multiple target cameras
  • the plurality of target cameras are connected to a plurality of different target data processing chips in a one-to-one correspondence.
  • each data processing chip is connected to multiple first cameras; the main control circuit is used for:
  • the at least one target data processing chip is controlled to process the first light field data collected by the at least one target camera, and not to process the first camera other than the at least one target camera.
  • the main control circuit is also used to:
  • the light field communication device further includes an eye tracking camera, a display processing circuit and a light field display screen;
  • the eye tracking camera is used to obtain a second gaze point position and transmit the second gaze point position to the light field display device.
  • the second gaze point position is the viewing angle of the light field communication device. The position of the gaze point of the subject's eyes on the light field display screen of the light field communication device;
  • the display processing circuit is configured to receive and process second light field data transmitted by the light field display device, where the second light field data is collected by at least one second camera in the light field display device, And the at least one second camera is determined based on the second gaze point position;
  • the light field display screen is used to perform light field display based on the second light field data.
  • a light field data transmission system including: a light field communication device and a light field display device;
  • the light field communication device is the light field communication device described in the above aspect
  • the light field display device is configured to send a first gaze point position to the light field communication device, and perform light field display based on the first light field data transmitted by the light field communication device.
  • a light field communication device in yet another aspect, includes: a plurality of first cameras, a processor, and a memory. Instructions are stored in the memory, and the instructions are loaded by the processor and Executed to implement the light field data transmission method as described in the above aspect.
  • a computer-readable storage medium is provided. Instructions are stored in the storage medium, and the instructions are loaded and executed by a processor to implement the light field data transmission method as described in the above aspect.
  • Figure 1 is a schematic structural diagram of a light field data transmission system provided by an embodiment of the present application.
  • Figure 2 is a schematic flowchart of a light field data transmission method provided by an embodiment of the present application
  • Figure 3 is a schematic flow chart of another light field data transmission method provided by an embodiment of the present application.
  • Figure 4 is a schematic structural diagram of another light field data transmission system provided by an embodiment of the present application.
  • Figure 5 is a schematic diagram of light field data transmission provided by an embodiment of the present application.
  • Figure 6 is a schematic diagram of a first camera collecting light field data provided by an embodiment of the present application.
  • Figure 7 is a schematic diagram of a light field display device displaying light field data provided by an embodiment of the present application.
  • Figure 8 is a connection topology diagram of a camera and a data processing chip in a light field communication device provided by an embodiment of the present application;
  • Figure 9 is a structural block diagram of a light field communication device provided by an embodiment of the present application.
  • FIG. 1 is a schematic structural diagram of a light field data transmission system provided by an embodiment of the present application.
  • the system includes: a light field communication device 10 and a light field display device 20 .
  • a wired or wireless communication connection is established between the light field communication device 10 and the light field display device 20 .
  • the light field communication device 10 may be a communication device having multiple cameras (for example, light field cameras) and capable of processing and transmitting light field data collected by the multiple cameras.
  • the light field communication device 10 can be a mobile phone, a tablet computer, a notebook computer, a desktop computer, a vehicle-mounted terminal or a wearable device, etc.
  • the light field display device 20 may be a display device having a light field display screen.
  • the light field display screen in the light field display device 20 may be a flat display screen.
  • the light field display device 20 can determine the first gaze point position of the viewing object's eye on the light field display screen, and send the first gaze point position to the light field communication device 10 .
  • the light field communication device 10 can determine at least one target camera corresponding to the first gaze point from a plurality of first cameras based on the first gaze point position sent by the light field display device 20, and collect data from the at least one target camera.
  • the obtained first light field data is processed and then transmitted to the light field display device 20 .
  • the light field display device 20 can further perform light field display based on the first light field data.
  • the above example uses the light field communication device 10 as a light field data collection device to provide light field data to the light field display device 20 , and uses the light field display device 20 as a display device to perform operations based on the light field data provided by the light field communication device 10
  • the light field display is explained as an example.
  • the light field communication device 10 may also include a light field display screen, and the light field communication device 10 may send a second gaze point position to the light field display device 20, where the second gaze point position is the light field communication device.
  • the light field display device 20 has a plurality of second cameras.
  • the light field display device 20 can determine at least one target camera corresponding to the second gaze point from the plurality of second cameras based on the second gaze point position, and determine The second light field data collected by the at least one target camera is processed and then transmitted to the light field communication device 10 .
  • the light field communication device 10 can further perform light field display on the second light field data.
  • bidirectional transmission and display of light field data between the light field communication device 10 and the light field display device 20 can be achieved.
  • the light field communication device 10 and the light field display device 20 can implement 3D video communication.
  • FIG. 2 is a schematic flowchart of a light field data transmission method provided by an example of this application. This method can be applied to a light field data transmission system, such as the light field data transmission system shown in FIG. 1 .
  • the method includes:
  • Step 101 The light field display device sends the first gaze point position to the light field communication device.
  • the light field display device can track the position of the eye (such as the pupil) of the viewing object to determine the position of the gaze point of the eye on the light field display screen, that is, the first gaze point. Location.
  • the first gaze point position may be the projection position of the viewing object's pupil on the light field display screen.
  • the light field display device can send the first gaze point position to the light field communication device.
  • the light field display device may obtain and send the first gaze point position after establishing a communication connection with the light field communication device.
  • the light field display device can acquire and send the first gaze point position in real time, or the light field display device can acquire and send the first gaze point position according to a preset tracking cycle.
  • Step 102 The light field communication device determines at least one target camera corresponding to the first gaze point position from the plurality of first cameras.
  • the light field communication device includes a plurality of first cameras.
  • the plurality of first cameras are used to collect light field data.
  • the plurality of first cameras may be light field cameras, and the light field data collected by the light field cameras may be image data.
  • the light field communication device obtains the first gaze point position fed back by the light field display device, it can determine at least one target camera corresponding to the first gaze point position from the plurality of first cameras based on the first gaze point position. .
  • the light field communication device may pre-store the corresponding relationship between the gaze point position and the camera.
  • the light field communication device can directly determine at least one target camera corresponding to the first gaze point position based on the corresponding relationship.
  • the light field communication device may determine the field of view range of the viewing object based on the first gaze point position, and then determine at least one target camera based on the field of view range and the light field data collection range of each first camera.
  • the display area of the first light field data collected by the at least one target camera on the light field display screen can cover the field of view range.
  • the number of the at least one target camera is less than the number of the plurality of first cameras.
  • the number of first cameras included in the light field communication device may be 10, and the number of target cameras may be 4.
  • the light field communication device can control the plurality of first cameras to collect light field data after establishing a communication connection with the light field display device.
  • Step 103 The light field communication device processes the first light field data collected by at least one target camera and then transmits it to the light field display device.
  • the light field communication device After the light field communication device determines at least one target camera from the plurality of first cameras, it can obtain the first light field data collected by the at least one target camera and process it. Afterwards, the light field communication device can transmit the processed first light field data to the light field display device.
  • the processing that the light field communication device can perform on the first light field data may include: distortion correction processing, projection transformation processing, color correction processing, encoding processing, etc.
  • Step 104 The light field display device performs light field display on the processed first light field data.
  • the light field display device can perform light field display on the received first light field data on its light field display screen through light field display technology, so as to realize the display of 3D images.
  • the light field communication device can only process the first light field data collected by at least one target camera corresponding to the gaze point position of the viewing object, and transmit it to the light field display device for light field display. As a result, redundant light field data can be avoided from occupying too much transmission bandwidth while ensuring viewing effects.
  • embodiments of the present application provide a method for transmitting light field data.
  • the light field communication device can, based on the first gaze point position fed back by the light field display device, at least one target corresponding to the first gaze point position.
  • the first light field data collected by the camera is processed and then transmitted to the light field display device.
  • the light field display device can then perform light field display on the first light field data on its light field display screen. Since the light field communication device only needs to process and transmit the light field data collected by the light field camera corresponding to the first gaze point position, it can effectively improve the efficiency of light field data processing without affecting the viewing effect, and Reduce the transmission bandwidth occupied by light field data.
  • FIG. 3 is a schematic flowchart of another light field data transmission method provided by an embodiment of the present application. Referring to Figure 3, the method may include the following steps:
  • Step 201 The light field display device sends the first gaze point position to the light field communication device.
  • the light field display device can track the position of the eye (such as the pupil) of the viewing object to determine the position of the gaze point of the eye on the light field display screen, that is, the first gaze point. Location.
  • the first gaze point position may be the projection position of the viewing object's pupil on the light field display screen.
  • the light field display device can send the first gaze point position to the light field communication device.
  • the light field display device may obtain and send the first gaze point position after establishing a communication connection with the light field communication device.
  • the light field display device can acquire and send the first gaze point position in real time, or the light field display device can acquire and send the first gaze point position according to a preset tracking cycle.
  • the light field display device 20 may include an eye tracking camera E1.
  • the eye tracking camera E1 can obtain the first gaze point position and send the first gaze point position to the light field communication device 10 .
  • the coordinates of the pupil of the eye in the image can be determined.
  • the coordinates of the pupil in the image can be converted according to the conversion relationship between the image coordinate system and the display screen coordinate system of the light field display screen 26, thereby obtaining the coordinates of the pupil in the display screen coordinate system, that is, the first Fixation point location.
  • the process of determining the position of the first gaze point by the eye tracking camera E1 can be called gaze point positioning.
  • Step 202 The light field communication device determines at least one target camera corresponding to the first gaze point position from a plurality of first cameras based on the correspondence between the gaze point position and the camera.
  • the light field communication device 10 includes a plurality of first cameras 11 .
  • the plurality of first cameras 11 are used to collect light field data.
  • the light field data may be image data, for example, it may be red green blue (RGB) image data, or may include RGB image data and depth map (Depth Map) image data, that is, RGBD image data.
  • RGB red green blue
  • Depth Map depth map
  • the light field communication device 10 may also include a main control circuit 12 .
  • the corresponding relationship between the gaze point position and the camera is pre-stored in the main control circuit 12 .
  • a plurality of different gaze point positions are recorded in this correspondence relationship, and the number of at least one first camera 11 corresponding to each gaze point position is smaller than the number of multiple first cameras 11.
  • the main control circuit 12 can receive the first gaze point position sent by the light field display device 20 and determine at least one target camera corresponding to the first gaze point position based on the corresponding relationship between the gaze point position and the camera.
  • the light field communication device 10 can determine the field of view range of the viewing object based on the first gaze point position, and then determine at least one target camera based on the field of view range and the light field data collection range of each first camera 11 .
  • the display area of the first light field data collected by the at least one target camera on the light field display screen can cover the field of view range.
  • the plurality of first cameras 11 are arranged at different locations, and accordingly, the light field data collection ranges of the plurality of first cameras 11 are different from each other.
  • the display area of the light field data collected by at least one first camera 11 corresponding to each gaze point position on the light field display screen can cover the field of view range at the gaze point position.
  • the process in which the main control circuit 12 determines at least one target camera through the first gaze point position can be called a data arbitration process.
  • Figure 6 is a schematic diagram of multiple first cameras 11 in the light field communication device collecting light field data of the cube O1.
  • the light field communication device includes a total of six first cameras C1_1 to C1_6.
  • the light field data collection range of the first camera C1_1 covers the B side of the cube O1;
  • the light field data collection range of the first camera C1_2 covers part of the A side of the cube O1 and part of the B side, and the A side is the cube.
  • the surface between surface B and surface C of O1; the light field data collection range of the first camera C1_3 covers surface A and part of surface B of the cube O1; the light field data collection range of the first camera C1_4 covers A of the cube O1 surface; the light field data collection range of the first camera C1_5 covers part of the C surface and part of the A surface of the cube O1; the light field data collection range of the first camera C1_6 covers the C surface of the cube O1.
  • Figure 7 is a schematic diagram of the viewing range corresponding to different gaze point positions in the light field display screen of the light field display device.
  • the viewing object's gaze point position is S1
  • the viewing object can view partial areas of surface A and surface B of cube O1 at the gaze point position S1 . Since the light field data collection range of the first cameras C1_3 and C1_4 covers part of surface A and surface B of the cube O1, the gaze point position S1 can correspond to the first cameras C1_3 and C1_4.
  • the viewing object can view the A surface and the B surface of the cube O1 at the gaze point position S2. Since the light field data collection range of the first cameras C1_1 to C1_3 covers surface A and surface B of the cube O1, the gaze point position S2 can correspond to the first cameras C1_1, C1_2 and C1_3. If the gaze point position of the viewing object is S3, the viewing object can view the A surface and the C surface of the cube O1 at the gaze point position S3. Since the light field data collection range of the first cameras C1_4, C1_5 and C1_6 covers the A and C surfaces of the cube O1, the gaze point position S3 corresponds to the first cameras C1_4, C1_5 and C1_6.
  • Step 203 The light field communication device processes the first light field data collected by at least one target camera through at least one target data processing chip among the plurality of data processing chips.
  • the light field communication device 10 may further include a data processing circuit 13 , which includes a plurality of data processing chips 131 .
  • the plurality of data processing chips 131 can establish connections with a plurality of first cameras 11 , and each data processing chip 131 is connected with at least one first camera 11 .
  • the main control circuit 12 of the light field communication device 10 can control the at least one target camera to collect light field data. As shown in Figure 5, this process can be called the acquisition of light field data. Afterwards, the main control circuit 12 can control at least one target data processing chip in the data processing circuit 13 to obtain the first light field data collected by the at least one target camera and process the first light field data.
  • the target data processing chip refers to a data processing chip connected to the target camera. Furthermore, the processing performed by each target data processing chip on the first light field data may include: distortion correction processing, projection transformation processing, color correction processing, encoding processing, etc.
  • each gaze point position can correspond to a plurality of first cameras 11, and the plurality of first cameras 11 are connected to a plurality of different data processing chips 131 in a one-to-one correspondence.
  • the first gaze point position corresponds to a plurality of target cameras, and the plurality of target cameras are connected to a plurality of different target data processing chips in a one-to-one correspondence.
  • each data processing chip 131 can be connected
  • the maximum number of first cameras 11 is n.
  • M and N are both integers greater than 1, n is an integer greater than 0, and Roundup means rounding up.
  • the number of first cameras 11 corresponding to each gaze point position is less than or equal to N.
  • each data processing chip 131 can connect to at most 2 first cameras 11.
  • the connection topology of the ten first cameras C1_1 to C1_10 and the five data processing chips U1_1 to U1_5 in the light field communication device 10 can be as shown in (a) of Figure 8 .
  • each data processing chip is connected to two first cameras.
  • the number of first cameras corresponding to each gaze point position is less than or equal to 5.
  • each data processing chip 131 can connect to at most 3 first cameras 11, and the 10 first cameras in the light field communication device 10
  • the connection topology of C1_1 to C1_10 and the four data processing chips U1_1 to U1_4 can be shown in (b) of Figure 7 .
  • data processing chips U1_1 and U1_2 can be connected to three first cameras respectively, and data processing chips U1_3 and U1_4 can be connected to two first cameras respectively.
  • the number of first cameras 11 corresponding to each gaze point position is less than or equal to 4.
  • the main control circuit 12 in the light field communication device 10 can control at least one target data processing chip to process the first light field data collected by at least one target camera, and do not remove the The light field data collected by at least one first camera 11 other than the target camera is processed.
  • the main control circuit 12 can control each first camera 11 to collect light field data. Moreover, after determining at least one target camera, the main control circuit 12 can send the identification of the at least one target camera to at least one target data processing chip connected to the at least one target camera. Each target data processing chip can then only The first light field data collected by a target camera connected to it is processed, but the light field data collected by other first cameras is not processed. Since the data processing chip 131 in the data processing circuit 13 does not need to process the light field data collected by multiple first cameras 11, it can be ensured that the data processing process takes up less processing resources, thereby ensuring higher data processing efficiency.
  • the main control circuit 12 in the light field communication device 10 can control at least one target camera to collect light field data, and control the plurality of first cameras 11 except at least one target camera.
  • the first camera stops transmitting light field data.
  • the main control circuit 12 may control the first camera 11 except at least one target camera to stop collecting light field data, or may control the first camera 11 except at least one target camera to continue collecting light field data. Collect light field data and stop transmitting the light field data to the data processing circuit 13.
  • the main control circuit 12 can not only control the first cameras 11 of the plurality of first cameras 11 except at least one target camera to stop transmitting light field data, but can also control at least one target data processing chip not to transmit light field data except at least one target camera.
  • the light field data collected by the first camera 11 other than the camera is processed.
  • Step 204 The light field communication device transmits the processed first light field data to the light field display device.
  • the light field communication device 10 may further include a transmission circuit 14 capable of transmitting the first light field data output by the data processing circuit 13 to the light field display device 20 .
  • the transmission circuit 14 can transmit the first light field data from different target cameras output by the data processing circuit 13 to the light field display device 20 in parallel.
  • the main control circuit 12 in the light field communication device 10 is also used to control the multiple target cameras to synchronously collect the first light field data. , and controls the data processing circuit 13 to synchronously output the first light field data.
  • Step 205 The light field display device performs light field display on the processed first light field data.
  • the light field display device 20 also includes a display processing circuit 25 and a light field display screen 26 .
  • the display processing circuit 25 can process the first light field data and perform 26 light field display through the light field display screen.
  • the processing performed by the display processing circuit 25 on the first light field data may include decoding processing and 3D rendering processing.
  • the display processing circuit 25 may be a graphics processing unit (GPU). Moreover, the display processing circuit 25 may be integrated in the light field display screen 26 , or may be provided independently of the light field display screen 26 .
  • GPU graphics processing unit
  • the number of the first cameras 11 in the light field communication device 10 is positively related to the effect of the light field display device 20 displaying light field data (ie, the imaging quality of the 3D image). Due to the large amount of light field data, the real-time collection and processing of light field data places greater demands on the processing resources and transmission bandwidth of light field communication equipment. If the light field communication device 10 processes and transmits all the light field data collected by the multiple first cameras 11 , the transmission bandwidth and the display effect of the light field data cannot be taken into consideration at the same time.
  • the main control circuit 12 in the light field communication device 10 can control the data processing circuit 13 to only respond to the first gaze point through data arbitration.
  • the first light field data collected by at least one target camera corresponding to the point position is processed and transmitted. Therefore, it is possible to avoid redundant light field data from occupying excessive processing resources and transmission bandwidth while ensuring the viewing effect of light field data, so as to achieve real-time collection, processing and transmission of light field data.
  • Step 206 The light field communication device obtains the second gaze point position through the eye tracking camera.
  • the light field communication device 10 may include: an eye tracking camera E2 and a light field display screen 16 .
  • the eye tracking camera E2 can track the position of the eye (eg, pupil) of the viewing object of the light field communication device 10 to determine the position of the gaze point of the eye on the light field display screen 16, that is, the second gaze. point location.
  • the second gaze point position may be the projection position of the viewing object's pupil on the light field display screen 16 .
  • the light field communication device 10 can not only serve as a light field data acquisition device to provide light field data to the light field display device 20 , but can also serve as a display device to perform light field display on the light field data.
  • the light field communication device 10 can obtain the second gaze point position through the eye tracking camera E2.
  • the light field communication device 10 can acquire and send the second gaze point position in real time, or the light field communication device 10 can acquire and send the second gaze point position according to a preset tracking cycle.
  • Step 207 The light field communication device transmits the second gaze point position to the light field display device.
  • the light field display device 20 may include a main control circuit 22 . After the eye tracking camera E2 in the light field communication device 10 obtains the second gaze point position, the second gaze point position can be transmitted to the main control circuit 22 of the light field display device 20 .
  • Step 208 The light field display device determines at least one target camera corresponding to the second gaze point position from the plurality of second cameras based on the correspondence between the gaze point position and the camera.
  • the light field display device 20 may include multiple second cameras 21 .
  • the main control circuit 22 of the light field display device 20 may also store the corresponding relationship between the gaze point position and the camera in advance. A plurality of different gaze point positions are recorded in this correspondence relationship, each gaze point position corresponds to at least one second camera 21 , and the number of at least one second camera 21 corresponding to each gaze point position is smaller than the number of second cameras 21 quantity.
  • the main control circuit 22 can receive the second gaze point position sent by the light field communication device 10 and determine at least one target camera corresponding to the second gaze point position based on the corresponding relationship between the gaze point position and the camera.
  • the light field display device 20 can determine the field of view range of the viewing object based on the second gaze point position, and then determine at least one target camera based on the field of view range and the light field data collection range of each second camera 21 .
  • the display area of the second light field data collected by the at least one target camera on the light field display screen 16 of the light field communication device 10 can cover the field of view range.
  • Step 209 The light field display device processes the second light field data collected by at least one target camera through at least one target data processing chip among the plurality of data processing chips.
  • the light field display device 20 may further include a data processing circuit 23 , and the data processing circuit 23 may include a plurality of data processing chips 231 .
  • the plurality of data processing chips 231 can establish connections with a plurality of second cameras 21 , and each data processing chip 231 is connected with at least one second camera 21 .
  • connection topology of the plurality of second cameras 21 and the plurality of data processing chips 231 in the light field display device 20 please refer to the above description for the plurality of first cameras 11 and the plurality of data processing chips in the light field communication device 10. The description of the connection topology of 131 will not be repeated here.
  • the main control circuit 22 determines at least one target camera from the plurality of second cameras 21, it can control at least one target data processing chip in the data processing circuit 23 to obtain the third image collected by the at least one target camera. two light field data, and process the second light field data.
  • Step 210 The light field display device sends the processed second light field data to the light field communication device.
  • step 204 For the implementation process of this step, reference can be made to the relevant description of step 204 above, which will not be described again here.
  • Step 211 The light field communication device performs light field display based on the second light field data.
  • step 205 For the implementation process of this step, reference can be made to the relevant description of step 205 above, which will not be described again here.
  • the light field communication device 10 can serve as a light field data collection device to provide light first field data to the light field display device 20 , and the light field display device 20 serves as a display device. Based on the light field communication device 10 The first light field data is provided for light field display. Moreover, the light field display device 20 can also serve as a light field data collection device to provide the second light field data to the light field communication device 10 , and the light field communication device 10 can further serve as a display device to provide the second light field data based on the light field display device 20 The second light field data is used for light field display. Thus, bidirectional transmission and display of light field data between the light field communication device 10 and the light field display device 20 can be achieved.
  • the light field communication device 10 and the light field display device 20 can implement 3D video communication. After the light field communication device 10 and the light field display device 20 establish a communication connection, each of the two devices can obtain at least one target corresponding to the gaze point position in real time based on the gaze point position fed back by the other device. The light field data collected by the camera is processed. Afterwards, the two devices can transmit the processed light field data and perform light field display on the received light field data. As a result, real-time, two-way 3D video communication can be achieved.
  • the order of the steps of the light field data transmission method provided by the embodiments of the present application can be appropriately adjusted, and the steps can also be increased or decreased accordingly according to the situation.
  • the above steps 206 to 211 can be deleted according to the situation.
  • the above steps 206 to 211 may be performed before step 201.
  • step 206 may be executed simultaneously with step 201. Any person familiar with the technical field can easily think of changing methods within the technical scope disclosed in this application, which should be covered by the protection scope of this application, and therefore will not be described again.
  • embodiments of the present application provide a method for transmitting light field data.
  • the light field communication device can, based on the first gaze point position fed back by the light field display device, at least one target corresponding to the first gaze point position.
  • the first light field data collected by the camera is processed and then transmitted to the light field display device.
  • the light field display device can then perform light field display on the first light field data on its light field display screen. Since the light field communication device only needs to process and transmit the light field data collected by the light field camera corresponding to the first gaze point position, it can effectively improve the efficiency of light field data processing without affecting the viewing effect, and Reduce the transmission bandwidth occupied by light field data.
  • the light field communication device 10 includes: a main control circuit 12 , a data processing circuit 13 and a plurality of first cameras 11 .
  • the main control circuit 12 is configured to receive a first gaze point position sent by the light field display device 20 .
  • the first gaze point position is the eye of the viewing object of the light field display device 20 in the light field display of the light field display device 20 .
  • the data processing circuit 13 is used to process the first light field data collected by at least one target camera and then transmit it to the light field display device 20 .
  • the processed first light field data is used for the light field display device 20 to perform processing. Light field display.
  • the main control circuit 12 may be a central processing unit (CPU) or a micro-controller unit (MCU).
  • CPU central processing unit
  • MCU micro-controller unit
  • the plurality of first cameras 11 may be disposed around the light field display screen.
  • the corresponding relationship between the gaze point position and the camera is stored in the main control circuit 12 .
  • a plurality of different gaze point positions are recorded in this correspondence relationship, each gaze point position corresponds to at least one first camera 11 , and the number of the corresponding at least one first camera 11 is less than the number of multiple first cameras 11 .
  • the main control circuit 12 is used to determine at least one target camera corresponding to the first gaze point position according to the corresponding relationship.
  • the data processing circuit 13 includes multiple data processing chips 131 , and each data processing chip 131 is connected to at least one first camera 11 .
  • At least one target data processing chip among the plurality of data processing chips 131 is connected to at least one target camera, and the at least one target data processing chip is used to process the first light field data collected by at least one target camera.
  • the data processing chip 131 may be a GPU.
  • the data processing chip 131 can be a programmable logic device (PLD), for example, a complex programmable logic device (CPLD), a field programmable logic gate array (field programmable gate array, FPGA) ) or general array logic (GAL), etc., the embodiments of this application do not limit this.
  • PLD programmable logic device
  • CPLD complex programmable logic device
  • FPGA field programmable logic gate array
  • GAL general array logic
  • the first gaze point position corresponds to multiple target cameras, and the multiple target cameras are connected to multiple different target data processing chips in a one-to-one correspondence.
  • each data processing chip 131 is connected to multiple first cameras 11; the main control circuit 12 is used to control at least one target data processing chip to process the first light field data collected by at least one target camera, And the first camera 11 except the at least one target camera is not processed.
  • the main control circuit 12 is also used to: control at least one target camera to collect light field data, and control the first cameras 11 among the plurality of first cameras 11 except at least one target camera to stop transmitting the light field data.
  • the light field communication device 10 may also include an eye tracking camera E2, a display processing circuit 15 and a light field display screen 16.
  • the eye tracking camera E2 is used to obtain a second gaze point position and transmit the second gaze point position to the light field display device 20 .
  • the second gaze point position is the viewing object of the light field communication device 10 .
  • the eye tracking camera E2 may be disposed around the light field display screen 16 .
  • the display processing circuit 15 is used to receive and process the second light field data transmitted by the light field display device 20 .
  • the second light field data is collected by at least one second camera 21 in the light field display device 20, and the at least one second camera 21 is determined based on the second gaze point position.
  • the light field display screen 26 is used for light field display based on the second light field data.
  • embodiments of the present application provide a light field communication device, which can detect at least one target camera corresponding to the first gaze point position based on the first gaze point position fed back by the light field display device.
  • the collected first light field data is processed and then transmitted to the light field display device.
  • the light field display device can then perform light field display on the first light field data on its light field display screen. Since the light field communication device only needs to process and transmit the light field data collected by the light field camera corresponding to the first gaze point position, it can effectively improve the efficiency of light field data processing without affecting the viewing effect, and Reduce the transmission bandwidth occupied by light field data.
  • the light field communication equipment provided in the above embodiments is only exemplified by the division of the above functional modules.
  • the above functions can be allocated to different functional modules according to needs, that is, the internal functions of the device
  • the structure is divided into different functional modules to complete all or part of the functions described above.
  • the light field communication device and light field data transmission method embodiments provided in the above embodiments belong to the same concept. Please refer to the method embodiments for details of their specific implementation processes, which will not be described again here.
  • An embodiment of the present application also provides a light field communication device, which may be a computer device.
  • the light field communication device includes: a plurality of first cameras 11 , a processor 101 and a memory 102 .
  • the memory 102 stores instructions, which are loaded and executed by the processor 101 to implement the light field data transmission method provided by the above method embodiment.
  • Embodiments of the present application also provide a computer-readable storage medium in which instructions are stored, and the instructions are loaded and executed by the processor to implement the light field data transmission method provided by the above method embodiments.
  • Embodiments of the present application also provide a computer program product or computer program, which includes computer instructions that are loaded and executed by a processor to implement the light field data transmission method as described above.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Optical Communication System (AREA)
  • Studio Devices (AREA)

Abstract

本申请公开了一种光场数据的传输方法、光场通信设备及系统，光场通信设备能够基于光场显示设备反馈的第一注视点位置，对该第一注视点位置对应的至少一个目标相机采集到的第一光场数据进行处理后传输至光场显示设备。该光场显示设备进而可以在其光场显示屏上对该第一光场数据进行光场显示。由于光场通信设备仅需处理并传输与第一注视点位置对应的部分光场相机采集到的光场数据，因此可以在不影响观看效果的前提下，有效提高光场数据处理的效率，并降低光场数据占用的传输带宽。

Description

光场数据的传输方法、光场通信设备及系统 技术领域
本申请涉及通信技术领域，特别涉及一种光场数据的传输方法、光场通信设备及系统。
背景技术
随着三维(3 Dimensions，3D)重建技术和光场显示技术的发展，将二者进行结合以实现3D沉浸式的视频交互已成为当前视频通信技术的主要发展方向。
相关技术中，可以通过两个光场显示设备实现3D视频通信。其中，每个光场显示设备均包括多个相机，且可以将该多个相机采集到的光场数据进行处理后发送至另一光场显示设备。该另一光场显示设备进而可以将该光场数据以3D的形式显示。
发明内容
本申请提供了一种光场数据的传输方法、光场通信设备及系统。所述技术方案如下：
一方面，提供了一种图像处理模型的训练方法，应用于光场通信设备，所述光场通信设备包括多个第一相机；所述方法包括：
接收光场显示设备发送的第一注视点位置，所述第一注视点位置为所述光场显示设备的观看对象的眼部在所述光场显示设备的光场显示屏上的注视点的位置；
从所述多个第一相机中确定与所述第一注视点位置对应的至少一个目标相机，所述至少一个目标相机的数量小于所述多个第一相机的数量；
对所述至少一个目标相机采集到的第一光场数据进行处理后传输至所述光场显示设备，处理后的所述第一光场数据用于供所述光场显示设备进行光场显示。
可选地，所述光场通信设备中存储有注视点位置和相机的对应关系，所述 对应关系中记录有多个不同的注视点位置，每个注视点位置对应至少一个第一相机，且每个注视点位置对应的至少一个第一相机的数量小于所述多个第一相机的数量；
所述从所述多个第一相机中确定与所述第一注视点位置对应的至少一个目标相机，包括：
根据所述对应关系，确定与所述第一注视点位置对应的至少一个目标相机。
可选地，所述光场通信设备还包括：多个数据处理芯片，每个数据处理芯片与至少一个第一相机连接；
所述对所述至少一个目标相机采集到的第一光场数据进行处理，包括：
通过所述多个数据处理芯片中的至少一个目标数据处理芯片，对所述至少一个目标相机采集到的第一光场数据进行处理，所述至少一个目标数据处理芯片与所述至少一个目标相机连接。
可选地，所述第一注视点位置对应多个目标相机；
所述多个目标相机与多个不同的目标数据处理芯片一一对应连接。
可选地，每个数据处理芯片与多个第一相机连接；
所述通过所述多个数据处理芯片中的至少一个目标数据处理芯片，对所述至少一个目标相机采集到的第一光场数据进行处理，包括：
控制所述至少一个目标数据处理芯片对所述至少一个目标相机采集到的第一光场数据进行处理，且不对除所述至少一个目标相机之外的第一相机进行处理。
可选地，在对所述至少一个目标相机采集到的第一光场数据进行处理之前，所述方法还包括：
控制所述至少一个目标相机采集光场数据，并控制所述多个第一相机中除所述至少一个目标相机之外的第一相机停止传输光场数据。
可选地，所述光场通信设备还包括眼部追踪相机和光场显示屏；所述方法还包括：
通过所述眼部追踪相机获取第二注视点位置，所述第二注视点位置为所述光场通信设备的观看对象的眼部在所述光场通信设备的光场显示屏上的注视点的位置；
将所述第二注视点位置传输至所述光场显示设备；
接收所述光场显示设备传输的第二光场数据，所述第二光场数据是所述光场显示设备中的至少一个第二相机采集到的，且所述至少一个第二相机基于所述第二注视点位置确定；
基于所述第二光场数据进行光场显示。
另一方面，提供了一种光场通信设备，所述光场通信设备包括：主控电路，数据处理电路以及多个第一相机；所述主控电路，用于：
接收光场显示设备发送的第一注视点位置，所述第一注视点位置为所述光场显示设备的观看对象的眼部在所述光场显示设备的光场显示屏上的注视点的位置；
以及，从所述多个第一相机中确定与所述第一注视点位置对应的至少一个目标相机，所述至少一个目标相机的数量小于所述多个第一相机的数量；
所述数据处理电路，用于对所述至少一个目标相机采集到的第一光场数据进行处理后传输至所述光场显示设备，处理后的所述第一光场数据用于供所述光场显示设备进行光场显示。
可选地，所述主控电路中存储有注视点位置和相机的对应关系，所述对应关系中记录有多个不同的注视点位置，每个注视点位置对应至少一个第一相机，且对应的至少一个第一相机的数量小于所述多个第一相机的数量；
所述主控电路，用于根据所述对应关系，确定与所述第一注视点位置对应的至少一个目标相机。
可选地，所述数据处理电路包括多个数据处理芯片，每个数据处理芯片与至少一个第一相机连接；
所述多个数据处理芯片中的至少一个目标数据处理芯片与所述至少一个目标相机连接，所述至少一个目标数据处理芯片用于对所述至少一个目标相机采集到的第一光场数据进行处理。
可选地，所述第一注视点位置对应多个目标相机；
所述多个目标相机与多个不同的目标数据处理芯片一一对应连接。
可选地，每个数据处理芯片与多个第一相机连接；所述主控电路，用于：
控制所述至少一个目标数据处理芯片对所述至少一个目标相机采集到的第一光场数据进行处理，且不对除所述至少一个目标相机之外的第一相机进行处理。
可选地，所述主控电路还用于：
控制所述至少一个目标相机采集光场数据，并控制所述多个第一相机中除所述至少一个目标相机之外的第一相机停止传输光场数据。
可选地，所述光场通信设备还包括眼部追踪相机、显示处理电路和光场显示屏；
所述眼部追踪相机，用于获取第二注视点位置，并将所述第二注视点位置传输至所述光场显示设备，所述第二注视点位置为所述光场通信设备的观看对象的眼部在所述光场通信设备的光场显示屏上的注视点的位置；
所述显示处理电路，用于接收并处理所述光场显示设备传输的第二光场数据，所述第二光场数据是所述光场显示设备中的至少一个第二相机采集到的，且所述至少一个第二相机基于所述第二注视点位置确定；
所述光场显示屏，用于基于所述第二光场数据进行光场显示。
又一方面，提供了一种光场数据的传输系统，所述系统包括：光场通信设备和光场显示设备；
其中，所述光场通信设备为上述方面所述的光场通信设备；
所述光场显示设备用于向所述光场通信设备发送第一注视点位置，以及基于所述光场通信设备传输的第一光场数据进行光场显示。
再一方面，提供了一种光场通信设备，所述光场通信设备包括：多个第一相机，处理器以及存储器，所述存储器中存储有指令，所述指令由所述处理器加载并执行以实现如上述方面所述的光场数据的传输方法。
再一方面，提供了一种计算机可读存储介质，所述存储介质中存储有指令，所述指令由处理器加载并执行以实现如上述方面所述的光场数据的传输方法。
附图说明
为了更清楚地说明本申请实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是本申请实施例提供的一种光场数据的传输系统的结构示意图；
图2是本申请实施例提供的一种光场数据的传输方法的流程示意图；
图3是本申请实施例提供的另一种光场数据的传输方法的流程示意图；
图4是本申请实施例提供的另一种光场数据的传输系统的结构示意图；
图5是本申请实施例提供的一种光场数据的传输示意图；
图6是本申请实施例提供的一种第一相机采集光场数据的示意图；
图7是本申请实施例提供的一种光场显示设备显示光场数据的示意图；
图8是本申请实施例提供的一种光场通信设备中相机和数据处理芯片的连接拓扑图；
图9是本申请实施例提供的一种光场通信设备的结构框图。
具体实施方式
为使本申请的目的、技术方案和优点更加清楚，下面将结合附图对本申请实施方式作进一步地详细描述。
图1是本申请实施例提供的一种光场数据的传输系统的结构示意图，参考图1，该系统包括：光场通信设备10和光场显示设备20。该光场通信设备10和光场显示设备20之间建立有线或无线通信连接。其中，该光场通信设备10可以为具有多个相机(例如光场相机)，且能够对该多个相机采集的光场数据进行处理并传输的通信设备。例如，该光场通信设备10可以为手机，平板电脑，笔记本电脑，台式电脑，车载终端或可穿戴设备等。该光场显示设备20可以为具有光场显示屏的显示设备。其中，该光场显示设备20中的光场显示屏可以为平面显示屏。
在本申请实施例中，该光场显示设备20能够确定其观看对象的眼部在光场显示屏上的第一注视点位置，并向光场通信设备10发送该第一注视点位置。该光场通信设备10能够基于光场显示设备20发送的第一注视点位置，从多个第一相机中确定出该第一注视点对应的至少一个目标相机，并对该至少一个目标相机采集到的第一光场数据进行处理后传输至光场显示设备20。该光场显示设备20进而可以基于该第一光场数据进行光场显示。
上述示例是以光场通信设备10作为光场数据的采集设备向光场显示设备20提供光场数据，并以光场显示设备20作为显示设备，基于光场通信设备10提供的光场数据进行光场显示为例进行的说明。
可选地，该光场通信设备10还可以包括光场显示屏，并且该光场通信设备 10可以向光场显示设备20发送第二注视点位置，该第二注视点位置为光场通信设备10的观看对象的眼部在光场通信设备10的光场显示屏上的注视点的位置。光场显示设备20具有多个第二相机，该光场显示设备20可以基于该第二注视点位置，从多个第二相机中确定出该第二注视点对应的至少一个目标相机，并对该至少一个目标相机采集到的第二光场数据进行处理后传输至光场通信设备10。光场通信设备10进而可以对该第二光场数据进行光场显示。由此，可以实现光场通信设备10和光场显示设备20之间双向的光场数据的传输和显示。示例的，该光场通信设备10和光场显示设备20可以实现3D视频通信。
图2是本申请实例提供的一种光场数据的传输方法的流程示意图，该方法可以应用于光场数据的传输系统，例如图1所示的光场数据的传输系统。参见图2，该方法包括：
步骤101、光场显示设备向光场通信设备发送第一注视点位置。
在本申请实施例中，光场显示设备能够对其观看对象的眼部(例如瞳孔)的位置进行追踪，以确定该眼部在光场显示屏上的注视点的位置，即第一注视点位置。其中，该第一注视点位置可以是观看对象的瞳孔在光场显示屏上的投影位置。之后，该光场显示设备能够将该第一注视点位置发送至光场通信设备。
可选地，光场显示设备可以在与光场通信设备建立通信连接后，获取并发送该第一注视点位置。例如，光场显示设备可以实时获取并发送该第一注视点位置，或者，光场显示设备可以按照预设的追踪周期获取并发送该第一注视点位置。
步骤102、光场通信设备从多个第一相机中确定与第一注视点位置对应的至少一个目标相机。
在本申请实施例中，该光场通信设备包括多个第一相机。该多个第一相机用于采集光场数据。该多个第一相机可以为光场相机，该光场相机采集到的光场数据可以为图像数据。光场通信设备获取到光场显示设备反馈的第一注视点位置后，能够基于该第一注视点位置，从多个第一相机中确定出与该第一注视点位置对应的至少一个目标相机。
可选地，该光场通信设备中可以预先存储有注视点位置和相机的对应关系。光场通信设备能够基于该对应关系，直接确定出该第一注视点位置所对应的至 少一个目标相机。或者，该光场通信设备可以基于该第一注视点位置确定观看对象的视场范围，进而根据该视场范围和每个第一相机的光场数据采集范围，确定出至少一个目标相机。该至少一个目标相机采集到的第一光场数据在光场显示屏上的显示区域能够覆盖该视场范围。
其中，该至少一个目标相机的数量小于多个第一相机的数量。示例的，光场通信设备中所包括的第一相机的数量可以为10，目标相机的数量可以为4。
可以理解的是，光场通信设备可以在与光场显示设备建立通信连接后，控制该多个第一相机采集光场数据。
步骤103、光场通信设备对至少一个目标相机采集到的第一光场数据进行处理后传输至光场显示设备。
光场通信设备从多个第一相机确定出至少一个目标相机后，能够获取该至少一个目标相机采集到的第一光场数据并进行处理。之后，该光场通信设备能够将处理后的第一光场数据传输至光场显示设备。其中，光场通信设备可以对第一光场数据进行的处理可以包括：畸变校正处理，投影变换处理、色彩校正处理以及编码处理等。
步骤104、光场显示设备对处理后的第一光场数据进行光场显示。
光场显示设备能够通过光场显示技术，在其光场显示屏上对接收到的第一光场数据进行光场显示，以实现3D图像的显示。
可以理解的是，受观看对象眼部的瞳孔尺寸的限制，光场显示设备显示的光场数据(即3D图像)无法全部进入观看对象的眼部。也即是，观看对象无法同时观看到光场显示设备显示的所有光场数据。因此，光场通信设备可以只对与观看对象的注视点位置对应的至少一个目标相机采集到的第一光场数据进行处理，并传输至光场显示设备进行光场显示。由此，可以在确保观看效果的前提下，避免冗余的光场数据占用过多的传输带宽。
综上所述，本申请实施例提供了一种光场数据的传输方法，光场通信设备能够基于光场显示设备反馈的第一注视点位置，对该第一注视点位置对应的至少一个目标相机采集到的第一光场数据进行处理后传输至光场显示设备。该光场显示设备进而可以在其光场显示屏上对该第一光场数据进行光场显示。由于光场通信设备仅需处理并传输与第一注视点位置对应的部分光场相机采集到的光场数据，因此可以在不影响观看效果的前提下，有效提高光场数据处理的效 率，并降低光场数据占用的传输带宽。
图3是本申请实施例提供的另一种光场数据的传输方法的流程示意图，参考图3，该方法可以包括如下步骤：
步骤201、光场显示设备向光场通信设备发送第一注视点位置。
在本申请实施例中，光场显示设备能够对其观看对象的眼部(例如瞳孔)的位置进行追踪，以确定该眼部在光场显示屏上的注视点的位置，即第一注视点位置。其中，该第一注视点位置可以是观看对象的瞳孔在光场显示屏上的投影位置。之后，该光场显示设备能够将该第一注视点位置发送至光场通信设备。
可选地，光场显示设备可以在与光场通信设备建立通信连接后，获取并发送该第一注视点位置。例如，光场显示设备可以实时获取并发送该第一注视点位置，或者，光场显示设备可以按照预设的追踪周期获取并发送该第一注视点位置。
示例的，参考图4，该光场显示设备20可以包括眼部追踪相机E1。该眼部追踪相机E1能够获取第一注视点位置，并将该第一注视点位置发送至光场通信设备10。例如，该眼部追踪相机E1拍摄观看对象的眼部的图像后，可以确定该眼部的瞳孔在图像中的坐标。之后，可以根据图像坐标系与光场显示屏26的显示屏坐标系之间的转换关系，对瞳孔在图像中的坐标进行转换，从而得到瞳孔在显示屏坐标系中的坐标，即该第一注视点位置。
如图5所示，该眼部追踪相机E1确定第一注视点位置的过程可以称为注视点定位。
步骤202、光场通信设备根据注视点位置和相机的对应关系，从多个第一相机中确定与第一注视点位置对应的至少一个目标相机。
在本申请实施例中，如图4所示，该光场通信设备10包括多个第一相机11。该多个第一相机11用于采集光场数据。该光场数据可以为图像数据，例如可以为红绿蓝(red green blue，RGB)图像数据，或者可以包括RGB图像数据和深度图(Depth Map)图像数据，即RGBD图像数据。
继续参考图4，该光场通信设备10还可以包括主控电路12。可选地，该主控电路12中预先存储有注视点位置和相机的对应关系。该对应关系中记录有多个不同的注视点位置，每个注视点位置对应的至少一个第一相机11的数量均小 于多个第一相机11的数量。该主控电路12能够接收光场显示设备20发送的第一注视点位置，并基于上述注视点位置和相机的对应关系确定出该第一注视点位置对应的至少一个目标相机。
或者，该光场通信设备10可以基于该第一注视点位置确定观看对象的视场范围，进而根据该视场范围和每个第一相机11的光场数据采集范围，确定出至少一个目标相机。该至少一个目标相机采集到的第一光场数据在光场显示屏上的显示区域能够覆盖该视场范围。
可以理解的是，该多个第一相机11的设置位置不同，相应的，该多个第一相机11的光场数据采集范围互不相同。每个注视点位置对应的至少一个第一相机11所采集到的光场数据在光场显示屏上的显示区域能够覆盖该注视点位置处的视场范围。
如图5所示，该主控电路12通过第一注视点位置确定至少一个目标相机的过程可以称为数据仲裁过程。
图6为光场通信设备中的多个第一相机11采集立方体O1的光场数据的示意图。如图6所示，假设光场通信设备包括C1_1至C1_6共6个第一相机。其中，第一相机C1_1的光场数据采集范围覆盖立方体O1的B面；第一相机C1_2的光场数据采集范围覆盖立方体O1的A面的部分区域以及B面的部分区域，A面为该立方体O1的B面和C面之间的面；第一相机C1_3的光场数据采集范围覆盖立方体O1的A面以及B面的部分区域；第一相机C1_4的光场数据采集范围覆盖立方体O1的A面；第一相机C1_5的光场数据采集范围覆盖立方体O1的C面的部分区域和A面的部分区域；第一相机C1_6的光场数据采集范围覆盖立方体O1的C面。
图7为光场显示设备的光场显示屏中不同注视点位置所对应的观看范围的示意图。参考图7，若观看对象的注视点位置为S1，则观看对象在该注视点位置S1处能够观看到立方体O1的A面和B面的部分区域。由于第一相机C1_3和C1_4的光场数据采集范围覆盖立方体O1的A面和B面的部分区域，因此该注视点位置S1可以与第一相机C1_3和C1_4对应。
若观看对象的注视点位置为S2，则观看对象在该注视点位置S2处能够观看到立方体O1的A面和B面。由于第一相机C1_1至C1_3的光场数据采集范围覆盖立方体O1的A面和B面，因此该注视点位置S2可以与第一相机C1_1、 C1_2以及C1_3对应。若观看对象的注视点位置为S3，则观看对象在该注视点位置S3处能够观看到立方体O1的A面和C面。由于第一相机C1_4、C1_5和C1_6的光场数据采集范围覆盖立方体O1的A面和C面，因此该注视点位置S3与第一相机C1_4、C1_5和C1_6对应。
步骤203、光场通信设备通过多个数据处理芯片中的至少一个目标数据处理芯片，对至少一个目标相机采集到的第一光场数据进行处理。
参考图4，该光场通信设备10还可以包括数据处理电路13，该数据处理电路13包括多个数据处理芯片131。该多个数据处理芯片131可以与多个第一相机11建立连接，且每个数据处理芯片131与至少一个第一相机11连接。
光场通信设备10的主控电路12从多个第一相机11确定出至少一个目标相机后，能够控制该至少一个目标相机采集光场数据。如图5所示，该过程可以称为光场数据的获取。之后，该主控电路12能够控制数据处理电路13中的至少一个目标数据处理芯片获取该至少一个目标相机采集到的第一光场数据，并对该第一光场数据进行处理。其中，该目标数据处理芯片是指与目标相机连接的数据处理芯片。并且，每个目标数据处理芯片对第一光场数据进行的处理可以包括：畸变校正处理，投影变换处理、色彩校正处理以及编码处理等。
可选地，在注视点位置与相机的对应关系中，每个注视点位置均可以对应多个第一相机11，该多个第一相机11与多个不同的数据处理芯片131一一对应连接。相应的，该第一注视点位置对应多个目标相机，该多个目标相机与多个不同的目标数据处理芯片一一对应连接。基于上述对应关系和连接关系，可以确保在传输每个注视点位置处的光场数据时，每个数据处理芯片131只处理并传输其连接的一个目标相机采集到的光场数据。由此，有效降低了数据处理过程占用的数据处理芯片131的处理资源，且降低了光场数据所占用的传输带宽。
在本申请实施例中，若光场通信设备10包括的第一相机11的数量为M，数据处理电路13中包括的数据处理芯片131的数量为N，则每个数据处理芯片131所能够连接的第一相机11的最大数量为n。其中，数量M、N和n满足公式：N＝Roundup(M/n)。M和N均为大于1的整数，n为大于0的整数，Roundup表示向上取整。并且，注视点位置与第一相机11的对应关系中，每个注视点位置所对应的第一相机11的数量小于等于N。
示例的，假设M＝10，N＝5，则基于上述公式，可以确定n＝2，即每个数据 处理芯片131至多能够连接2个第一相机11。该光场通信设备10中的10个第一相机C1_1至C1_10与5个数据处理芯片U1_1至U1_5的连接拓扑可以如图8中的(a)所示。参考图8中的(a)，每个数据处理芯片均与两个第一相机连接。并且，基于该连接关系，在注视点位置与第一相机的对应关系中，每个注视点位置所对应的第一相机的数量小于等于5。
假设M＝10，N＝4，则基于上述公式，可以确定n＝3，即每个数据处理芯片131至多能够连接3个第一相机11，该光场通信设备10中的10个第一相机C1_1至C1_10与4个数据处理芯片U1_1至U1_4的连接拓扑可以如图7中的(b)所示。参考图8中的(b)，数据处理芯片U1_1和U1_2可以分别与3个第一相机连接，数据处理芯片U1_3和U1_4可以分别与2个第一相机连接。基于该连接关系，在注视点位置与第一相机的对应关系中，每个注视点位置所对应的第一相机11的数量小于等于4。
作为步骤203的第一种可能的实现方式，光场通信设备10中的主控电路12可以控制至少一个目标数据处理芯片对至少一个目标相机采集到的第一光场数据进行处理，且不对除至少一个目标相机之外的第一相机11采集到的光场数据进行处理。
在该第一种实现方式中，光场通信设备10在与光场显示设备20建立通信连接后，主控电路12能够控制各个第一相机11均采集光场数据。并且，主控电路12在确定出至少一个目标相机后，可以将该至少一个目标相机的标识发送至与该至少一个目标相机连接的至少一个目标数据处理芯片，每个目标数据处理芯片进而可以只处理其所连接的一个目标相机采集的第一光场数据，而不处理其他第一相机采集的光场数据。由于数据处理电路13中的数据处理芯片131无需处理多个第一相机11采集的光场数据，因此可以确保数据处理过程占用的处理资源较少，进而确保数据处理的效率较高。
作为步骤203的第二种可能的实现方式，光场通信设备10中的主控电路12可以控制至少一个目标相机采集光场数据，并控制多个第一相机11中除至少一个目标相机之外的第一相机停止传输光场数据。
在该第二种实现方式中，主控电路12可以控制除至少一个目标相机之外的第一相机11停止采集光场数据，或者，可以控制除至少一个目标相机之外的第一相机11继续采集光场数据，并停止向数据处理电路13传输光场数据。
还可以理解的是，上述两种实现方式还可以结合。也即是，主控电路12不仅可以控制多个第一相机11中除至少一个目标相机之外的第一相机11停止传输光场数据，还可以控制至少一个目标数据处理芯片不对除至少一个目标相机之外的第一相机11采集的光场数据进行处理。
步骤204、光场通信设备将处理后的第一光场数据传输至光场显示设备。
参考图4，该光场通信设备10还可以包括传输电路14，该传输电路14能够将数据处理电路13输出的第一光场数据传输至光场显示设备20。例如，该传输电路14能够将数据处理电路13输出的来自不同目标相机的第一光场数据并行传输至光场显示设备20。
可选地，对于第一注视点位置对应多个目标相机的场景，参考图4，该光场通信设备10中的主控电路12还用于控制该多个目标相机同步采集第一光场数据，并控制该数据处理电路13同步输出第一光场数据。
步骤205、光场显示设备对处理后的第一光场数据进行光场显示。
如图4所示，光场显示设备20还包括显示处理电路25和光场显示屏26。该显示处理电路25接收到光场通信设备10传输的第一光场数据后，能够对该第一光场数据进行处理，并通过该光场显示屏进26行光场显示。参考图5，显示处理电路25对第一光场数据进行的处理可以包括解码处理和3D渲染处理。
可选地，该显示处理电路25可以是图形处理器(graphics processing unit，GPU)。并且，该显示处理电路25可以集成在光场显示屏26中，或者，可以独立于光场显示屏26设置。
可以理解的是，光场通信设备10中的第一相机11的数量与光场显示设备20显示光场数据的效果(即3D图像的成像质量)正相关。由于光场数据的数据量较大，因此光场数据的实时采集与处理对光场通信设备的处理资源和传输带宽的需求较大。若光场通信设备10对多个第一相机11采集到的光场数据均进行处理和传输，则无法同时兼顾传输带宽和光场数据的显示效果。
并且，受观看对象眼部的瞳孔尺寸的限制，光场显示屏26显示的光场数据(即3D图像)无法全部进入观看对象的眼部。也即是，观看对象无法同时观看到光场显示屏26显示的所有光场数据。因此，在该光场数据的传输方法中，冗余的光场数据占用着过多的传输带宽。
而在本申请实施例中，基于光场显示设备20反馈的第一注视点位置，光场 通信设备10中的主控电路12通过数据仲裁，可以控制数据处理电路13只对与该第一注视点位置对应的至少一个目标相机采集到的第一光场数据进行处理并传输。由此，可以在确保光场数据观看效果的前提下，避免冗余的光场数据占用过多的处理资源和传输带宽，以实现光场数据的实时采集、处理与传输。
步骤206、光场通信设备通过眼部追踪相机获取第二注视点位置。
可选地，继续参考图4，该光场通信设备10可以包括：眼部追踪相机E2和光场显示屏16。该眼部追踪相机E2能够对光场通信设备10的观看对象的眼部(例如瞳孔)的位置进行追踪，以确定该眼部在光场显示屏16上的注视点的位置，即第二注视点位置。其中，该第二注视点位置可以是观看对象的瞳孔在光场显示屏16上的投影位置。
在本申请实施例中，该光场通信设备10不仅可以作为光场数据采集设备向光场显示设20备提供光场数据，还能够作为显示设备对光场数据进行光场显示。其中，光场通信设备10在与光场显示设备20建立通信连接后，可以通过该眼部追踪相机E2获取该第二注视点位置。例如，光场通信设备10可以实时获取并发送该第二注视点位置，或者，光场通信设备10可以按照预设的追踪周期获取并发送该第二注视点位置。
步骤207、光场通信设备将第二注视点位置传输至光场显示设备。
参考图4，该光场显示设备20可以包括主控电路22。光场通信设备10中的眼部追踪相机E2获取到第二注视点位置后，可以将该第二注视点位置传输至光场显示设备20的主控电路22。
步骤208、光场显示设备根据注视点位置和相机的对应关系，从多个第二相机中确定与第二注视点位置对应的至少一个目标相机。
在本申请实施例中，如图4所示，该光场显示设备20可以包括多个第二相机21。光场显示设备20的主控电路22也可以预先存储有注视点位置和相机的对应关系。该对应关系中记录有多个不同的注视点位置，每个注视点位置对应至少一个第二相机21，且每个注视点位置对应的至少一个第二相机21的数量小于多个第二相机21的数量。该主控电路22能够接收光场通信设备10发送的第二注视点位置，并基于上述注视点位置和相机的对应关系确定出该第二注视点位置对应的至少一个目标相机。
或者，该光场显示设备20可以基于该第二注视点位置确定观看对象的视场 范围，进而根据该视场范围和每个第二相机21的光场数据采集范围，确定出至少一个目标相机。该至少一个目标相机采集到的第二光场数据在光场通信设备10的光场显示屏16上的显示区域能够覆盖该视场范围。
步骤209、光场显示设备通过多个数据处理芯片中的至少一个目标数据处理芯片，对至少一个目标相机采集到的第二光场数据进行处理。
参考图4，该光场显示设备20还可以包括数据处理电路23，该数据处理电路23可以包括多个数据处理芯片231。该多个数据处理芯片231可以与多个第二相机21建立连接，且每个数据处理芯片231与至少一个第二相机21连接。该光场显示设备20中的多个第二相机21与多个数据处理芯片231的连接拓扑，可以参考上述描述中对于光场通信设备10中的多个第一相机11与多个数据处理芯片131的连接拓扑描述，在此不再赘述。
在本申请实施例中，主控电路22从多个第二相机21确定出至少一个目标相机后，能够控制数据处理电路23中的至少一个目标数据处理芯片获取该至少一个目标相机采集到的第二光场数据，并对第二光场数据进行处理。
步骤210、光场显示设备将处理后的第二光场数据发送至光场通信设备。
该步骤的实现过程可以参考上述步骤204的相关描述，此处不再赘述。
步骤211、光场通信设备基于第二光场数据进行光场显示。
该步骤的实现过程可以参考上述步骤205的相关描述，此处不再赘述。
在本申请实施例中，该光场通信设备10可以作为光场数据的采集设备向光场显示设备20提供光第一场数据，光场显示设备20作为显示设备，基于该光场通信设备10提供的第一光场数据进行光场显示。并且，该光场显示设备20还可以作为光场数据的采集设备向光场通信设备10提供第二光场数据，该光场通信设备10进而可以作为显示设备，基于该光场显示设备20提供的第二光场数据进行光场显示。由此，可以实现光场通信设备10和光场显示设备20之间双向的光场数据的传输和显示。
示例的，该光场通信设备10和光场显示设备20可以实现3D视频通信。在光场通信设备10和光场显示设备20建立通信连接后，该两个设备中的每个设备均可以基于另一设备反馈的注视点位置，实时获取该注视点位置所对一个的至少一个目标相机采集到的光场数据并进行处理。之后，该两个设备可以对处理后的光场数据进行传输，并对接收到的光场数据进行光场显示。由此，可以 实现实时的、双向的3D视频通信。
可以理解的是，本申请实施例提供的光场数据的传输方法的步骤的先后顺序可以进行适当调整，步骤也可以根据情况进行相应增减。例如，上述步骤206至步骤211可以根据情况删除。或者，上述步骤206至步骤211可以在步骤201之前执行。再或者，步骤206可以和步骤201同步执行。任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化的方法，都应涵盖在本申请的保护范围之内，因此不再赘述。
综上所述，本申请实施例提供了一种光场数据的传输方法，光场通信设备能够基于光场显示设备反馈的第一注视点位置，对该第一注视点位置对应的至少一个目标相机采集到的第一光场数据进行处理后传输至光场显示设备。该光场显示设备进而可以在其光场显示屏上对该第一光场数据进行光场显示。由于光场通信设备仅需处理并传输与第一注视点位置对应的部分光场相机采集到的光场数据，因此可以在不影响观看效果的前提下，有效提高光场数据处理的效率，并降低光场数据占用的传输带宽。
本申请实施例提供了一种光场通信设备，参考图4，该光场通信设备10包括：主控电路12，数据处理电路13以及多个第一相机11。
该主控电路12用于：接收光场显示设备20发送的第一注视点位置，该第一注视点位置为光场显示设备20的观看对象的眼部在光场显示设备20的光场显示屏26上的注视点的位置；以及，从多个第一相机11中确定与第一注视点位置对应的至少一个目标相机，该至少一个目标相机的数量小于多个第一相机11的数量。
该数据处理电路13，用于对至少一个目标相机采集到的第一光场数据进行处理后传输至光场显示设备20，该处理后的第一光场数据用于供光场显示设备20进行光场显示。
其中，该主控电路12可以是中央处理器(central processing unit，CPU)或微控制单元(micro-controller unit，MCU)。
可选地，参考图6，该多个第一相机11可以设置在光场显示屏的周围。
可选地，主控电路12中存储有注视点位置和相机的对应关系。该对应关系中记录有多个不同的注视点位置，每个注视点位置对应至少一个第一相机11， 且对应的至少一个第一相机11的数量小于多个第一相机11的数量。该主控电路12，用于根据对应关系，确定与第一注视点位置对应的至少一个目标相机。
可选地，如图4所示，该数据处理电路13包括多个数据处理芯片131，每个数据处理芯片131与至少一个第一相机11连接。多个数据处理芯片131中的至少一个目标数据处理芯片与至少一个目标相机连接，该至少一个目标数据处理芯片用于对至少一个目标相机采集到的第一光场数据进行处理。
其中，该数据处理芯片131可以是GPU。或者该数据处理芯片131可以是可编程逻辑器件(programmable logic device，PLD)，例如可以是复杂可编程逻辑器件(complex programmable logic device，CPLD)，现场可编程逻辑门阵列(field programmable gate array，FPGA)或通用阵列逻辑(generic array logic，GAL)等，本申请实施例对此不做限定。
可选地，第一注视点位置对应多个目标相机，该多个目标相机与多个不同的目标数据处理芯片一一对应连接。
可选地，每个数据处理芯片131与多个第一相机11连接；该主控电路12，用于控制至少一个目标数据处理芯片对至少一个目标相机采集到的第一光场数据进行处理，且不对除至少一个目标相机之外的第一相机11进行处理。
可选地，该主控电路12还用于：控制至少一个目标相机采集光场数据，并控制多个第一相机11中除至少一个目标相机之外的第一相机11停止传输光场数据。
可选地，如图4所示，该光场通信设备10还可以包括眼部追踪相机E2、显示处理电路15和光场显示屏16。
其中，该眼部追踪相机E2，用于获取第二注视点位置，并将该第二注视点位置传输至光场显示设备20，该第二注视点位置为光场通信设备10的观看对象的眼部在光场通信设备10的光场显示屏16上的注视点的位置。该眼部追踪相机E2可以设置在光场显示屏16的周围。
该显示处理电路15，用于接收并处理光场显示设备20传输的第二光场数据。该第二光场数据是光场显示设备20中的至少一个第二相机21采集到的，且该至少一个第二相机21基于第二注视点位置确定。该光场显示屏26，用于基于第二光场数据进行光场显示。
综上所述，本申请实施例提供了一种光场通信设备，该光场通信设备能够 基于光场显示设备反馈的第一注视点位置，对该第一注视点位置对应的至少一个目标相机采集到的第一光场数据进行处理后传输至光场显示设备。该光场显示设备进而可以在其光场显示屏上对该第一光场数据进行光场显示。由于光场通信设备仅需处理并传输与第一注视点位置对应的部分光场相机采集到的光场数据，因此可以在不影响观看效果的前提下，有效提高光场数据处理的效率，并降低光场数据占用的传输带宽。
可以理解的是，上述实施例提供的光场通信设备，仅以上述各功能模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的功能模块完成，即将设备的内部结构划分成不同的功能模块，以完成以上描述的全部或者部分功能。
另外，上述实施例提供的图光场通信设备和光场数据的传输方法实施例属于同一构思，其具体实现过程详见方法实施例，这里不再赘述。
本申请实施例还提供了一种光场通信设备，该光场通信设备可以为计算机设备。如图8所示，该光场通信设备包括：多个第一相机11，处理器101和存储器102。该存储器102中存储有指令，该指令由处理器101加载并执行以实现上述方法实施例提供的光场数据的传输方法。
本申请的实施例还提供了一种计算机可读存储介质，该存储介质中存储有指令，指令由处理器加载并执行以实现上述方法实施例提供的光场数据的传输方法。
本申请的实施例还提供了一种计算机程序产品或计算机程序，该计算机程序产品或计算机程序包括计算机指令，由处理器加载并执行以实现如上述方面所述的光场数据的传输方法。
可以理解的是，本申请中术语“至少一个”是指一个或多个，“多个”的含义是指两个或两个以上。
本领域普通技术人员可以理解实现上述实施例的全部或部分步骤可以通过硬件来完成，也可以通过程序来指令相关的硬件完成，所述的程序可以存储于一种计算机可读存储介质中，上述提到的存储介质可以是只读存储器，磁盘或光盘等。
以上仅为本申请的示例性实施例，并不用以限制本申请，凡在本申请的精 神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本申请的保护范围之内。

Claims (15)

  1. 一种光场数据的传输方法，其特征在于，应用于光场通信设备，所述光场通信设备包括多个第一相机；所述方法包括：
    接收光场显示设备发送的第一注视点位置，所述第一注视点位置为所述光场显示设备的观看对象的眼部在所述光场显示设备的光场显示屏上的注视点的位置；
    从所述多个第一相机中确定与所述第一注视点位置对应的至少一个目标相机，所述至少一个目标相机的数量小于所述多个第一相机的数量；
    对所述至少一个目标相机采集到的第一光场数据进行处理后传输至所述光场显示设备，处理后的所述第一光场数据用于供所述光场显示设备进行光场显示。
  2. 根据权利要求1所述的方法，其特征在于，所述光场通信设备中存储有注视点位置和相机的对应关系，所述对应关系中记录有多个不同的注视点位置，每个注视点位置对应至少一个第一相机，且每个注视点位置对应的至少一个第一相机的数量小于所述多个第一相机的数量；
    所述从所述多个第一相机中确定与所述第一注视点位置对应的至少一个目标相机，包括：
    根据所述对应关系，确定与所述第一注视点位置对应的至少一个目标相机。
  3. 根据权利要求1或2所述的方法，其特征在于，所述光场通信设备还包括：多个数据处理芯片，每个数据处理芯片与至少一个第一相机连接；
    所述对所述至少一个目标相机采集到的第一光场数据进行处理，包括：
    通过所述多个数据处理芯片中的至少一个目标数据处理芯片，对所述至少一个目标相机采集到的第一光场数据进行处理，所述至少一个目标数据处理芯片与所述至少一个目标相机连接。
  4. 根据权利要求3所述的方法，其特征在于，所述第一注视点位置对应多个目标相机；
    所述多个目标相机与多个不同的目标数据处理芯片一一对应连接。
  5. 根据权利要求3或4所述的方法，其特征在于，每个数据处理芯片与多个第一相机连接；
    所述通过所述多个数据处理芯片中的至少一个目标数据处理芯片，对所述至少一个目标相机采集到的第一光场数据进行处理，包括：
    控制所述至少一个目标数据处理芯片对所述至少一个目标相机采集到的第一光场数据进行处理，且不对除所述至少一个目标相机之外的第一相机进行处理。
  6. 根据权利要求1至5任一所述的方法，其特征在于，在对所述至少一个目标相机采集到的第一光场数据进行处理之前，所述方法还包括：
    控制所述至少一个目标相机采集光场数据，并控制所述多个第一相机中除所述至少一个目标相机之外的第一相机停止传输光场数据。
  7. 根据权利要求1至6任一所述的方法，其特征在于，所述光场通信设备还包括眼部追踪相机和光场显示屏；所述方法还包括：
    通过所述眼部追踪相机获取第二注视点位置，所述第二注视点位置为所述光场通信设备的观看对象的眼部在所述光场通信设备的光场显示屏上的注视点的位置；
    将所述第二注视点位置传输至所述光场显示设备；
    接收所述光场显示设备传输的第二光场数据，所述第二光场数据是所述光场显示设备中的至少一个第二相机采集到的，且所述至少一个第二相机基于所述第二注视点位置确定；
    基于所述第二光场数据进行光场显示。
  8. 一种光场通信设备，其特征在于，所述光场通信设备包括：主控电路，数据处理电路以及多个第一相机；所述主控电路，用于：
    接收光场显示设备发送的第一注视点位置，所述第一注视点位置为所述光场显示设备的观看对象的眼部在所述光场显示设备的光场显示屏上的注视点的 位置；
    以及，从所述多个第一相机中确定与所述第一注视点位置对应的至少一个目标相机，所述至少一个目标相机的数量小于所述多个第一相机的数量；
    所述数据处理电路，用于对所述至少一个目标相机采集到的第一光场数据进行处理后传输至所述光场显示设备，处理后的所述第一光场数据用于供所述光场显示设备进行光场显示。
  9. 根据权利要求8所述的光场通信设备，其特征在于，所述主控电路中存储有注视点位置和相机的对应关系，所述对应关系中记录有多个不同的注视点位置，每个注视点位置对应至少一个第一相机，且对应的至少一个第一相机的数量小于所述多个第一相机的数量；
    所述主控电路，用于根据所述对应关系，确定与所述第一注视点位置对应的至少一个目标相机。
  10. 根据权利要求8或9所述的光场通信设备，其特征在于，所述数据处理电路包括多个数据处理芯片，每个数据处理芯片与至少一个第一相机连接；
    所述多个数据处理芯片中的至少一个目标数据处理芯片与所述至少一个目标相机连接，所述至少一个目标数据处理芯片用于对所述至少一个目标相机采集到的第一光场数据进行处理。
  11. 根据权利要求10所述的光场通信设备，其特征在于，所述第一注视点位置对应多个目标相机；
    所述多个目标相机与多个不同的目标数据处理芯片一一对应连接。
  12. 根据权利要求10或11所述的光场通信设备，其特征在于，每个数据处理芯片与多个第一相机连接；所述主控电路，用于：
    控制所述至少一个目标数据处理芯片对所述至少一个目标相机采集到的第一光场数据进行处理，且不对除所述至少一个目标相机之外的第一相机进行处理。
  13. 根据权利要求8至12任一所述的光场通信设备，其特征在于，所述主控电路还用于：
    控制所述至少一个目标相机采集光场数据，并控制所述多个第一相机中除所述至少一个目标相机之外的第一相机停止传输光场数据。
  14. 根据权利要求8至13任一所述的光场通信设备，其特征在于，所述光场通信设备还包括眼部追踪相机、显示处理电路和光场显示屏；
    所述眼部追踪相机，用于获取第二注视点位置，并将所述第二注视点位置传输至所述光场显示设备，所述第二注视点位置为所述光场通信设备的观看对象的眼部在所述光场通信设备的光场显示屏上的注视点的位置；
    所述显示处理电路，用于接收并处理所述光场显示设备传输的第二光场数据，所述第二光场数据是所述光场显示设备中的至少一个第二相机采集到的，且所述至少一个第二相机基于所述第二注视点位置确定；
    所述光场显示屏，用于基于所述第二光场数据进行光场显示。
  15. 一种光场数据的传输系统，其特征在于，所述系统包括：光场通信设备和光场显示设备；
    其中，所述光场通信设备为权利要求8至14任一所述的光场通信设备；
    所述光场显示设备用于向所述光场通信设备发送第一注视点位置，以及基于所述光场通信设备传输的第一光场数据进行光场显示。
PCT/CN2022/089380 2022-04-26 2022-04-26 光场数据的传输方法、光场通信设备及系统 WO2023206098A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202280000932.9A CN117413512A (zh) 2022-04-26 2022-04-26 光场数据的传输方法、光场通信设备及系统
PCT/CN2022/089380 WO2023206098A1 (zh) 2022-04-26 2022-04-26 光场数据的传输方法、光场通信设备及系统

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2022/089380 WO2023206098A1 (zh) 2022-04-26 2022-04-26 光场数据的传输方法、光场通信设备及系统

Publications (1)

Publication Number Publication Date
WO2023206098A1 true WO2023206098A1 (zh) 2023-11-02

Family

ID=88516551

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/089380 WO2023206098A1 (zh) 2022-04-26 2022-04-26 光场数据的传输方法、光场通信设备及系统

Country Status (2)

Country Link
CN (1) CN117413512A (zh)
WO (1) WO2023206098A1 (zh)

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101072366A (zh) * 2007-05-24 2007-11-14 上海大学 基于光场和双目视觉技术的自由立体显示系统和方法
US20140104392A1 (en) * 2012-10-11 2014-04-17 Sony Mobile Communications Ab Generating image information
US20180173303A1 (en) * 2016-12-21 2018-06-21 Oculus Vr, Llc Eye tracking using a light field camera on a head-mounted display
CN108604390A (zh) * 2016-05-25 2018-09-28 谷歌有限责任公司 针对头戴式显示设备的光场视点和像素剔除
WO2019233210A1 (zh) * 2018-06-06 2019-12-12 北京七鑫易维信息技术有限公司 一种智能眼镜、眼球轨迹的追踪方法、装置及存储介质
US10558264B1 (en) * 2016-12-21 2020-02-11 X Development Llc Multi-view display with viewer detection
US10712572B1 (en) * 2016-10-28 2020-07-14 Facebook Technologies, Llc Angle sensitive pixel array including a liquid crystal layer
US10979685B1 (en) * 2017-04-28 2021-04-13 Apple Inc. Focusing for virtual and augmented reality systems
US20210132693A1 (en) * 2019-11-01 2021-05-06 Raxium, Inc. Light Field Displays Incorporating Eye Trackers and Methods for Generating Views for a Light Field Display Using Eye Tracking Information
CN113138664A (zh) * 2021-03-30 2021-07-20 青岛小鸟看看科技有限公司 基于光场感知的眼球追踪系统、方法

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101072366A (zh) * 2007-05-24 2007-11-14 上海大学 基于光场和双目视觉技术的自由立体显示系统和方法
US20140104392A1 (en) * 2012-10-11 2014-04-17 Sony Mobile Communications Ab Generating image information
CN108604390A (zh) * 2016-05-25 2018-09-28 谷歌有限责任公司 针对头戴式显示设备的光场视点和像素剔除
US10712572B1 (en) * 2016-10-28 2020-07-14 Facebook Technologies, Llc Angle sensitive pixel array including a liquid crystal layer
US20180173303A1 (en) * 2016-12-21 2018-06-21 Oculus Vr, Llc Eye tracking using a light field camera on a head-mounted display
US10558264B1 (en) * 2016-12-21 2020-02-11 X Development Llc Multi-view display with viewer detection
US10979685B1 (en) * 2017-04-28 2021-04-13 Apple Inc. Focusing for virtual and augmented reality systems
WO2019233210A1 (zh) * 2018-06-06 2019-12-12 北京七鑫易维信息技术有限公司 一种智能眼镜、眼球轨迹的追踪方法、装置及存储介质
US20210132693A1 (en) * 2019-11-01 2021-05-06 Raxium, Inc. Light Field Displays Incorporating Eye Trackers and Methods for Generating Views for a Light Field Display Using Eye Tracking Information
CN113138664A (zh) * 2021-03-30 2021-07-20 青岛小鸟看看科技有限公司 基于光场感知的眼球追踪系统、方法

Also Published As

Publication number Publication date
CN117413512A (zh) 2024-01-16

Similar Documents

Publication Publication Date Title
JP7001675B2 (ja) ３ｄテレプレゼンスシステム
EP2383995B1 (en) Display system, method and computer program for capturing images using multiple integrated image sensors
US10613624B2 (en) Display driving circuit, driving method thereof and display device
CN107852487A (zh) 用于生成360度三维图像的电子设备及用于该电子设备的方法
WO2023071574A1 (zh) 3d影像的重构方法、装置、电子设备及存储介质
CN103430531B (zh) 视频处理装置、视频处理系统和视频处理方法
EP4270315A1 (en) Method and device for processing three-dimensional video, and storage medium
US11892637B2 (en) Image generation apparatus, head-mounted display, content processing system, and image display method
US11783445B2 (en) Image processing method, device and apparatus, image fitting method and device, display method and apparatus, and computer readable medium
CN107018386A (zh) 一种视频流多分辨率观测系统
WO2023071603A1 (zh) 视频融合方法、装置、电子设备及存储介质
WO2018129963A1 (zh) 一种显示方法和装置和计算机存储介质
WO2018000892A1 (zh) 一种全景立体影像的成像方法、装置及系统
WO2018092992A1 (ko) 룩업테이블 기반의 실시간 파노라마 영상 제작 시스템 및 이를 이용한 실시간 파노라마 영상 제작 방법
CN110930932A (zh) 显示屏校正方法及系统
WO2023206098A1 (zh) 光场数据的传输方法、光场通信设备及系统
CN206611512U (zh) 三维全景视频监控装置
US11388391B2 (en) Head-mounted display having an image sensor array
US11106042B2 (en) Image processing apparatus, head-mounted display, and image displaying method
CN102959937B (zh) 用于稳健颜色传送的方法和设备
CN116860112B (zh) 一种基于xr技术的组合场景体验生成方法、系统及介质
JP2020167657A (ja) 画像処理装置、ヘッドマウントディスプレイ、および画像表示方法
JP7365183B2 (ja) 画像生成装置、ヘッドマウントディスプレイ、コンテンツ処理システム、および画像表示方法
CN111986333B (zh) 基于增强现实的图像生成方法、装置、终端及存储介质
CN116860113B (zh) 一种xr组合场景体验生成方法、系统及存储介质

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 202280000932.9

Country of ref document: CN

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22938954

Country of ref document: EP

Kind code of ref document: A1