WO2021134575A1 - Procédé et dispositif de commande d'affichage - Google Patents

Procédé et dispositif de commande d'affichage Download PDF

Info

Publication number
WO2021134575A1
WO2021134575A1 PCT/CN2019/130733 CN2019130733W WO2021134575A1 WO 2021134575 A1 WO2021134575 A1 WO 2021134575A1 CN 2019130733 W CN2019130733 W CN 2019130733W WO 2021134575 A1 WO2021134575 A1 WO 2021134575A1
Authority
WO
WIPO (PCT)
Prior art keywords
display
image
area
instruction information
preset
Prior art date
Application number
PCT/CN2019/130733
Other languages
English (en)
Chinese (zh)
Inventor
李栋
饶雄斌
Original Assignee
深圳市大疆创新科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市大疆创新科技有限公司 filed Critical 深圳市大疆创新科技有限公司
Priority to PCT/CN2019/130733 priority Critical patent/WO2021134575A1/fr
Priority to CN201980094964.8A priority patent/CN113661429B/zh
Priority to CN202410876485.4A priority patent/CN118625530A/zh
Publication of WO2021134575A1 publication Critical patent/WO2021134575A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays

Definitions

  • the embodiments of the present application relate to the field of control technology, and in particular, to a display control method and device.
  • head-mounted display devices With the development of augmented reality (Augmented Reality, AR) technology, head-mounted display devices have become more and more widely used. At present, head-mounted display devices can be combined with mobile platforms (such as drones and unmanned vehicles). .
  • the camera on the movable platform can take a real-time picture of the surrounding environment of the movable platform, and display the real-time picture through the head-mounted display device.
  • FV First Personal View
  • the user puts on the head-mounted display device he can only see the picture taken by the camera of the movable platform, and cannot observe his surrounding environment, which is prone to danger.
  • the embodiments of the present application provide a display control method and device, which are used to enable users to observe their surrounding environment through a head-mounted display device to avoid danger.
  • an embodiment of the present application provides a display control method, which is applied to a head-mounted display device, and the head-mounted display device includes a photographing device for capturing images of the surrounding environment of the head-mounted display device and Display device.
  • the method includes: receiving a first image collected by the movable platform from a movable platform; acquiring a second image collected by the photographing device; acquiring display instruction information, according to the display instruction information from the first image and Determine the target image in the second image; control the display device to display the target image.
  • an embodiment of the present application provides a display control method, which is applied to a head-mounted display device, and the head-mounted display device includes a photographing device for capturing images of the surrounding environment of the head-mounted display device and Display device.
  • the method includes: receiving a first image collected by the movable platform from a movable platform; acquiring a second image collected by the photographing device; controlling the display device to simultaneously display the first image and the second image .
  • an embodiment of the present application provides a head-mounted display device, including: a photographing device, a display device, a communication device, and a processor.
  • the communication device is configured to receive the first image collected by the movable platform from the movable platform.
  • the photographing device is used to collect a second image of the surrounding environment of the head-mounted display device.
  • the processor is configured to obtain display instruction information, determine a target image from the first image and the second image according to the display instruction information, and control the display device to display the target image.
  • the display device is used to display the target image.
  • an embodiment of the present application provides a head-mounted display device, including: a photographing device, a display device, a communication device, and a processor.
  • the communication device is configured to receive the first image collected by the movable platform from the movable platform.
  • the photographing device is used to collect a second image of the surrounding environment of the head-mounted display device.
  • the processor is configured to control the display device to simultaneously display the first image and the second image.
  • the display device is used for simultaneously displaying the first image and the second image.
  • an embodiment of the present application provides a readable storage medium with a computer program stored on the readable storage medium; when the computer program is executed, the embodiment of the present application is implemented as in the first aspect or the second aspect.
  • the display control method when the computer program is executed, the embodiment of the present application is implemented as in the first aspect or the second aspect.
  • an embodiment of the present application provides a program product, the program product includes a computer program, the computer program is stored in a readable storage medium, and at least one processor of a head-mounted display device can read from the computer program.
  • the storage medium reads the computer program, and the at least one processor executes the computer program to enable the head-mounted display device to implement the display control method according to the embodiment of the present application in the first aspect or the second aspect.
  • the display instruction information is acquired by receiving the first image collected by the movable platform from the movable platform and acquiring the second image collected by the shooting device in the head-mounted display device,
  • the target image is determined from the first image and the second image according to the display instruction information, and the display device in the head-mounted display device is controlled to display the target image.
  • the user wears the head-mounted display device, the user can view the target image through the display device. Since the target image is determined from the first image and the second image, the user can view the surrounding environment where he is Danger, ensure safety.
  • Figure 1 is a schematic diagram of an application scenario provided by an embodiment of the application
  • FIG. 2 is a schematic diagram of a head-mounted display device provided by an embodiment of the application.
  • FIG. 3 is a flowchart of a display control method provided by an embodiment of the application.
  • 4a is a schematic diagram of a display device provided by an embodiment of the application simultaneously displaying a first image and a second image;
  • 4b is a schematic diagram of the display device provided by another embodiment of the application simultaneously displaying the first image and the second image;
  • FIG. 4c is a schematic diagram of the display device simultaneously displaying the first image and the second image according to another embodiment of the application;
  • FIG. 5 is a schematic diagram of a switch button used to detect a user's display instruction operation in a control terminal according to an embodiment of the application;
  • FIG. 6 is a flowchart of a display control method provided by another embodiment of this application.
  • FIG. 7 is a schematic structural diagram of a head-mounted display device provided by an embodiment of the application.
  • FIG. 8 is a schematic structural diagram of a head-mounted display device provided by another embodiment of the application.
  • FIG. 9 is a schematic structural diagram of a display control system provided by an embodiment of the application.
  • a component when referred to as being "fixed to” another component, it can be directly on the other component or a centered component may also exist. When a component is considered to be “connected” to another component, it can be directly connected to the other component or there may be a centered component at the same time.
  • the display control method and device provided by the embodiments of the present application can be applied to scenes that display images taken during the movement of the movable platform.
  • the movable platform can be configured to move by external force or be driven by its own power.
  • the movable platform is, for example, an unmanned aerial vehicle, an unmanned vehicle, a robot, etc.
  • the unmanned aerial vehicle may be, for example, a traversing machine.
  • Fig. 1 is a schematic diagram of an application scenario provided by an embodiment of the application. As shown in Fig. 1, Fig. 1 shows that the movable platform is a drone 101, a control terminal 102 of the drone, and a head-mounted display device 103.
  • the control terminal 102 of the drone 101 may be one or more of a remote control, a smart phone, a desktop computer, a laptop computer, and a wearable device (watch, bracelet).
  • the control terminal 102 is a remote controller 1021 and a terminal device 1022 as an example for schematic description.
  • the terminal device 1022 is, for example, a smart phone, a wearable device, a tablet computer, etc., but the embodiment of the present application is not limited thereto.
  • the head-mounted display device 103 may be, for example, head-mounted display glasses, as shown in FIG. 2.
  • the drone 101 includes a photographing device 1011, and the drone 101 can obtain an image output by the photographing device 1011.
  • the head-mounted display device 103 may receive the image obtained from the drone 101 through direct or indirect, wired or wireless communication.
  • the head-mounted display device 103 includes a photographing device 1031, and the head-mounted display device 103 can acquire an image output by the photographing device 1031.
  • the head-mounted display device 103 further includes a display device 1032, and the head-mounted display device 103 can control the display device 1032 to display the target image based on the image obtained from the drone 101 and the image obtained from the photographing device 1031.
  • the head-mounted display device 103 can directly communicate with the drone 101, or the head-mounted display device 103 can directly communicate with the control terminal 102 of the drone, or it can communicate with the drone 101 through the control terminal 102. To communicate.
  • the user can control the head-mounted display device 103 by operating the control terminal 102, or directly operate the head-mounted display device 103 to control the head-mounted display device 103.
  • FIG. 3 is a flowchart of a display control method provided by an embodiment of the application. As shown in FIG. 3, the method of this embodiment can be applied to a head-mounted display device, and the method of this embodiment can include:
  • S301 Receive a first image collected by the movable platform from the movable platform.
  • the head-mounted display device includes a photographing device (such as the photographing device 1031 in FIG. 1) and a display device (such as the photographing device 1031 in FIG. 1) for collecting images of the surrounding environment of the head-mounted display device.
  • Display device 1032 The head-mounted display device may receive an image collected by a movable platform of the movable platform, the image is an image of the surrounding environment of the movable platform, and the image is called the first image.
  • the head-mounted display device can directly communicate with the movable platform, and the head-mounted display device receives the first image sent by the movable platform.
  • the head-mounted display device can receive the first image sent by the movable platform in a wired or wireless manner. image.
  • the head-mounted display device may indirectly communicate with the movable platform, and the movable platform sends the collected first image to the control terminal of the movable platform, and then the head-mounted display device receives the first image sent by the control terminal .
  • the head-mounted display device includes a photographing device that can photograph an image, and the photographing device photographs an image of the surrounding environment of the head-mounted display device, and the image is called a second image.
  • the head-mounted display device can acquire the second image output by the photographing device.
  • S303 Obtain display instruction information, and determine a target image from the first image and the second image according to the display instruction information.
  • the head-mounted display device further includes a display device.
  • the head-mounted display device acquires display instruction information, and the display instruction information can be used to indicate the target image to be displayed by the head-mounted display device. Then, according to the display instruction information, a target image is determined from the first image and the second image, and the target image is the image to be displayed by the display device.
  • S304 Control the display device to display the target image.
  • the head-mounted display device controls the display device to display the target image.
  • the user can view the target image through the display device, which has the viewing experience of FPV and has an immersive effect.
  • the target image may be the first image.
  • a possible implementation of S304 is to control the display device to display the first image.
  • the target image is the second image
  • a possible implementation of S304 is to control the display device to display the second image.
  • the target image is the first image and the second image.
  • a possible implementation of S304 is to control the display device to display the first image and the second image at the same time. Therefore, the user can view the first image or the second image through the display device, or view the first image and the second image at the same time.
  • the display device is provided on the housing of the head-mounted display device, so that when the user wears the head-mounted display device, the display device can capture an image of the surrounding environment where the user is located, for example, as shown in FIG. 1
  • the position of the display device 1032 in the head-mounted display device 103 shown in this embodiment is not limited to this.
  • the shooting direction of the shooting device deviates from the display direction of the display device, for example, as shown in FIG. 1, so that the shooting direction of the shooting device is the same as the viewing direction of the user's eyes, so that when the display device displays the shooting device shooting The image is as if the user sees it with his own eyes.
  • the first image collected by the movable platform is received from the movable platform, and the second image collected by the shooting device in the head-mounted display device is obtained to obtain the display instruction information.
  • the display instruction information determines the target image from the first image and the second image, and controls the display device in the head-mounted display device to display the target image.
  • the user wears the head-mounted display device, the user can view the target image through the display device. Since the target image is determined from the first image and the second image, the user can view the surrounding environment where he is Danger, ensure safety.
  • different target images are determined from the first image and the second image by obtaining different display indication information.
  • the display indication information may be, for example, the first display indication information or the second display indication information or the third display indication information.
  • a possible implementation of the above S303-S304 is: acquiring the display instruction information as the first display instruction information, and then determining the first image from the first image and the second image according to the first display instruction information, that is, determining that the display device requires The displayed target image is the first image, and then the display device is controlled to display the first image.
  • a possible implementation of the above S303-S304 is: acquiring the display instruction information as the second display instruction information, and then determining the second image from the first image and the second image according to the second display instruction information, that is, determining that the display device requires The displayed target image is the second image, and then the display device is controlled to display the second image.
  • a possible implementation of the above S303-S304 is: acquiring the display instruction information as the third display instruction information, and then determine the first image and the second image from the first image and the second image according to the third display instruction information, that is, It is determined that the target image to be displayed by the display device is the first image and the second image, and then the display device is controlled to simultaneously display the first image and the second image.
  • this embodiment controls the display device to display different images by acquiring different display indication information, so as to meet the display requirements in different application scenarios.
  • a possible implementation manner of controlling the display device to display the first image and the second image at the same time is: displaying the first image in the first preset display area of the display device; and The second image is displayed in a second preset display area of the display device.
  • the display area of the display device can be divided into two parts, one is the first preset display area, the other is the second preset display area, the first preset display area is used to display the first image, and the second The preset display area is used to display the second image.
  • the first preset display area and the second display area respectively occupy different display areas of the display device. Users can watch different images by watching different display areas.
  • the area of the first preset display area is larger than the area of the second preset display area. Since the area of the first preset display area is larger than the area of the second preset display area, the display device mainly displays images taken by the movable platform, and the user mainly views the movable platform through the first preset display area with a larger area The captured image, when the user wants to view the image of the surrounding environment, the image of the surrounding environment where the user is located can be viewed through the second preset display area with a smaller area.
  • the second preset display area is a part of the display area in the first preset display area.
  • the first preset display area may be the entire display area of the display device, and the second preset display area is, for example, a part of the display area in the lower right corner, which is equivalent to the part where the second image is superimposed and displayed on the first image.
  • the second image may obscure part of the image in the first image.
  • a possible implementation of controlling the display device to display the first image and the second image at the same time is: according to the first image and the second image, the first image and the second image are spliced to obtain the splicing Then, the display device is controlled to display the spliced image, where the spliced image includes part of the first image and the second image.
  • the first image and the second image are spliced to obtain the spliced image, for example, according to the first image and the second image, the same image in the first image and the second image is determined, and the first image is determined according to the same image.
  • the image and the second image are stitched together to obtain a stitched image.
  • the image of the surrounding environment of the movable platform captured by the camera device (the first image) will be the same as the image captured by the movable platform (Second image)
  • the first image and the second image are spliced, and the screen of the movable platform in the second image is replaced with the first image, and the image shown in Figure 4c can be obtained.
  • the display image, the middle image is the second image, and the surrounding images are the second image. Therefore, the user can watch the image taken by the movable platform through the middle screen, and watch the image of the surrounding environment through the surrounding screen.
  • a possible implementation manner of controlling the display device to display the first image and the second image at the same time is: fusing the first image and the second image through Augmented Reality (AR) technology, and Control the display device to display the fused image.
  • AR Augmented Reality
  • This can bring users an experience of alternating AR and virtual reality (Virtual Reality, VR) or the integration of AR and VR.
  • the area of the first preset display area and the area of the second preset display area are determined according to a relative position parameter between a reference object and the movable platform.
  • the area of the first preset display area and the area of the second preset display area are determined according to the relative orientation between the reference object and the movable platform.
  • the reference object is the user of the head-mounted display device, the head-mounted display device, or the control terminal of the movable platform.
  • the user is a user wearing a head-mounted display device, and its position may reflect the position of the head-mounted display device.
  • the control terminal of the movable platform is mainly operated by the user, so the position of the control terminal can reflect the position of the user wearing the head-mounted display device.
  • the head-mounted display device displays the first image in the first preset display area of the display device, and before displaying the second image in the second preset display area of the display device,
  • the mobile platform can obtain the relative position parameter or relative position between the reference object and the movable platform, and determine the area of the first preset display area and the area of the second preset display area according to the relative position parameter or relative position, and then wear it to the head
  • the display device sends the area of the first preset display area and the area of the second preset display area
  • the head-mounted display device receives the area of the first preset display area and the area of the second preset display area from the movable platform , According to the area of the first preset display area and the area of the second preset display area, determine the first preset display area and the second preset display area in the display device.
  • the head-mounted display device of this embodiment displays the first image in a first preset display area of the display device and displays the second image in a second preset display area of the display device Previously, the relative position parameter between the reference object and the movable platform was obtained, and the area of the first preset display area and the area of the second preset display area were determined according to the relative position parameter or relative orientation. According to the area of the first preset display area and the area of the second preset display area, the first preset display area and the second preset display area in the display device are determined.
  • the above-mentioned relative position parameter includes a relative distance, wherein when the relative distance is within a first preset distance range, the area of the first preset display area is smaller than that of the second preset display area Area. When the relative distance is in the second preset distance range, the area of the first preset display area is larger than the area of the second preset display area. Therefore, when the relative distance between the reference corresponding and the movable platform is in different preset distance ranges, the size relationship between the area of the first preset display area and the area of the second preset display area is different.
  • the area of the first preset display area is smaller than the area of the second preset display area, the larger part of the image that the user sees is the image taken by the camera in the head-mounted display device, and the smaller part of the image is Images taken on a movable platform.
  • the area of the first preset display area is larger than the area of the second preset display area, the larger part of the image that the user sees is the image taken by the movable platform, and the smaller part of the image is The image captured by the camera in the head-mounted display device.
  • the first preset distance range is smaller than the second preset distance range.
  • the second images which are taken by the camera in the head-mounted display device. Image.
  • most of the images displayed by the display device are the first image, that is, the image taken by the mobile platform.
  • the camera in the head-mounted display device it is also difficult for the camera in the head-mounted display device to capture the image taken by the movable platform, so the area of the first preset display area is set to be larger than the second preset display area With a large area, users can mainly watch the images taken by the movable platform.
  • the camera in the head-mounted display device may also capture the image taken by the movable platform, so by setting the area of the first preset display area to be smaller than the second preset display
  • the area of the area allows the user to mainly watch the image taken by the camera in the head-mounted display device, which guarantees safety to a greater extent.
  • a possible implementation manner of obtaining the display indication information is: obtaining the display indication information sent by the control terminal of the movable platform, where the display indication information is the display indication of the control terminal detecting the user Operation generated.
  • the control terminal detects the user's display instruction operation, it generates a display instruction according to the display instruction operation And send the display instruction information to the head-mounted display device, and accordingly, the head-mounted display device receives the display instruction information sent by the control terminal.
  • the user When the user wants to control the head-mounted display device to display the first image, the user performs a first display instruction operation on the control terminal, and the control terminal detects the user's first display instruction operation, and generates the first display instruction operation according to the first display instruction operation.
  • the indication information is displayed, and the first display indication information is sent to the head-mounted display device. Accordingly, the head-mounted display device receives the first display indication information sent by the control terminal.
  • the user When the user wants to control the head-mounted display device to display the second image, the user performs a second display instruction operation on the control terminal, and the control terminal detects the second display instruction operation of the user, and generates a second display instruction operation according to the second display instruction operation.
  • the indication information is displayed, and the second display indication information is sent to the head-mounted display device. Accordingly, the head-mounted display device receives the second display indication information sent by the control terminal.
  • the user When the user wants to control the head-mounted display device to display the first image and the second image, the user performs a third display instruction operation on the control terminal, and the control terminal detects the user's third display instruction operation, and performs the third display instruction operation according to the third display instruction Operate to generate third display instruction information, and send the third display instruction information to the head-mounted display device. Accordingly, the head-mounted display device receives the third display instruction information sent by the control terminal.
  • the user can operate the switch key (as shown in FIG. 5) in the remote control 1021 to perform different display instruction operations.
  • the user can switch the display image of the head-mounted display device between the first image, the second image, the first image, and the third image by performing different display instruction operations on the control terminal.
  • a possible implementation manner of obtaining the display indication information is to detect a display indication operation of the user, and obtain the display indication information according to the display indication operation.
  • the user when the user needs to control the head-mounted display device to display the corresponding target image, the user will perform the corresponding display instruction operation on the head-mounted display device. After the head-mounted display device detects the user’s display instruction operation, it will The display instruction operation generates display instruction information.
  • the user When the user wants to control the head-mounted display device to display the first image, the user performs a first display instruction operation on the head-mounted display device, and the head-mounted display device detects the user's first display instruction operation, and performs the first display instruction operation according to the first display instruction operation.
  • the display instruction operation generates first display instruction information.
  • the user When the user wants to control the head-mounted display device to display the second image, the user performs a second display instruction operation on the head-mounted display device, and the head-mounted display device detects the user's second display instruction operation, and performs the second display instruction operation according to the second display instruction operation.
  • the display instruction operation generates second display instruction information.
  • the user When the user wants to control the head-mounted display device to display the first image and the second image, the user performs a third display instruction operation on the head-mounted display device, and the head-mounted display device detects the user's third display instruction operation, and According to the third display instruction operation, third display instruction information is generated.
  • switch key on the head-mounted display device, and the user can operate the switch key to perform different display instructions.
  • the switch key can be a physical button on the head-mounted display device, or it can be in the head-mounted display device. Virtual buttons displayed on the display device.
  • the user can perform different display instruction operations on the head-mounted display device to switch the display image of the head-mounted display device between the first image, the second image, the first image, and the third image.
  • the display indication information is determined according to a relative position parameter between the reference object and the movable platform.
  • the display indication information is determined according to the relative position between the reference object and the movable platform.
  • the reference object is the user of the head-mounted display device, the head-mounted display device, or the control terminal of the movable platform.
  • the user is a user wearing a head-mounted display device, and its position may reflect the position of the head-mounted display device.
  • the control terminal of the movable platform is mainly operated by the user, so the position of the control terminal can reflect the position of the user wearing the head-mounted display device.
  • the relative position parameter or relative position between the reference object and the movable platform is different, and the display indication information obtained by the head-mounted display device may be different.
  • the movable platform may obtain the relative position parameter or relative position between the reference object and the movable platform, determine the display indication information according to the relative position parameter or relative position, and then send The head-mounted display device sends the display instruction information.
  • a possible implementation manner for the head-mounted display device to obtain the display indication information is to receive the display indication information sent by the movable platform.
  • a possible implementation manner for the head-mounted display device to obtain the display indication information is: obtain the relative position parameter or relative position between the reference object and the movable platform, and determine the display indication information according to the relative position parameter or relative position .
  • the above-mentioned relative position parameters include relative distances.
  • the determined display indication information is different.
  • the display indication information may be, for example, the first One display instruction information, second display instruction information, or third display prompt information.
  • the acquired display instruction information is the first display instruction information according to the relative distance.
  • the acquired display indication information is the second display indication information according to the relative distance.
  • the acquired display indication information is the third display indication information.
  • the display instruction information acquired by the head-mounted display device is the first display instruction information, it is determined from the first image and the second image that the target image is the first image according to the first display instruction information. If the display instruction information acquired by the head-mounted display device is the second display instruction information, it is determined from the first image and the second image that the target image is the second image according to the first display instruction information. If the display instruction information acquired by the head-mounted display device is the third display instruction information, then according to the first display instruction information, it is determined from the first image and the second image that the target image is the first image and the second image.
  • the third preset distance range is greater than the fifth preset distance range
  • the fifth preset distance range is greater than the fourth preset distance range
  • the head-mounted display device acquires the second display indication information, And according to the second display instruction information, it is determined that the target image is the second image, and the display device is controlled to display the second image, that is, the image taken by the shooting device in the head-mounted display device.
  • the display device still displays the second image.
  • the head-mounted display device acquires the third display instruction information, and determines the target image according to the third display instruction information It is the first image and the second image, and controls the display device to display the first image and the second image at the same time, that is, the image taken by the mobile platform and the image taken by the camera in the head-mounted display device.
  • the head-mounted display device acquires the first display indication information, and determines the target image according to the first display indication information Is the first image, and controls the display device to display the first image, that is, the image taken by the mobile platform.
  • the picture displayed on the head-mounted display device enlarges the image taken by the movable platform until the displayed picture is switched from the image taken by the camera in the head-mounted display device to the movable platform.
  • the images taken by the mobile platform there is a user's field of vision looking at the movable platform from a distance, and then continuously approaching the movable platform, and finally becomes the field of view of the movable platform.
  • the head-mounted display device acquires the first display indication information, And according to the first display instruction information, it is determined that the target image is the first image, and the display device is controlled to display the first image, that is, the image taken by the mobile platform.
  • the display device still displays the first image.
  • the head-mounted display device acquires the third display instruction information, and determines the target image according to the third display instruction information It is the first image and the second image, and controls the display device to display the first image and the second image at the same time, that is, the image taken by the mobile platform and the image taken by the camera in the head-mounted display device.
  • the head-mounted display device acquires the second display indication information, and determines the target image according to the second display indication information Is the second image, and controls the display device to display the second image, that is, the image taken by the photographing device in the head-mounted display device.
  • the picture displayed on the head-mounted display device enlarges the image taken by the camera in the head-mounted display device until the displayed picture is switched from the image taken by the movable platform to the head
  • the image taken by the camera in the wearable display device there is a user's field of vision that sees the user from a distance, then keeps approaching the user, and finally becomes the field of view of the camera in the user's head-mounted display device.
  • a possible implementation manner of controlling the display device to display the first image and the second image at the same time is:
  • the first preset display area displays the first image; and the second preset display area of the display device displays the second image.
  • the fifth preset distance range includes the above-mentioned first preset distance range and the second preset distance range.
  • another possible implementation manner for controlling the display device to display the first image and the second image at the same time is: when the movable platform and the When the relative distance of the reference object is less than the preset distance, the area of the display area for displaying the first image in the display device is smaller than the area of the display area for the second image.
  • the display The area of the display area for displaying the first image in the device is equal to the area of the display area for the second image.
  • the area of the display area for displaying the first image in the display device is greater than The area of the display area of the second image.
  • the area of the display area for displaying the first image in the display device is positively correlated with the aforementioned pair of relative distances
  • the area of the display area for displaying the second image in the display device is inversely correlated with the aforementioned pair of relative distances.
  • the relative distance between the movable platform and the reference object when the relative distance between the movable platform and the reference object is within the fifth preset distance range and the movable platform moves away from the reference object, when the relative distance between the movable platform and the reference object is less than the preset distance, and the relative distance
  • the area of the display area for displaying the first image gradually increases and the area of the display area for displaying the second image gradually decreases, and the area of the display area for displaying the first image is smaller than that for The display area where the second image is displayed.
  • the relative distance between the movable platform and the reference object gradually increases to be equal to the preset distance
  • the area of the display area for displaying the first image is equal to the area of the display area for displaying the second image.
  • the area of the display area for displaying the first image gradually increases and the area of the display area for displaying the second image gradually decreases. It is small, and the area of the display area for displaying the first image is larger than the display area for displaying the second image. Therefore, as the movable platform keeps moving away, the picture displayed by the display device on the head-mounted display device gradually enlarges the picture taken by the movable platform.
  • the relative distance between the movable platform and the reference object when the relative distance between the movable platform and the reference object is within the fifth preset distance range and the movable platform moves close to the reference object, when the relative distance between the movable platform and the reference object is greater than the preset distance, and follow the relative distance
  • the area of the display area for displaying the first image gradually decreases and the area of the display area for displaying the second image gradually increases, and the area of the display area for displaying the first image is larger than that for displaying The display area of the second image.
  • the relative distance between the movable platform and the reference object is gradually reduced to be equal to the preset distance, the area of the display area for displaying the first image is equal to the area of the display area for displaying the second image.
  • the area of the display area for displaying the first image gradually decreases and the area of the display area for displaying the second image gradually increases. It is large, and the area of the display area for displaying the first image is smaller than the display area for displaying the second image. Therefore, as the movable platform gets closer, the picture displayed by the display device on the head-mounted display device gradually enlarges the picture of the camera in the head-mounted display device (for example, the picture of the surrounding environment where the user is located).
  • the display indication information is determined according to the working state of the movable platform.
  • the images that users need to watch may also be different, and accordingly, the display instructions may be different.
  • the movable platform may obtain the working status of the movable platform, and obtain the display indication information according to the working status of the movable platform, and then send it to the head-mounted display device The display instructions.
  • a possible implementation manner for the head-mounted display device to obtain the display indication information is to receive the display indication information sent by the movable platform.
  • a possible implementation manner for the head-mounted display device to obtain the display indication information is to obtain the working state of the movable platform, and obtain the display indication information according to the working state.
  • the head-mounted display device acquiring the working status of the movable platform may be: the head-mounted display device receives the working status indication information sent by the movable platform, and the working status indication information is used to indicate the position of the movable platform. In the working state, the head-mounted display device obtains the working state of the movable platform according to the working state indication information.
  • the acquired display instruction information is the first display instruction information, and accordingly, the display device is controlled to display the first image.
  • the working state of the movable platform is the second working state
  • the acquired display instruction information is the second display instruction information, and accordingly, the display device is controlled to display the second image.
  • the working state of the movable platform is the third working state
  • the acquired display instruction information is the third display instruction information, and accordingly, the display device is controlled to simultaneously display the first image and the second image.
  • the acquired display instruction information is the second display instruction information, and accordingly, the display device is controlled to display the second image.
  • the acquired display instruction information is the first display instruction information, and accordingly, the display device is controlled to display the first image.
  • the following uses a movable platform as an example of a drone to illustrate how to obtain the display indication information according to the working status.
  • the working state of the drone is the cruise state
  • the acquired display instruction information is the first display instruction information.
  • the target image is determined to be the first image from the first image and the second image according to the first display instruction information, and then the display device is controlled to display the first image.
  • the working state of the drone is the take-off or landing state
  • the acquired display instruction information is the second display instruction information.
  • the target image is determined to be the second image from the first image and the second image according to the second display instruction information, and then the display device is controlled to display the second image so that the user can observe the image of the surrounding environment where the user is located , To ensure the safety of takeoff and landing of the drone.
  • FIG. 6 is a flowchart of a display control method provided by another embodiment of this application. As shown in FIG. 6, the method of this embodiment can be applied to a head-mounted display device, and the head-mounted display device includes A photographing device and a display device for an image of the surrounding environment of the head-mounted display device.
  • the method of this embodiment may include:
  • S601 Receive a first image collected by the movable platform from the movable platform.
  • S603 Control a display device to display the first image and the second image at the same time.
  • the display device in the head-mounted display device is controlled to simultaneously display the first image and the second image .
  • the user can view the images collected by the movable platform and the images collected by the shooting device in the head-mounted display device through the display device (that is, the surrounding environment of the user). image).
  • the display device is provided on the housing of the head-mounted display device, so that when the user wears the head-mounted display device, the display device can capture an image of the surrounding environment where the user is located, for example, as shown in FIG. 1
  • the position of the display device 1032 in the head-mounted display device 103 shown in this embodiment is not limited to this.
  • the shooting direction of the shooting device deviates from the display direction of the display device, for example, as shown in FIG. 1, so that the shooting direction of the shooting device is the same as the viewing direction of the user's eyes, so that when the display device displays the shooting device shooting The image is as if the user sees it with his own eyes.
  • the first image collected by the movable platform is received from the movable platform, and the second image collected by the shooting device in the head-mounted display device is obtained, and then the display device is controlled to display at the same time.
  • a possible implementation of S603 is: displaying the first image in a first preset display area of the display device, and displaying the first image in a second preset display area of the display device. The second image.
  • the area of the first preset display area is larger than the area of the second preset display area.
  • the area of the first preset display area and the area of the second preset display area are determined according to a relative position parameter between the reference object and the movable platform .
  • the method of this embodiment further includes: acquiring a relative position parameter between the reference object and the movable platform, and determining the first preset display area according to the relative position parameter The area of and the area of the second preset display area, wherein the reference object is the user of the head-mounted display device, the head-mounted display device, or the control terminal of the movable platform.
  • the relative position parameter includes a relative distance, and when the relative distance is within a first preset distance range, the area of the first preset display area is smaller than that of the second preset distance. Set the area of the display area; when the relative distance is within the second preset distance range, the area of the first preset display area is larger than the area of the second preset display area.
  • the display device displays the first image or displays the first image and the second image at the same time, the display device will also display the power and movement parameters of the movable platform (such as speed, altitude, etc.) And other information, which is sent to the head-mounted display device by the mobile platform.
  • the power and movement parameters of the movable platform such as speed, altitude, etc.
  • An embodiment of the present application also provides a computer storage medium.
  • the computer storage medium stores program instructions.
  • the program execution may include some or all of the steps of the display control method in any of the above-mentioned corresponding embodiments.
  • FIG. 7 is a schematic structural diagram of a head-mounted display device provided by an embodiment of the application.
  • the head-mounted display device 700 of this embodiment may include: a photographing device 701, a display device 702, a communication device 703, and Processor 704.
  • the photographing device 701, the display device 702, the communication device 703, and the processor 704 are connected by a communication bus, for example.
  • the communication device 703 is configured to receive the first image collected by the movable platform from the movable platform.
  • the photographing device 701 is used to collect a second image of the surrounding environment of the head-mounted display device 700.
  • the processor 704 is configured to obtain display instruction information, determine a target image from the first image and the second image according to the display instruction information, and control the display device to display the target image.
  • the display device 702 is used to display the target image.
  • the target image is one of a first image, a second image, a first image, and a second image.
  • the display instruction information includes first display instruction information, where the processor 704 is specifically configured to: obtain the first display instruction information, and obtain the first display instruction information from the first display instruction information according to the first display instruction information.
  • the first image is determined from the image and the second image, and the display device 702 is controlled to display the first image.
  • the display instruction information includes second display instruction information, where the processor 704 is specifically configured to: obtain the second display instruction information, and obtain the second display instruction information from the first image and the second image according to the second display instruction information.
  • the second image is determined, and the display device 702 is controlled to display the second image.
  • the display instruction information includes third display instruction information
  • the processor 704 is specifically configured to: obtain third display instruction information, and obtain third display instruction information from the first display instruction information according to the third display instruction information.
  • the first image and the second image are determined in the image and the second image, and the display device 702 is controlled to simultaneously display the first image and the second image.
  • the processor 704 is specifically configured to: control the display device 702 to display the first image in a first preset display area, and display the second image in a second preset display area .
  • the area of the first preset display area is larger than the area of the second preset display area.
  • the area of the first preset display area and the area of the second preset display area are determined according to a relative position parameter between a reference object and the movable platform, wherein the The reference object is the user of the head-mounted display device 700, the head-mounted display device 700, or the control terminal of the movable platform.
  • the processor 704 is further configured to: obtain a relative position parameter between the reference object and the movable platform, and determine the position of the first preset display area according to the relative position parameter. The area and the area of the second preset display area.
  • the relative position parameter includes a relative distance, wherein, when the relative distance is within a first preset distance range, the area of the first preset display area is smaller than that of the second preset display area When the relative distance is in the second preset distance range, the area of the first preset display area is greater than the area of the second preset display area.
  • the communication device 703 is further configured to receive display instruction information sent by the control terminal of the movable platform, where the display instruction information is generated by the control terminal detecting a user's display instruction operation .
  • the processor 704 is specifically configured to obtain the display indication information received by the communication device 703.
  • the processor 704 is specifically configured to: detect a display instruction operation of the user, and obtain display instruction information according to the display instruction operation.
  • the display indication information is determined according to a relative position parameter between the reference object and the movable platform.
  • the reference object is the user of the head-mounted display device 700, the head-mounted display device or the control terminal of the movable platform.
  • the processor 704 is specifically configured to: obtain a relative position parameter between the reference object and the movable platform, and determine the display indication information according to the relative position parameter.
  • the relative position parameter includes a relative distance
  • the display instruction information includes first display instruction information, and the processor 704 is specifically configured to: when the relative distance is within a third preset distance range, obtain first display instruction information, and according to the first display instruction The information determines the first image from the first image and the second image.
  • the display instruction information includes second display instruction information, and the processor 704 is specifically configured to: when the relative distance is within a fourth preset distance range, obtain second display instruction information, and according to the second display instruction The information determines the second image from the first image and the second image.
  • the display indication information includes third display indication information
  • the processor 704 is specifically configured to: when the relative distance is within the fifth preset distance range, obtain the third display indication information, according to The third display instruction information determines the first image and the second image from the first image and the second image.
  • the display indication information is determined according to the working state of the movable platform.
  • the processor 704 is specifically configured to obtain the working state of the movable platform, and obtain the display indication information according to the working state.
  • the movable platform includes a drone.
  • the display instruction information includes first display instruction information
  • the processor 704 is specifically configured to: when the working state is a cruise state, obtain first display instruction information, and obtain first display instruction information from the first display instruction information according to the first display instruction information.
  • the first image is determined from the first image and the second image.
  • the movable platform includes a drone.
  • the display instruction information includes second display instruction information
  • the processor 704 is specifically configured to: when the working state is a take-off or landing state, obtain the second display instruction information, and obtain the second display instruction information from the second display instruction information according to the second display instruction information.
  • the second image is determined from the first image and the second image.
  • the display device 702 is provided on a housing in the head-mounted display device 700.
  • the photographing direction of the photographing device 701 deviates from the display direction of the display device 702.
  • the head-mounted display device of this embodiment can be used to implement the technical solution of the head-mounted display device in the above-mentioned FIG. 3 and the corresponding method embodiment of this application.
  • the implementation principles and technical effects are similar, and will not be repeated here.
  • FIG. 8 is a schematic structural diagram of a head-mounted display device provided by another embodiment of the application.
  • the head-mounted display device 800 of this embodiment may include: a photographing device 801, a display device 802, and a communication device 803 And processor 804.
  • the photographing device 801, the display device 802, the communication device 803, and the processor 804 are connected by a communication bus, for example.
  • the communication device 803 is configured to receive the first image collected by the movable platform from the movable platform.
  • the photographing device 801 is used to collect a second image of the surrounding environment of the head-mounted display device 800.
  • the processor 804 is configured to control the display device 802 to simultaneously display the first image and the second image.
  • the display device 802 is configured to display the first image and the second image at the same time.
  • the processor 804 is specifically configured to: control the display device 802 to display the first image in a first preset display area, and to display the second image in a second preset display area.
  • the area of the first preset display area is larger than the area of the second preset display area.
  • the area of the first preset display area and the area of the second preset display area are determined according to a relative position parameter between a reference object and the movable platform, wherein the The reference object is the user of the head-mounted display device 800, the head-mounted display device 800, or the control terminal of the movable platform.
  • the processor 804 is further configured to: obtain a relative position parameter between the reference object and the movable platform, and determine the position of the first preset display area according to the relative position parameter. The area and the area of the second preset display area.
  • the relative position parameter includes a relative distance, wherein, when the relative distance is within a first preset distance range, the area of the first preset display area is smaller than that of the second preset display area When the relative distance is in the second preset distance range, the area of the first preset display area is greater than the area of the second preset display area.
  • the display device 802 is provided on a housing in the head-mounted display device 800.
  • the photographing direction of the photographing device 801 deviates from the display direction of the display device 802.
  • the head-mounted display device of this embodiment can be used to implement the technical solution of the head-mounted display device in the above-mentioned FIG. 6 and the corresponding method embodiment of this application.
  • the implementation principles and technical effects are similar, and will not be repeated here.
  • FIG. 9 is a schematic structural diagram of a display control system provided by an embodiment of this application.
  • the display control system 900 of this embodiment may include: a movable platform 901 and a head-mounted display device 902.
  • the head-mounted display device 902 can adopt the structure of the embodiment shown in FIG. 7 or FIG. 8, which correspondingly can execute the technical solutions of the head-mounted display device in the foregoing method embodiments, and its implementation principles and technical effects are similar. , I won’t repeat it here.
  • the display control system 900 of this embodiment may further include: a control terminal 903, where the control terminal 903 can execute the terminal control solution involved in the foregoing method embodiment, which will not be repeated here.
  • a person of ordinary skill in the art can understand that all or part of the steps in the above method embodiments can be implemented by a program instructing relevant hardware.
  • the foregoing program can be stored in a computer readable storage medium. When the program is executed, it is executed. Including the steps of the foregoing method embodiment; and the foregoing storage medium includes: read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disks or optical disks, etc., which can store program codes Medium.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Optics & Photonics (AREA)
  • Controls And Circuits For Display Device (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

L'invention concerne un procédé de commande d'affichage et un dispositif de visiocasque (103). Le procédé de commande d'affichage est appliqué au dispositif de visiocasque (103), et le dispositif de visiocasque (103) comprend un appareil photographique (1031) utilisé pour collecter des images d'un environnement ambiant du dispositif de visiocasque (103) et un appareil d'affichage (1032). Le procédé de commande d'affichage consiste à recevoir d'une plate-forme mobile (101) une première image collectée par la plate-forme mobile (101) (S301), à acquérir une seconde image collectée par l'appareil de photographie (1031) dans le dispositif de visiocasque (103) (S302), à acquérir des informations d'instruction d'affichage, et déterminer une image cible parmi la première image et la seconde image selon les informations d'instruction d'affichage (S303), et à commander l'appareil d'affichage (1032) dans le dispositif de visiocasque (103) pour afficher l'image cible (S304). Lorsqu'un utilisateur porte le dispositif d'affichage monté sur la tête (103), l'utilisateur peut visualiser l'image cible au moyen de l'appareil d'affichage (1032). Comme l'image cible est déterminée parmi la première image et la seconde image, l'utilisateur peut visualiser l'environnement ambiant dans lequel se trouve ledit utilisateur, ce qui permet d'éviter le danger et d'assurer la sécurité.
PCT/CN2019/130733 2019-12-31 2019-12-31 Procédé et dispositif de commande d'affichage WO2021134575A1 (fr)

Priority Applications (3)

Application Number Priority Date Filing Date Title
PCT/CN2019/130733 WO2021134575A1 (fr) 2019-12-31 2019-12-31 Procédé et dispositif de commande d'affichage
CN201980094964.8A CN113661429B (zh) 2019-12-31 2019-12-31 显示控制方法和设备
CN202410876485.4A CN118625530A (zh) 2019-12-31 2019-12-31 显示控制方法和设备

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2019/130733 WO2021134575A1 (fr) 2019-12-31 2019-12-31 Procédé et dispositif de commande d'affichage

Publications (1)

Publication Number Publication Date
WO2021134575A1 true WO2021134575A1 (fr) 2021-07-08

Family

ID=76687169

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/130733 WO2021134575A1 (fr) 2019-12-31 2019-12-31 Procédé et dispositif de commande d'affichage

Country Status (2)

Country Link
CN (2) CN118625530A (fr)
WO (1) WO2021134575A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023137758A1 (fr) * 2022-01-24 2023-07-27 深圳市大疆创新科技有限公司 Procédé de commande pour plateforme mobile, dispositif monté sur la tête, système, et support de stockage

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030227542A1 (en) * 2001-12-20 2003-12-11 Xiang Zhang Single-computer real-time stereo augmented reality system
CN204203554U (zh) * 2014-11-05 2015-03-11 昆山优力电能运动科技有限公司 头戴式显示设备
CN105572870A (zh) * 2014-11-05 2016-05-11 优利科技有限公司 头戴式显示设备
CN106445088A (zh) * 2015-08-04 2017-02-22 上海宜维计算机科技有限公司 现实增强的方法及系统
CN108932058A (zh) * 2018-06-29 2018-12-04 联想(北京)有限公司 显示方法、装置及电子设备
CN109041591A (zh) * 2017-09-12 2018-12-18 深圳市大疆创新科技有限公司 图像传输方法、设备、可移动平台、监控设备及系统

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104065911B (zh) * 2013-03-18 2019-01-15 联想(北京)有限公司 显示控制方法及装置

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030227542A1 (en) * 2001-12-20 2003-12-11 Xiang Zhang Single-computer real-time stereo augmented reality system
CN204203554U (zh) * 2014-11-05 2015-03-11 昆山优力电能运动科技有限公司 头戴式显示设备
CN105572870A (zh) * 2014-11-05 2016-05-11 优利科技有限公司 头戴式显示设备
CN106445088A (zh) * 2015-08-04 2017-02-22 上海宜维计算机科技有限公司 现实增强的方法及系统
CN109041591A (zh) * 2017-09-12 2018-12-18 深圳市大疆创新科技有限公司 图像传输方法、设备、可移动平台、监控设备及系统
CN108932058A (zh) * 2018-06-29 2018-12-04 联想(北京)有限公司 显示方法、装置及电子设备

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023137758A1 (fr) * 2022-01-24 2023-07-27 深圳市大疆创新科技有限公司 Procédé de commande pour plateforme mobile, dispositif monté sur la tête, système, et support de stockage

Also Published As

Publication number Publication date
CN113661429A (zh) 2021-11-16
CN118625530A (zh) 2024-09-10
CN113661429B (zh) 2024-07-19

Similar Documents

Publication Publication Date Title
US11235871B2 (en) Control method, control system, and smart glasses for first person view unmanned aerial vehicle flight
US10116922B2 (en) Method and system for automatic 3-D image creation
US11563886B2 (en) Automated eyewear device sharing system
US11320667B2 (en) Automated video capture and composition system
WO2016025962A1 (fr) Dispositif et procédé de communication vidéo tridimensionnelle
US20180176459A1 (en) Method and device for changing focal point of camera
US11195341B1 (en) Augmented reality eyewear with 3D costumes
CN109002248B (zh) Vr场景截图方法、设备及存储介质
US11826635B2 (en) Context-sensitive remote eyewear controller
US20210406542A1 (en) Augmented reality eyewear with mood sharing
WO2023184816A1 (fr) Procédé et appareil d'affichage de bureau en nuage, dispositif et support de stockage
WO2021134575A1 (fr) Procédé et dispositif de commande d'affichage
US20180270424A1 (en) Repositioning camera lenses during capturing of media
CN108259755A (zh) 摄像装置及控制方法
US20230007227A1 (en) Augmented reality eyewear with x-ray effect
US9860480B2 (en) Method for processing information and electronic device
WO2024166171A1 (fr) Terminal d'informations portable et procédé de commande d'affichage pour terminal d'informations portable
EP4352966A1 (fr) Dispositif à étranglement dynamique de transcodage

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19958461

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19958461

Country of ref document: EP

Kind code of ref document: A1