CN112118432B - Display device, holographic projection device, system, method, apparatus, and medium - Google Patents

Display device, holographic projection device, system, method, apparatus, and medium Download PDF

Info

Publication number
CN112118432B
CN112118432B CN201910538834.0A CN201910538834A CN112118432B CN 112118432 B CN112118432 B CN 112118432B CN 201910538834 A CN201910538834 A CN 201910538834A CN 112118432 B CN112118432 B CN 112118432B
Authority
CN
China
Prior art keywords
image
user
holographic projection
displayed
display device
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910538834.0A
Other languages
Chinese (zh)
Other versions
CN112118432A (en
Inventor
刘子君
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
BOE Technology Group Co Ltd
Original Assignee
BOE Technology Group Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by BOE Technology Group Co Ltd filed Critical BOE Technology Group Co Ltd
Priority to CN201910538834.0A priority Critical patent/CN112118432B/en
Publication of CN112118432A publication Critical patent/CN112118432A/en
Application granted granted Critical
Publication of CN112118432B publication Critical patent/CN112118432B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/12—Picture reproducers
    • H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
    • H04N9/3141—Constructional details thereof
    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B35/00—Stereoscopic photography
    • G03B35/18—Stereoscopic photography by simultaneous viewing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/4104—Peripherals receiving signals from specially adapted client devices
    • H04N21/4122—Peripherals receiving signals from specially adapted client devices additional display device, e.g. video projector
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
    • H04N21/4363—Adapting the video stream to a specific local network, e.g. a Bluetooth® network
    • H04N21/43637—Adapting the video stream to a specific local network, e.g. a Bluetooth® network involving a wireless protocol, e.g. Bluetooth, RF or wireless LAN [IEEE 802.11]

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Holo Graphy (AREA)

Abstract

The application provides a display device, a holographic projection device, a system, a method, equipment and a medium, wherein the display device is used for acquiring an image to be displayed selected by a user and a position of the user, and sending the position of the user and the image to be displayed to the holographic projection device, the holographic projection device is used for receiving the image to be displayed selected by the user and the position of the user sent by the display device, respectively generating an initial position and a target position according to the position of the user and the position of the display device stored in advance, moving to the initial position and then moving to the target position, and displaying the image to be displayed in the process of moving from the initial position to the target position. Therefore, the user can see that the image to be displayed can be moved to the display device from the user position, so that the interaction mode between the user and the display device is richer, the interaction interest is improved, the display effect of the image is optimized, and the user experience is improved.

Description

Display device, holographic projection device, system, method, apparatus, and medium
Technical Field
The present disclosure relates to the field of multimedia devices, and more particularly, to a display device, a holographic projection device, a system, a method, a device, and a medium.
Background
In the correlation technique, display device for example the electronic drawing screen can show different paintings according to user's selection, for example, have user experience district in exhibition halls such as museum, fine arts gallery, and the user can show different paintings on the control drawing screen through pronunciation, touch-control or gesture.
However, the interaction mode between the user and the display device realized through voice, touch or gestures is single, the interestingness is poor, the image display effect is poor, and the user experience is not good.
Disclosure of Invention
The embodiment of the application provides a display device, a holographic projection device, a system, a method, equipment and a medium, and is used for solving the technical problems that in the related art, the interaction mode between a user and the display device realized through voice, touch or gestures is single, the interestingness is poor, the display effect of an image is poor, and the user experience is not good.
To this end, an embodiment of an aspect of the present application provides a display apparatus, including a processor, configured to obtain an image to be displayed selected by a user and a location of the user; and the first communication component is used for communicating with the holographic projection device and sending the position of the user and the image to be displayed to the holographic projection device.
As a first possible implementation manner of the first aspect of the present invention, the first communication component is further configured to receive feedback information sent by the holographic projection apparatus; the display device further includes: and the first display is used for displaying the image to be displayed.
As a second possible implementation manner of the first aspect of the present invention, the display device further includes an image acquisition component, configured to acquire images of the user and a scene where the user is located; the processor is specifically configured to obtain a position of a user from an image obtained by the image acquisition component.
The embodiment of the application in another aspect provides a holographic projection device, which comprises a second communication component, a second display component and a second communication component, wherein the second communication component is used for communicating with a display device and receiving an image to be displayed and a position of a user, which are selected by the user and sent by the display device; a memory for storing a position of the display device; a controller for generating an initial position and a target position according to the position of the user and the position of the display device, respectively; the driving component is used for driving the holographic projection device to move to the initial position and then to the target position; and the second display is used for displaying the image to be displayed in the process of moving from the initial position to the target position.
As a first possible implementation manner of the second aspect of the present invention, the second communication component is further configured to: and sending feedback information to the display device.
As a second possible implementation manner of the second aspect of the present invention, the second display is further configured to stop displaying the image to be displayed after the second communication component sends feedback information to the display device, so that the display device starts displaying the image to be displayed.
As a third possible implementation manner of the second aspect of the present invention, the second display includes a holographic projection component, configured to perform holographic projection on the image to be displayed.
In another embodiment of the present application, a holographic projection system is provided, which includes the display device of the first aspect and the holographic projection device of the second aspect.
As a first possible implementation manner of the third aspect of the present invention, the holographic projection system further includes: the semi-transparent and semi-reflective film is arranged in front of the holographic projection device and forms an angle of 45 degrees with the horizontal plane, wherein the transmissivity and the reflectivity of the semi-transparent and semi-reflective film are both 50%.
In another aspect, an embodiment of the present application provides a holographic projection method, including the following steps:
the method comprises the steps that a display device obtains an image to be displayed selected by a user and the position of the user;
and the display device sends the image to be displayed selected by the user and the position of the user to the holographic projection device.
As a first possible implementation manner of the fourth aspect of the present invention, the method further includes:
receiving feedback information sent by the holographic projection device;
and displaying the image to be displayed.
As a first possible implementation manner of the fourth aspect of the present invention, the acquiring, by the display apparatus, an image to be displayed selected by a user and a position of the user includes:
and the display device acquires the position of the user according to the image of the user and the scene thereof acquired by the image acquisition assembly.
In another aspect, an embodiment of the present application provides a holographic projection method, including the following steps:
the holographic projection device receives an image to be displayed selected by a user and sent by a display device and the position of the user;
generating an initial position and a target position according to the position of the user and the position of the display device stored in advance respectively;
and moving to the target position after moving to the initial position, and displaying the image to be displayed in the process of moving from the initial position to the target position.
As a first possible implementation manner of the fifth aspect of the present invention, after the moving to the target position, the method further includes: and sending feedback information to the display device, and stopping displaying the image to be displayed so as to enable the display device to start displaying the image to be displayed.
As a second possible implementation manner of the fifth aspect of the present invention, the displaying the image to be displayed includes:
and the holographic projection device performs holographic projection on the image to be displayed through a holographic projection component.
In yet another embodiment of the present application, an electronic device is provided, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor executes the computer program to implement the holographic projection method according to the fourth embodiment or the fifth embodiment.
A further aspect of the present application provides a computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, is configured to implement the holographic projection method according to the fourth or fifth aspect of the present application.
The technical scheme disclosed in the application has the following beneficial effects:
the holographic projection device moves according to the position of the user acquired by the display device and the position of the display device, and displays the image to be displayed in the moving process, so that the user can see that the image to be displayed can move to the display device from the position of the user, the interaction mode between the user and the display device is richer, the interaction interest is improved, the display effect of the image is optimized, and the user experience is improved.
Additional aspects and advantages of the present application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the present application.
Drawings
The above and/or additional aspects and advantages of the present application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
fig. 1 is a schematic view of a related art stage holographical projection apparatus according to an embodiment of the present application;
FIG. 2 is a schematic diagram of a holographic projection system according to an embodiment of the present application;
FIG. 3 is a schematic structural diagram of a display device according to an embodiment of the present application;
FIG. 4 is a schematic diagram of a holographic projection apparatus according to an embodiment of the present application;
FIG. 5 is a schematic diagram of a holographic projection system according to an embodiment of the present application;
FIG. 6 is an exemplary diagram of a holographic projection system of an embodiment of the present application;
FIG. 7 is an exemplary illustration of an arrangement of drive assemblies according to one embodiment of the present application;
FIG. 8 is a schematic diagram of a holographic projection system according to another embodiment of the present application;
FIG. 9 is a schematic flow chart diagram of a holographic projection method according to an embodiment of the present application;
FIG. 10 is a schematic flow chart of a holographic projection method provided by another embodiment of the present application;
fig. 11 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
Reference will now be made in detail to embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the accompanying drawings are illustrative and intended to explain the present application and should not be construed as limiting the present application.
The embodiments in the present application provide a display device, a holographic projection system, a holographic projection method, an electronic device, and a computer-readable storage medium, mainly for solving the technical problems in the related art that an interaction manner between a user and the display device is single, the user is not interesting, the image display effect is poor, and the user experience is not good, which are realized through voice, touch, or gesture.
The holographic projection system comprises a display device and a holographic projection device, wherein the display device can send the position of a user and an image to be displayed to the holographic projection device after acquiring the image to be displayed selected by the user and the position of the user, and then the holographic projection device can generate an initial position according to the position of the user, generate a target position according to the position of the display device stored in advance, move to the initial position and then move to the target position, and display the image to be displayed in the process of moving from the initial position to the target position. Therefore, the holographic projection device moves according to the position of the user and the position of the display device, and displays the image to be displayed in the moving process, so that the user can see that the image to be displayed can move to the display device from the position of the user, the interaction mode between the user and the display device is richer, the interaction interest is improved, the display effect of the image is optimized, and the user experience is improved.
The principle of the pepper's ghost utilized in the embodiment of the present application will be described first with reference to a schematic diagram of a related art apparatus for performing stage holographic projection shown in fig. 1.
As shown in fig. 1, a film is disposed at a certain angle in the center of the device, and if the audience looks at the stage from the direction shown by the arrow in fig. 1, the light emitted from the stage is refracted when entering the film and leaving the film to the audience, so that the audience can see the stage through the film, when the stage holographic projection is performed, the image can be played by using the projector installed above the device, so that part of the light of the image is reflected by the film and can also enter the sight of the audience, and then the audience can see the light of the stage and the image at the same time, thereby forming the effect that the image seems to be played on the stage when the audience looks, i.e. showing a pepper's ghost.
A display device, a holographic projection system, a holographic projection method, an electronic apparatus, and a computer-readable storage medium according to embodiments of the present application are described below with reference to the accompanying drawings.
First, referring to fig. 2, a holographic projection system provided in an embodiment of the present application will be specifically described.
FIG. 2 is a schematic diagram of a holographic projection system according to an embodiment of the present application.
As shown in fig. 2, the holographic projection system of the present application may include: a display device 1 and a holographic projection device 2.
First, the display device 1 will be explained.
As shown in fig. 3, the display device 1 includes a processor 11 for acquiring an image to be displayed selected by a user and a position of the user; the first communication component 12 is used for communicating with the holographic projection device and sending the position of the user and the image to be displayed to the holographic projection device.
Specifically, the display device 1 in the holographic projection system provided in the embodiment of the present application may be any hardware device having a display screen, such as a drawing screen.
In specific implementation, the display device 1 may be connected to the mobile terminal through any Wireless network such as bluetooth, wiFi (Wireless Fidelity, wiFi for short), so that a user may select an image to be displayed, that is, an image to be displayed, in the mobile terminal through touch control, voice or any other manner, and trigger projection of the image to be displayed, so that after the mobile terminal obtains the image to be displayed according to the selection of the user, the image to be displayed may be sent to the display device 1 through the Wireless network, and the processor 11 configured in the display device 1 may obtain the image to be displayed selected by the user.
The mobile terminal can be a hardware device with various operating systems, touch screens and/or display screens, such as a smart phone, a tablet computer, a personal digital assistant and a wearable device.
Specifically, the processor 11 configured in the display apparatus 1 may acquire the position of the user in various ways.
In an exemplary embodiment, as shown in fig. 3, an image capturing component 13, such as a camera, may be further configured in the display device 1, and is used to capture an image of a user and a scene where the user is located, so that after the processor 11 obtains an image to be displayed, the image capturing component 13 may be controlled to capture the image towards the direction where the user is located, then the image capturing component 13 may send the captured image to the processor 11, and then the processor 11 processes the obtained image of the user and the scene where the user is located, obtains the position of the user, and sends the position of the user and the image to be displayed to the holographic projection apparatus 2 through the first communication component 12.
The processor 11 may obtain the position of the user according to the image acquired by the image acquisition component by adopting any method of determining the position of the user according to the image of the scene where the user is located in the prior art, which is not limited in the present application.
After receiving the image to be displayed selected by the user and the position of the user, which are sent by the display device 1, the holographic projection device 2 can generate an initial position according to the position of the user, generate a target position according to the position of the display device, move to the initial position and then move to the target position, and display the image to be displayed in the process of moving from the initial position to the target position.
In an exemplary embodiment, as shown in fig. 4, the holographic projection device 2 may include a second communication component 21, configured to communicate with the display device 1, and receive the image to be displayed selected by the user and the position of the user, which are sent by the display device 1; a memory 22 for storing the position of the display apparatus 1; a controller 23 for generating an initial position and a target position according to a position of a user and a position of a display device, respectively; the driving component 24 is used for driving the holographic projection device 2 to move to the initial position and then to the target position; and a second display 25 for displaying the image to be displayed in the process of moving from the initial position to the target position.
The holographic projection device 2 may be any hardware device capable of projecting a picture or an image.
It should be noted that, in the embodiment of the present application, the image to be displayed, which is displayed by the holographic projection device 2 and is seen by the user, may be a virtual image implemented by the pepper's ghost principle. That is, the second display 25 may comprise a holographic projection component for holographic projection of the image to be displayed. As shown in fig. 5, a transflective film may be disposed in front of the holographic projection apparatus 2, so that the holographic projection apparatus 2 is located below the transflective film, and the user is located at a side a of the transflective film, so that when the holographic projection apparatus 2 acquires an image to be displayed selected by the user and holographically projects the image to be displayed, the light of the image to be displayed is reflected to the eyes of the user through the transflective film, so that the user can see a virtual image at the other side of the transflective film, i.e., B side, by using the principle of pepper's ghost.
The angle between the semi-transparent and semi-reflective film and the horizontal plane can be 45 degrees, and the transmittance and the reflectance of the semi-transparent and semi-reflective film can be 50 percent, so that a user can see the display device 1 on the side B clearly through the semi-transparent and semi-reflective film on the side a shown in fig. 5, and can see the virtual image formed on the side B by the reflection of the semi-transparent and semi-reflective film on the image to be displayed by the holographic projection device 2 clearly.
It is understood that, in the embodiment of the present application, when the user is made to see that the image to be displayed can be moved from the user position to the position of the display device 1, which is substantially when the user is made to face the transflective film, the virtual image of the image to be displayed formed on the B side of the transflective film can be moved from the position where the user is directly facing to the position where the display device 1 is located.
That is, the principle of generating the target position is: when the holographic projection device 2 displays the image to be displayed at the target position, a virtual image corresponding to the image to be displayed by the holographic projection device 2 appears to the user to be located at the display device 1.
Specifically, the memory 22 in the holographic projection apparatus 2 may store the position of the display apparatus 1 in advance, so that after the second communication module 21 receives the image to be displayed selected by the user and the user position sent by the display apparatus 1, the controller 23 may generate the target position according to the generation principle of the target position according to the position of the display apparatus 1.
In addition, in order to make the virtual image of the image to be displayed appear to the user to move from the position where the user is facing the display device 1 during the movement of the holographic projection device 2, it is necessary that the virtual image of the image to be displayed is positioned right in front of the user when the holographic projection device 2 starts displaying the image to be displayed, and therefore, it is also necessary that the holographic projection device 2 generates a home position in advance according to the position of the user.
Wherein, the generation principle of the initial position is as follows: when the holographic projection device 2 displays the image to be displayed at the initial position, the virtual image corresponding to the image to be displayed by the holographic projection device 2 appears to be located right in front of the user by the user.
Therefore, after the holographic projection device 2 obtains the position of the user through the second communication component 21, the controller 23 may generate the starting position according to the generation principle of the starting position, and generate the moving path according to the starting position and the target position, so that the driving component 24 may drive the holographic projection device 2 to move from the current position to the starting position first, and then move from the starting position to the target position according to the moving path, and the second display 25 displays the image to be displayed in the process of moving from the starting position to the target position. Thereby, the virtual image of the image to be displayed can be made to appear to the user from the position at which the user is facing to the position of the display device 1.
For example, assuming that the holographic projection device 2 is currently located at the position 1 as shown in fig. 6, after the holographic projection device 2 acquires the position of the user through the second communication component 21, the controller 23 may determine the position 2 according to the principle of generating the start position, and determine the position 3 according to the principle of generating the target position according to the stored position of the display device 1. When the holographic projection device 2 displays the image to be displayed at the position 2, the virtual image of the image to be displayed is positioned right in front of the user when viewed by the user, and when the holographic projection device 2 displays the image to be displayed at the position 3, the virtual image of the image to be displayed is positioned on the display device 2 when viewed by the user. So that the driving component 24 in the holographic projection device 2 can drive the holographic projection device 2 to move from the current position 1 to the position 2, then the second display 25 starts to display the image to be displayed, and then the driving component 24 drives the holographic projection device 2 to move to the position 3, so that the virtual image of the image to be displayed can be seen by the user to move from the front of the user to the position of the display device 1.
In an exemplary embodiment, the driving assembly 24 of the holographic projection apparatus 2 may be a motor, so that the movement of the holographic projection apparatus 2 is achieved using the motor and a slide rail.
For example, as shown in fig. 7, a slide rail may be provided in the x, y, and z directions, the hologram projector 2 may be provided on the slide rail, and the hologram projector 2 may be slid on the slide rail, so that after the controller 23 generates the start position and the target position, the hologram projector 2 may be driven by a motor to move on the slide rail to the start position and then to the target position.
As can be seen from the above examples, when the holographic projection device 2 moves in the horizontal direction, the virtual image of the image to be displayed by the holographic projection device 2 moves in the vertical direction, that is, the moving direction of the holographic projection device 2 is perpendicular to the moving direction of the virtual image of the image to be displayed by the holographic projection device. Because holographic projection device 2 generally is greater than the displacement range of vertical direction at the displacement range of horizontal direction, and the virtual image of waiting to show the image need remove great range at the horizontal direction, consequently, in this application a possible implementation form, can be as shown in fig. 8, set up the speculum in holographic projection device 2's one end, thereby make holographic projection device 2 when the horizontal direction removes, the virtual image of its demonstration waiting to show the image also can remove at the horizontal direction, thereby the virtual image that makes to show the image obtains bigger displacement range at the horizontal direction.
In addition, in order to obtain a better display effect for the image to be displayed, in the embodiment of the present application, the holographic projection apparatus 2 may further process the image to be displayed. For example, the image to be displayed may be rendered, so that the rendered image shows that the image to be displayed is a front view, and a stereoscopic view of the frame and the bottom surface is added, and has an effect of a certain inclination angle.
In addition, in a specific implementation, in order to make the virtual image of the image to be displayed appear to the user as if the virtual image is moved from the position of the user to the display device 1 and then can be displayed on the display device 1, in the embodiment of the present application, the display device 1 may further display the image to be displayed after the holographic projection device 2 is moved to the target position, and at the same time, the holographic projection device 2 stops displaying the image to be displayed.
That is, as shown in fig. 3, the holographic projection apparatus may further include a first display 14, after the holographic projection apparatus 2 moves to the target position, the feedback information may be sent to the display apparatus 1 through the second communication component 21, then the second display 25 stops displaying the image to be displayed, and after the display apparatus 1 receives the feedback information sent by the holographic projection apparatus 2 through the first communication component 11, the first display 14 displays the image to be displayed again.
The holographic projection system comprises a display device and a holographic projection device, wherein the display device comprises a processor and a processor, wherein the processor is used for acquiring an image to be displayed selected by a user and the position of the user; the first communication component is used for communicating with the holographic projection device and sending the position of a user and an image to be displayed to the holographic projection device; the holographic projection device comprises a second communication component, a first communication component and a second communication component, wherein the second communication component is used for communicating with the display device and receiving the image to be displayed and the position of the user, which are sent by the display device and selected by the user; a memory for storing a position of the display device; a controller for generating an initial position and a target position according to a position of a user and a position of a display device, respectively; the driving component is used for driving the holographic projection device to move to the initial position and then move to the target position; and the second display is used for displaying the image to be displayed in the process of moving from the initial position to the target position. Therefore, the holographic projection device moves according to the position of the user and the position of the display device, and displays the image to be displayed in the moving process, so that the user can see that the image to be displayed can move to the display device from the position of the user, the interaction mode between the user and the display device is richer, the interaction interest is improved, the display effect of the image is optimized, and the user experience is improved.
The following describes a display device provided in an embodiment of the present application.
As shown in fig. 3, a display device 1 provided in an embodiment of the present application includes:
the processor 11 is configured to obtain an image to be displayed selected by a user and a position of the user;
the first communication component 12 is used for communicating with the holographic projection device and sending the position of the user and the image to be displayed to the holographic projection device.
The display device 1 may be any hardware device having a display screen, such as a drawing screen.
Specifically, the display device 1 may be connected to the mobile terminal through any wireless network such as bluetooth or WiFi, so that a user may select an image to be displayed in the display device 1 in the mobile terminal through touch control, voice or any other mode, that is, the image to be displayed, and trigger projection of the image to be displayed, so that after the mobile terminal obtains the image to be displayed according to the selection of the user, the image to be displayed may be sent to the display device 1, and thus the processor 11 configured in the display device 1 may obtain the image to be displayed selected by the user.
The mobile terminal can be a hardware device with various operating systems, touch screens and/or display screens, such as a smart phone, a tablet computer, a personal digital assistant and a wearable device.
Specifically, the processor 11 configured in the display apparatus 1 may acquire the position of the user in various ways.
In an exemplary embodiment, as shown in fig. 3, an image capturing component 13, such as a camera, may be further configured in the display device 1, and is used to capture an image of a user and a scene where the user is located, so that after the processor 11 obtains an image to be displayed, the image capturing component 13 may be controlled to capture the image towards the direction where the user is located, then the image capturing component 13 may send the captured image to the processor 11, and then the processor 11 processes the obtained image of the user and the scene where the user is located, obtains the position of the user, and sends the position of the user and the image to be displayed to the holographic projection apparatus 2 through the first communication component 12.
The processor 11 may obtain the position of the user according to the image acquired by the image acquisition component by adopting any method of determining the position of the user according to the image of the scene where the user is located in the prior art, which is not limited in the present application.
In addition, in a specific implementation, in order to make the virtual image of the image to be displayed appear to the user after moving from the position of the user to the display device 1, the display device 1 may also display the image to be displayed after the holographic projection device 2 moves to the target position in the embodiment of the present application.
Specifically, after the holographic projection device 2 moves to the target position, the second communication component 21 may send feedback information to the display device 1, and then the second display 25 stops displaying the image to be displayed, so that the first display 14 displays the image to be displayed again after the display device 1 receives the feedback information sent by the holographic projection device 2 through the first communication component 11.
Namely, the first communication component 11 is further configured to receive feedback information sent by the holographic projection apparatus;
the display device further includes: a first display 14 for displaying an image to be displayed.
It should be noted that the foregoing explanation of the embodiment of the holographic projection system is also applicable to the display device of this embodiment, and is not repeated here.
According to the display device provided by the embodiment of the application, the processor can acquire the to-be-displayed image selected by the user and the position of the user, and then the position of the user and the to-be-displayed image are sent to the holographic projection device through the first communication assembly, so that the holographic projection device moves according to the position of the user and the position of the display device stored in advance, and in the moving process, the to-be-displayed image is displayed, so that the to-be-displayed image can be moved to the display device from the position of the user when viewed by the user, further, the interaction mode between the user and the display device is richer, the interaction interest is improved, the display effect of the image is optimized, and the user experience is improved.
The following describes a holographic projection apparatus provided in an embodiment of the present application.
As shown in fig. 4, a holographic projection apparatus 2 provided in the embodiment of the present application includes:
the second communication component 21 is used for communicating with the display device and receiving the image to be displayed selected by the user and the position of the user, which are sent by the display device;
a memory 22 for storing the position of the display device;
a controller 23 for generating an initial position and a target position according to a position of a user and a position of a display device, respectively;
the driving component 24 is used for driving the holographic projection device to move to the initial position and then to the target position;
and a second display 25 for displaying the image to be displayed in the process of moving from the initial position to the target position.
The holographic projection device 2 may be any hardware device capable of projecting a picture or an image.
In the embodiment of the present application, the image to be displayed by the holographic projection device 2 viewed by the user may be a virtual image implemented by the pepper's ghost principle using a transflective film.
The generation principle of the target position is as follows: when the holographic projection device 2 displays the image to be displayed at the target position, a virtual image corresponding to the image to be displayed by the holographic projection device 2 appears to the user to be located at the display device 1.
Specifically, the memory 22 of the holographic projection apparatus 2 may store the position of the display apparatus 1 in advance, so that after the second communication component 21 receives the image to be displayed selected by the user and the position of the user, which are sent by the display apparatus 1, the controller 23 may generate the target position according to the position of the display apparatus 1 and the generation principle of the target position.
In addition, in order to make the virtual image of the image to be displayed appear to the user to move from the position where the user is facing the display device 1 during the movement of the holographic projection device 2, it is necessary that the virtual image of the image to be displayed is positioned right in front of the user when the holographic projection device 2 starts displaying the image to be displayed, and therefore, it is also necessary that the holographic projection device 2 generates a home position in advance according to the position of the user.
Wherein, the generation principle of the initial position is as follows: when the holographic projection device 2 displays the image to be displayed at the initial position, the virtual image corresponding to the image to be displayed by the holographic projection device 2 appears to the user to be located right in front of the user.
Therefore, after the holographic projection device 2 acquires the position of the user through the second communication component 21, the controller 23 may generate the starting position according to the generation principle of the starting position, and generate the moving path according to the starting position and the target position, so that the driving component may drive the holographic projection device 2 to move from the current position to the starting position first, and then move from the starting position to the target position according to the moving path, and the second display displays the image to be displayed in the process of moving from the starting position to the target position. Thereby, the virtual image of the image to be displayed can be made to appear to the user from the position directly facing the user to the position of the display apparatus 1.
In an exemplary embodiment, the driving assembly 24 of the holographic projection device 2 may be a motor, so that the movement of the holographic projection device 2 is achieved using the motor and a slide rail.
In addition, in order to obtain a better display effect for the image to be displayed, in the embodiment of the present application, the holographic projection apparatus 2 may further process the image to be displayed. For example, the image to be displayed may be rendered, so that the rendered image shows that the image to be displayed is a front view, and a stereoscopic view of the frame and the bottom surface is added, and has an effect of a certain inclination angle.
In addition, in a specific implementation, in order to make the virtual image of the image to be displayed appear to the user after moving from the position of the user to the display device 1, the display device 1 may also display the image to be displayed after the holographic projection device 2 moves to the target position in the embodiment of the present application.
Accordingly, after the holographic projection device 2 moves to the target position, the feedback information can be sent to the display device 1 through the second communication component 21, and then the second display 25 stops displaying the image to be displayed, so that the display device 1 starts displaying the image to be displayed after receiving the feedback information sent by the holographic projection device 2.
I.e. the second communication component 21, is further configured to: and sending the feedback information to the display device.
And the second display 25 is further configured to stop displaying the image to be displayed after the second communication component sends the feedback information to the display device, so that the display device 1 starts displaying the image to be displayed.
It should be noted that the foregoing explanation of the embodiment of the holographic projection system is also applicable to the holographic projection apparatus of the embodiment, and is not repeated herein.
The holographic projection device provided by the embodiment of the application comprises a second communication component, a second communication component and a second communication component, wherein the second communication component is used for communicating with a display device and receiving an image to be displayed and a position of a user, which are sent by the display device and selected by the user; a memory for storing a position of the display device; a controller for generating an initial position and a target position according to a position of a user and a position of a display device, respectively; the driving component is used for driving the holographic projection device to move to the initial position and then move to the target position; and the second display is used for displaying the image to be displayed in the process of moving from the initial position to the target position. The holographic projection device moves according to the position of the display device and the position of the user sent by the display device, and displays the image to be displayed in the moving process, so that the user can see that the image to be displayed can move to the display device from the position of the user, the interaction mode between the user and the display device is richer, the interaction interest is improved, the display effect of the image is optimized, and the user experience is improved.
The holographic projection method provided by the embodiment of the present application is described below with reference to fig. 9.
Fig. 9 is a schematic flowchart of a holographic projection method according to an embodiment of the present application.
As shown in fig. 9, a holographic projection method provided in an embodiment of the present application includes:
step 101, a display device obtains an image to be displayed selected by a user and a position of the user.
And 102, the display device sends the image to be displayed selected by the user and the position of the user to the holographic projection device.
The holographic projection method provided by the embodiment of the application can be executed by the display device provided by the embodiment of the application. The display device may be any hardware device having a display screen, such as a painted screen.
Specifically, the display device may be connected to the mobile terminal through any Wireless network such as bluetooth, wiFi (Wireless Fidelity, wiFi for short), so that the user may select an image to be displayed in the display device, that is, an image to be displayed, in the mobile terminal through touch control, voice or any other manner, and trigger projection of the image to be displayed, so that the mobile terminal may send the image to be displayed to the display device after acquiring the image to be displayed according to the selection of the user.
In addition, the display device may acquire the position of the user in various ways.
In an exemplary embodiment, the display device may be configured with a processor and an image capturing component, such as a camera, for capturing images of a user and a scene where the user is located, so that after an image to be displayed is obtained, the processor may control the image capturing component to capture images in a direction where the user is located, then the image capturing component may send the captured images to the processor, and then the processor processes the obtained images of the user and the scene where the user is located, so as to obtain a position of the user.
That is, step 101 may be implemented by:
and the display device acquires the position of the user according to the image of the user and the scene thereof acquired by the image acquisition assembly.
In addition, the display device may include a first communication component to communicate with the holographic projection device, so that the display device may send the position of the user and the image to be displayed to the holographic projection device through the first communication component, so that the holographic projection device moves according to the position of the user and the position of the display device stored in advance, and displays the image to be displayed in the moving process.
In addition, in a specific implementation, in order to make the virtual image of the image to be displayed appear to the user after moving from the position of the user to the display device, in this embodiment of the application, the holographic projection device may further send feedback information to the display device after moving to the target position, and stop displaying the image to be displayed, and then start displaying the image to be displayed by the display device after receiving the feedback information sent by the holographic projection device.
That is, the holographic projection method provided in the embodiment of the present application may further include: receiving feedback information sent by the holographic projection device; and displaying the image to be displayed.
It should be noted that the foregoing explanation of the embodiments of the holographic projection system and the display device is also applicable to the holographic projection method provided in the embodiments of the present application, and is not repeated herein.
According to the holographic projection method, after the display device obtains the image to be displayed selected by the user and the position of the user, the image to be displayed selected by the user and the position of the user can be sent to the holographic projection device, so that the holographic projection device can move according to the position of the user and the position of the display device stored in advance, the image to be displayed is displayed in the moving process, the image to be displayed can be moved to the display device from the position of the user when seen by the user, the interaction mode between the user and the display device is enriched, the interaction interest is improved, the display effect of the image is optimized, and the user experience is improved.
A holographic projection method according to another embodiment of the present application will be described with reference to fig. 10.
Fig. 10 is a schematic flow chart of a holographic projection method according to another embodiment of the present application.
As shown in fig. 10, a holographic projection method provided in an embodiment of the present application includes:
step 201, the holographic projection device receives the image to be displayed and the position of the user, which are selected by the user and sent by the display device.
The holographic projection method provided by the embodiment of the application can be executed by the holographic projection device provided by the embodiment of the application. The holographic projection device may be any hardware device capable of projecting a picture or an image.
Specifically, the holographic projection device may include a second communication component, so that the second communication component may implement communication with the display device, and receive the position of the user and the image to be displayed, which are sent by the display device.
Step 202, generating an initial position and a target position according to the position of the user and the position of the display device stored in advance.
And 203, moving to the initial position and then moving to the target position, and displaying the image to be displayed in the process of moving from the initial position to the target position.
In the embodiment of the present application, the image to be displayed by the holographic projection device viewed by the user may be a virtual image implemented by the pepper's ghost principle by using a transflective film.
Accordingly, in this embodiment, the holographic projection apparatus may include a holographic projection component, and the step 203 of displaying the image to be displayed may include: the holographic projection device performs holographic projection on an image to be displayed through the holographic projection component.
The generation principle of the target position is as follows: when the holographic projection device displays the image to be displayed at the target position, the virtual image corresponding to the image to be displayed by the holographic projection device is positioned at the display device when seen by a user.
Specifically, the holographic projection apparatus may store the position of the display apparatus in advance, so that the target position may be generated according to the generation principle of the target position according to the position of the display apparatus.
In addition, in order to make the virtual image of the image to be displayed appear to the user to move from the position where the user is facing the display device during the movement of the holographic projection device, the holographic projection device is also required to generate a home position in accordance with the position of the user since the virtual image of the image to be displayed is positioned right in front of the user when the holographic projection device starts displaying the image to be displayed. Wherein, the generation principle of the initial position is as follows: when the holographic projection device displays the image to be displayed at the initial position, the virtual image corresponding to the image to be displayed by the holographic projection device is positioned right in front of the user. Therefore, after the holographic projection device obtains the position of a user, the holographic projection device can generate the initial position according to the generation principle of the initial position, and generate the moving path according to the initial position and the target position, so that after the holographic projection device moves from the current position to the initial position, the holographic projection device moves from the initial position to the target position according to the moving path, and the image to be displayed is displayed in the process of moving from the initial position to the target position. Thus, the virtual image of the image to be displayed can be made to appear to the user from a position at which the user is facing the display device to a position of the display device.
In an exemplary embodiment, a driving assembly, such as a motor, may be included in the holographic projection apparatus, so that the movement of the holographic projection apparatus is achieved using the motor and the slide rail.
In addition, in order to enable the image to be displayed to obtain a better display effect, in the embodiment of the application, the holographic projection device may further process the image to be displayed. For example, the image to be displayed may be rendered, so that the rendered image shows that the image to be displayed is a front view, and a stereoscopic view of the frame and the bottom surface is added, and has an effect of a certain inclination angle.
In addition, in a specific implementation, in order to make the virtual image of the image to be displayed appear to the user to be displayed on the display device after the virtual image is moved from the position of the user to the display device, in the embodiment of the present application, the display device may further display the image to be displayed after the holographic projection device is moved to the target position.
Correspondingly, after the holographic projection device moves to the target position, the holographic projection device can send feedback information to the display device, and stop displaying the image to be displayed, so that the display device starts displaying the image to be displayed after receiving the feedback information sent by the holographic projection device.
It should be noted that the foregoing explanation of the embodiments of the holographic projection system and the holographic projection apparatus is also applicable to the holographic projection method provided in the embodiments of the present application, and will not be repeated herein.
According to the holographic projection method provided by the embodiment of the application, after receiving the image to be displayed selected by the user and the position of the user, which are sent by the display device, the holographic projection device can generate the initial position and the target position according to the position of the user and the position of the display device stored in advance, then move to the initial position and then move to the target position, and display the image to be displayed in the process of moving from the initial position to the target position. Therefore, the image to be displayed can be moved to the display device from the position of the user, so that the interaction mode between the user and the display device is richer, the interaction interest is improved, the display effect of the image is optimized, and the user experience is improved.
In order to implement the above embodiments, the present application further provides an electronic device.
Fig. 11 is a schematic structural diagram of an electronic device according to an embodiment of the present application. The electronic device shown in fig. 11 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present application.
As shown in fig. 11, the electronic device 200 includes: a memory 210, a processor 220 and a computer program stored in the memory 210 and executable on the processor 220, wherein the processor 220 implements the holographic projection method described in the fourth or fifth aspect embodiment when executing the program.
It should be noted that, for the implementation process and the technical principle of the electronic device of this embodiment, reference is made to the foregoing explanation of the display device of the second embodiment and the explanation of the holographic projection device of the third embodiment, and details are not described here again.
The electronic equipment provided by the embodiment of the application can be the display device or the holographic projection device in the embodiment, the obtained position of the user and the image to be displayed selected by the user are sent to the holographic projection device through the display device, the image to be displayed is moved and displayed in the moving process through the holographic projection device according to the position of the user and the position of the display device stored in advance, the image to be displayed can be moved to the screen from the position of the user in view of the user, the interaction mode of the user and the screen is enriched, the interaction interest is improved, and the user experience is improved.
In order to implement the foregoing embodiments, the present application further provides a computer-readable storage medium.
Wherein the computer-readable storage medium has stored thereon a computer program which, when being executed by a processor, is adapted to carry out the holographic projection method as described in the fourth or fifth embodiment.
To achieve the above embodiments, the present application further proposes a computer program, which when executed by a processor, executes the holographic projection method according to the fourth aspect or the fifth aspect.
In the description of the present specification, reference to the description of "one embodiment," "some embodiments," "an example," "a specific example," or "some examples" or the like means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application.
Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or to implicitly indicate the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and the scope of the preferred embodiments of the present application includes other implementations in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present application.
It should be understood that portions of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
It will be understood by those skilled in the art that all or part of the steps carried by the method for implementing the above embodiments may be implemented by hardware related to instructions of a program, which may be stored in a computer readable storage medium, and when the program is executed, the program includes one or a combination of the steps of the method embodiments.
The storage medium mentioned above may be a read-only memory, a magnetic or optical disk, etc. While embodiments of the present application have been shown and described above, it will be understood that the above embodiments are exemplary and should not be construed as limiting the present application and that changes, modifications, substitutions and alterations in the above embodiments may be made by those of ordinary skill in the art within the scope of the present application.

Claims (17)

1. A display device, comprising:
the processor is used for acquiring an image to be displayed selected by a user and the position of the user;
the first communication component is used for communicating with the holographic projection device and sending the position of the user and the image to be displayed to the holographic projection device;
the holographic projection device can generate an initial position according to the position of the user, generate a target position according to the position of the display device stored in advance, move to the initial position and then move to the target position, and display the image to be displayed in the process of moving from the initial position to the target position.
2. The display device of claim 1, wherein the first communication component is further configured to receive feedback information sent by the holographic projection device;
the display device further includes:
and the first display is used for displaying the image to be displayed.
3. The display device of claim 1, further comprising:
the image acquisition component is used for acquiring images of the user and the scene where the user is located;
the processor is specifically configured to acquire a position of a user according to the image acquired by the image acquisition component.
4. A holographic projection device, comprising:
the second communication assembly is used for communicating with a display device and receiving the image to be displayed selected by the user and the position of the user, which are sent by the display device;
a memory for storing a position of the display device;
a controller for generating an initial position and a target position according to the position of the user and the position of the display device, respectively;
the driving component is used for driving the holographic projection device to move to the initial position and then to the target position;
and the second display is used for displaying the image to be displayed in the process of moving from the initial position to the target position.
5. The holographic projection device of claim 4, wherein the second communication component is further to:
and sending feedback information to the display device.
6. The holographic projection device of claim 5, wherein the second display is further configured to stop displaying the image to be displayed after the second communication component sends feedback information to the display device, so that the display device starts displaying the image to be displayed.
7. The holographic projection device of claim 4, in which the second display comprises a holographic projection component for holographic projection of the image to be displayed.
8. Holographic projection system, comprising a display device according to any of claims 1-3 and a holographic projection device according to any of claims 4-7.
9. The holographic projection system of claim 8, wherein the holographic projection system further comprises: the semi-transparent and semi-reflective film is arranged in front of the holographic projection device and forms an angle of 45 degrees with the horizontal plane, wherein the transmissivity and the reflectivity of the semi-transparent and semi-reflective film are both 50%.
10. A holographic projection method, comprising the steps of:
the method comprises the steps that a display device obtains an image to be displayed selected by a user and the position of the user;
the display device sends the image to be displayed selected by the user and the position of the user to the holographic projection device;
the holographic projection device can generate an initial position according to the position of the user, generate a target position according to the position of the display device stored in advance, move to the initial position and then move to the target position, and display the image to be displayed in the process of moving from the initial position to the target position.
11. The holographic projection method of claim 10, wherein the method further comprises:
receiving feedback information sent by the holographic projection device;
and displaying the image to be displayed.
12. The holographic projection method of claim 10, wherein the display device acquiring the image to be displayed selected by the user and the position of the user comprises:
and the display device acquires the position of the user according to the image of the user and the scene thereof acquired by the image acquisition assembly.
13. A holographic projection method, comprising the steps of:
the holographic projection device receives an image to be displayed selected by a user and sent by a display device and the position of the user;
generating an initial position and a target position according to the position of the user and the position of the display device which is stored in advance respectively;
and moving to the target position after moving to the initial position, and displaying the image to be displayed in the process of moving from the initial position to the target position.
14. The holographic projection method of claim 13, after the moving to the target location, further comprising: and sending feedback information to the display device, and stopping displaying the image to be displayed so as to enable the display device to start displaying the image to be displayed.
15. The holographic projection method of claim 13, wherein the displaying the image to be displayed comprises:
and the holographic projection device performs holographic projection on the image to be displayed through a holographic projection component.
16. An electronic device, comprising: memory, processor and computer program stored on the memory and executable on the processor, which when executed by the processor implements a holographic projection method as claimed in any of claims 10 to 15.
17. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out a holographic projection method as claimed in any of the claims 10 to 15.
CN201910538834.0A 2019-06-20 2019-06-20 Display device, holographic projection device, system, method, apparatus, and medium Active CN112118432B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910538834.0A CN112118432B (en) 2019-06-20 2019-06-20 Display device, holographic projection device, system, method, apparatus, and medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910538834.0A CN112118432B (en) 2019-06-20 2019-06-20 Display device, holographic projection device, system, method, apparatus, and medium

Publications (2)

Publication Number Publication Date
CN112118432A CN112118432A (en) 2020-12-22
CN112118432B true CN112118432B (en) 2023-02-28

Family

ID=73796055

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910538834.0A Active CN112118432B (en) 2019-06-20 2019-06-20 Display device, holographic projection device, system, method, apparatus, and medium

Country Status (1)

Country Link
CN (1) CN112118432B (en)

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2009031338A (en) * 2007-07-24 2009-02-12 Funai Electric Co Ltd Projection type video display device
US8754831B2 (en) * 2011-08-02 2014-06-17 Microsoft Corporation Changing between display device viewing modes
EP3268916A4 (en) * 2015-03-11 2018-10-24 Ventana 3D LLC Holographic interactive retail system
CN107505825A (en) * 2017-08-30 2017-12-22 珠海格力电器股份有限公司 Holographic display method and device and user terminal

Also Published As

Publication number Publication date
CN112118432A (en) 2020-12-22

Similar Documents

Publication Publication Date Title
CN108351691B (en) Remote rendering for virtual images
US9723293B1 (en) Identifying projection surfaces in augmented reality environments
US9196067B1 (en) Application specific tracking of projection surfaces
US11107195B1 (en) Motion blur and depth of field for immersive content production systems
CN108038726B (en) Article display method and device
CN111164971B (en) Parallax viewer system for 3D content
WO2013112635A1 (en) Virtual environment generating system
KR20150091474A (en) Low latency image display on multi-display device
CN111970456B (en) Shooting control method, device, equipment and storage medium
US11720996B2 (en) Camera-based transparent display
CN112805755B (en) Information processing apparatus, information processing method, and recording medium
US20210042980A1 (en) Method and electronic device for displaying animation
JP2019512177A (en) Device and related method
CN111295693A (en) Image processing method and device
US9329679B1 (en) Projection system with multi-surface projection screen
US10437323B2 (en) Controlling a head mounted device
CN112118432B (en) Display device, holographic projection device, system, method, apparatus, and medium
US20230405475A1 (en) Shooting method, apparatus, device and medium based on virtual reality space
CN110377253B (en) Multimedia resource sharing method, device, system, equipment and medium
US20220075477A1 (en) Systems and/or methods for parallax correction in large area transparent touch interfaces
US9124786B1 (en) Projecting content onto semi-persistent displays
CN107133028B (en) Information processing method and electronic equipment
CN113227876B (en) Modified slow scan drive signal
US9536453B2 (en) Computer-implemented communication assistant for the hearing-impaired
Cebulla Projection-based augmented reality

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant