CN114572112A - Augmented reality method and system for automobile front windshield - Google Patents
Augmented reality method and system for automobile front windshield Download PDFInfo
- Publication number
- CN114572112A CN114572112A CN202210179099.0A CN202210179099A CN114572112A CN 114572112 A CN114572112 A CN 114572112A CN 202210179099 A CN202210179099 A CN 202210179099A CN 114572112 A CN114572112 A CN 114572112A
- Authority
- CN
- China
- Prior art keywords
- vehicle
- information
- front windshield
- dynamic icon
- dynamic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 230000003190 augmentative effect Effects 0.000 title claims abstract description 59
- 238000000034 method Methods 0.000 title claims abstract description 51
- 230000015654 memory Effects 0.000 claims description 18
- 230000003068 static effect Effects 0.000 claims description 17
- 230000000007 visual effect Effects 0.000 description 10
- 230000008569 process Effects 0.000 description 7
- 238000004891 communication Methods 0.000 description 6
- 230000003993 interaction Effects 0.000 description 5
- 230000005291 magnetic effect Effects 0.000 description 5
- 238000004590 computer program Methods 0.000 description 4
- 230000006399 behavior Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000001133 acceleration Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000006855 networking Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000012216 screening Methods 0.000 description 2
- 206010063385 Intellectualisation Diseases 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000012804 iterative process Methods 0.000 description 1
- 230000002265 prevention Effects 0.000 description 1
- 230000000630 rising effect Effects 0.000 description 1
- 239000000523 sample Substances 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 230000001052 transient effect Effects 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
- 230000003936 working memory Effects 0.000 description 1
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
- B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
- B60W50/08—Interaction between the driver and the control system
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
- B60W50/08—Interaction between the driver and the control system
- B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/20—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of display used
- B60R2300/205—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of display used using a head-up display
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/20—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of display used
- B60R2300/207—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of display used using multi-purpose displays, e.g. camera image and navigation or video on same display
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/80—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement
- B60R2300/8073—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement for vehicle security, e.g. parked vehicle surveillance, burglar detection
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
- B60W50/08—Interaction between the driver and the control system
- B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
- B60W2050/146—Display means
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W2540/00—Input parameters relating to occupants
- B60W2540/223—Posture, e.g. hand, foot, or seat position, turned or inclined
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W2554/00—Input parameters relating to objects
- B60W2554/20—Static objects
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W2554/00—Input parameters relating to objects
- B60W2554/40—Dynamic objects, e.g. animals, windblown objects
Abstract
The invention aims to provide an augmented reality method and equipment for an automobile front windshield, wherein the virtual information content of a dynamic icon is overlaid and displayed on the transparent automobile front windshield in the real world, so that a user can still check the driving environment condition through the transparent automobile front windshield, the user does not need to pay attention to a mobile phone or an electronic screen to check navigation information or to a dashboard to check the running parameters of a vehicle during driving operation, the user can conveniently check the vehicle-end dynamic icon overlaid and displayed on the transparent automobile front windshield in an augmented reality mode, and the driving safety of the user is ensured.
Description
Technical Field
The invention provides an augmented reality method and system for an automobile front windshield.
Background
With the continuous development of the science and technology of the automobile industry, the demand of users on automobile intellectualization and networking is continuously rising. In the automobile human-computer interaction scope, the iterative process of automobile development is accelerated by intelligent service and user experience, the workload of drivers needing active operation is continuously reduced, and the requirement on driving experience is increased.
With more and more vehicles on urban roads, the working conditions are developed in a large and complex trend. The content of the navigation information display becomes complicated, and the time and the judgment time for distracting the driver are increased. Particularly, at the road sections such as a fork road and a viaduct ramp, the sight of the driver needs to repeatedly move on the central control screen and the road surface to observe and select a route, which is easy to cause dangerous situations.
Disclosure of Invention
The invention provides an augmented reality method and system for an automobile front windshield.
The invention provides an augmented reality method for an automobile front windshield, wherein the method comprises the following steps:
acquiring running state information and running environment information of a vehicle;
and generating a corresponding vehicle end dynamic icon based on the running state information and the running environment information of the vehicle, and displaying the dynamic icon on the transparent automobile front windshield in an augmented reality mode in an overlapping manner.
Further, in the above method, generating a corresponding vehicle-side dynamic icon based on the operating state information and the operating environment information of the vehicle, and displaying the vehicle-side dynamic icon on the transparent front windshield of the automobile in an augmented reality manner by superimposing the dynamic icon, the method includes:
generating a corresponding running state dynamic icon based on the running state information of the vehicle;
generating a corresponding navigation dynamic icon based on the running environment information of the vehicle;
and displaying the running state dynamic icon on the transparent automobile front windshield in an augmented reality mode in an overlaid manner, and displaying the navigation dynamic icon on the transparent automobile front windshield in an overlaid manner in an augmented reality mode.
Further, in the method, after displaying the vehicle-end dynamic icon on the transparent front windshield of the automobile in an augmented reality manner, the method further includes:
acquiring an operation gesture of a user on the vehicle-end dynamic icon;
and recognizing the operation gesture to obtain a corresponding vehicle end control instruction, and performing driving control on the vehicle based on the vehicle end control instruction.
Further, in the above method, the method further includes:
acquiring running state information of a mobile terminal;
and generating a corresponding mobile terminal dynamic icon based on the running state information of the mobile terminal, and displaying the mobile terminal dynamic icon on the transparent automobile front windshield in an augmented reality mode in an overlapping manner.
Further, in the above method, after displaying the mobile terminal dynamic icon on the transparent front windshield of the automobile in an augmented reality manner, the method further includes:
acquiring an operation gesture of a user on the mobile terminal dynamic icon;
and identifying the operation gesture to obtain a corresponding mobile terminal control instruction, and carrying out operation control on the mobile terminal based on the mobile control instruction.
Further, in the above method, acquiring the operating environment information of the vehicle includes:
acquiring road dynamic information through a camera and a sensor of a vehicle;
acquiring static road information through a 3D map;
and obtaining the running environment information of the vehicle based on the road dynamic information and the road static information.
Further, in the method, the running environment information of the vehicle is obtained based on the road dynamic information and the road static information.
Acquiring first road dynamic information through a camera and a sensor of a vehicle of the vehicle;
acquiring second road dynamic information through a camera and a sensor of other vehicles near the vehicle communicating with the vehicle;
obtaining running environment information of the vehicle based on the road dynamic information and the road static information, wherein the running environment information comprises the following steps:
and obtaining the running environment information of the vehicle based on the first road dynamic information, the second road dynamic information and the road static information.
According to another aspect of the present invention, there is also provided an augmented reality apparatus for a front windshield of an automobile, wherein the apparatus includes:
an acquisition device for acquiring running state information and running environment information of a vehicle;
and the display equipment is used for generating a corresponding vehicle end dynamic icon based on the running state information and the running environment information of the vehicle, and displaying the dynamic icon on the transparent automobile front windshield in an augmented reality mode in an overlapping mode.
According to another aspect of the present invention, there is also provided a computing-based device, including:
a processor; and
a memory arranged to store computer executable instructions that, when executed, cause the processor to:
acquiring running state information and running environment information of a vehicle;
and generating a corresponding vehicle end dynamic icon based on the running state information and the running environment information of the vehicle, and displaying the dynamic icon on the transparent automobile front windshield in an augmented reality mode in an overlapping manner.
According to another aspect of the present invention, there is also provided a computer-readable storage medium having stored thereon computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, cause the processor to:
acquiring running state information and running environment information of a vehicle;
and generating a corresponding vehicle end dynamic icon based on the running state information and the running environment information of the vehicle, and displaying the dynamic icon on the transparent automobile front windshield in an augmented reality mode in an overlapping manner.
According to the invention, the virtual information content of the dynamic icon is superposed and displayed on the transparent automobile front windshield in the real world, so that the driving environment condition can still be checked through the transparent automobile front windshield on the aspect of a user, the user does not need to pay attention to a mobile phone or an electronic screen to check navigation information or pay attention to an instrument panel to check the running parameters of the vehicle during driving operation, the user can conveniently check the vehicle-end dynamic icon superposed and displayed on the transparent automobile front windshield in an augmented reality manner, and the driving safety of the user is ensured.
In addition, the operation gesture of the user on the vehicle end dynamic icon is obtained, the operation gesture is recognized to obtain the corresponding vehicle end control instruction, and the vehicle is controlled in a driving mode based on the vehicle end control instruction, so that the driving user can control the vehicle conveniently and reliably.
In addition, the operation gesture of the user on the dynamic icon of the mobile terminal is obtained, the operation gesture is recognized to obtain the corresponding control instruction of the mobile terminal, the mobile terminal is operated and controlled based on the control instruction, and the mobile terminal can be controlled conveniently and reliably by the driving user.
Drawings
FIG. 1 is a flow chart of an information display method for an augmented reality intelligent front windshield of the present invention;
FIG. 2 is a logical framework diagram of the 3D navigation and information display method of the present invention;
FIG. 3 is a schematic diagram illustrating the navigation visual effect output of the front windshield in the travel mode according to the present invention;
FIG. 4 is a schematic view of the navigation visual effect output of the front windshield in the night driving mode of the present invention;
FIG. 5 is a schematic view of the navigation visual effect output of the front windshield in extreme weather according to the present invention;
fig. 6 shows the information interaction intention of the driver and the intelligent front windshield, which takes the call as an example, of the invention.
Detailed Description
The present invention is described in further detail below with reference to the attached drawing figures.
In a typical configuration of the present application, the terminal, the device serving the network, and the trusted party each include one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include non-transitory computer readable media (transient media), such as modulated data signals and carrier waves.
The invention provides an augmented reality method of an automobile front windshield, which comprises the following steps:
step S1, acquiring running state information and running environment information of the vehicle;
and step S2, generating a corresponding vehicle-side dynamic icon based on the running state information and the running environment information of the vehicle, and displaying the dynamic icon on the transparent automobile front windshield in an augmented reality mode in an overlaying mode.
Specifically, as shown in fig. 1, the central control processor receives and processes navigation information, internet of vehicles, communication information and the like, generates information visualization data of the vehicle together with environmental conditions faced by a user (driver) and gesture control information received by the camera, displays information of the front windshield, and displays the displayed content to provide information forenotice and early warning such as communication information display, pedestrian prompting of a column blind area, and marking of a key road section for the user, so that combination of external information and user input information is completed.
Further, as shown in fig. 2, the central control processor is calculated to integrate various types of information to output virtual reality information, that is, the dynamic icon is displayed on the transparent front windshield of the automobile in an augmented reality manner in an overlaid manner.
Here, in order to distinguish the vehicle-side dynamic icon from the following mobile-side dynamic icon, the corresponding icon generated based on the information of the vehicle side is referred to as a vehicle-side dynamic icon. In the following embodiments, the corresponding icon generated according to the running state information of the mobile terminal is called a mobile terminal dynamic icon.
The invention displays the virtual information content of the dynamic icon on the transparent automobile front windshield in the real world in an overlapping way, so that a user can still check the driving environment condition through the transparent automobile front windshield, the user does not need to pay attention to a mobile phone or an electronic screen to check navigation information or to an instrument panel to check the running parameters of a vehicle during driving operation, the user can conveniently check the vehicle-end dynamic icon which is displayed on the transparent automobile front windshield in an overlapping way in an Augmented Reality way, and the driving safety of the user is ensured.
According to the invention, the dynamic icon virtual information content is superposed and displayed on the transparent automobile front windshield in the real world, so that a user can still check the driving environment condition through the transparent automobile front windshield, the user does not need to pay attention to a mobile phone or an electronic screen to check navigation information or a dashboard to check the running parameters of the vehicle during driving operation, the user can conveniently check the dynamic icon at the vehicle end superposed and displayed on the transparent automobile front windshield in an augmented reality manner, and the driving safety of the user is ensured.
In addition, the operation gesture of the user on the vehicle end dynamic icon is obtained, the operation gesture is recognized to obtain the corresponding vehicle end control instruction, and the vehicle is controlled in a driving mode based on the vehicle end control instruction, so that the driving user can control the vehicle conveniently and reliably.
In addition, the operation gesture of the user on the dynamic icon of the mobile terminal is obtained, the operation gesture is recognized to obtain the corresponding control instruction of the mobile terminal, the mobile terminal is operated and controlled based on the control instruction, and the mobile terminal can be controlled conveniently and reliably by the driving user.
In an embodiment of the augmented reality method for the front windshield of the automobile, in step S2, a corresponding vehicle-side dynamic icon is generated based on the operating state information and the operating environment information of the vehicle, and the vehicle-side dynamic icon is displayed on the transparent front windshield of the automobile in an augmented reality manner in an overlaid manner, including:
step S21, generating a corresponding running state dynamic icon based on the running state information of the vehicle;
here, the running state information of the vehicle may be, for example, a vehicle speed, an engine speed, a remaining oil amount, and the like, and may obtain a current running state parameter of the vehicle from an instrument panel of the vehicle or a sensor of the vehicle;
the running state dynamic icon may be a simplified dynamic icon corresponding to the vehicle speed, the engine speed, and the remaining oil amount, respectively. Here, the dynamic icon may be an icon that displays a numerical value corresponding to a change in the numerical values of the vehicle speed, the engine speed, and the remaining oil amount. The simplified dynamic icons can be adopted, simple and clear driving reference information can be provided for driving users, the complex icon users are prevented from observing the surrounding driving environment through the transparent automobile front windshield, and the driving safety is ensured;
step S22, generating a corresponding navigation dynamic icon based on the running environment information of the vehicle;
here, the running environment information of the vehicle may be, for example, information such as a road route, a road condition, weather, and an obstacle. Corresponding navigation dynamic icons, such as a driving direction icon, a destination distance icon, an arrival time icon, a speed limit icon, a lane line and the like;
and step S23, displaying the running state dynamic icon on the transparent automobile front windshield in an augmented reality mode, and displaying the navigation dynamic icon on the transparent automobile front windshield in an augmented reality mode.
In an embodiment of the augmented reality method for an automobile front windshield, step S2, after displaying the dynamic icon at the vehicle end on the transparent automobile front windshield in an augmented reality manner, further includes:
step S3, acquiring the operation gesture of the user on the vehicle end dynamic icon;
and step S4, recognizing the operation gesture to obtain a corresponding vehicle end control command, and performing driving control on the vehicle based on the vehicle end control command.
Specifically, in the dynamic end icons of the vehicle, the dynamic end icons of the vehicle generate corresponding running state dynamic icons based on the running state information of the vehicle, the current state motion parameters of the vehicle are mainly obtained, the dynamic process of the vehicle in the running process is also the dynamic change process of the motion state parameters of the vehicle, the corresponding navigation dynamic icons are generated based on the running environment information of the vehicle, the information of roads, road conditions, weather, obstacles and the like in the running process of the vehicle is mainly fed back in time, and the dynamic end icons of the vehicle can combine the running state information of the vehicle with the running environment information of the vehicle, so that the vehicle presents a humanized driving mode of vehicle-road cooperation for a user in the form of the dynamic end icons of the vehicle, and further, the user can freely customize running state dynamic charts and navigation dynamic icons suitable for the current driving habits and driving conditions and conforming to the properties and potentials of the driving vehicle, and personalized experience more suitable for the user is obtained.
Specifically, as shown in fig. 1, the method for acquiring the operation gesture of the user on the vehicle-side dynamic icon includes that the user performs gesture control, receives the operation gesture through the camera, and transmits the operation gesture to the central control processor, and the central control processor can perform gesture judgment and recognition, and optionally, the central control processor can perform gesture screening and judgment such as misoperation and shaking prevention.
The driving workload of a user can be reduced to a certain extent by obtaining the operation gesture of the user (driver) on the vehicle end dynamic chart, and a more accurate representation method of the vehicle end dynamic chart is obtained by screening the misoperation and the like of the user in the operation gesture and judging the anti-shaking behavior, so that better visual experience is provided for the user.
Here, the driving control of the vehicle may be a vehicle control method such as controlling acceleration, steering, braking, turning on a headlight, turning on an air conditioner, or the like of the vehicle, and for example, a gesture of a scissor hand may be used to indicate acceleration of the vehicle, and a gesture of a fist may be used to indicate braking of the vehicle, or the like.
According to the implementation, the operation gesture of the user on the vehicle end dynamic icon is acquired, the operation gesture is recognized to obtain a corresponding vehicle end control instruction, and the vehicle is controlled to be driven based on the vehicle end control instruction, so that the driving user can conveniently and reliably control the vehicle.
In an embodiment of the augmented reality method for the front windshield of the automobile, the method further includes:
step S5, acquiring the running state information of the mobile terminal;
here, the operation state information of the mobile terminal may be, for example, an incoming call notification, a WeChat notification, a short message notification, or the like;
and step S6, generating a corresponding mobile terminal dynamic icon based on the running state information of the mobile terminal, and displaying the mobile terminal dynamic icon on the transparent automobile front windshield in an augmented reality mode.
The carrier of the running state of the mobile terminal can be a mobile phone, a tablet computer, a notebook computer or the like, and can be connected with a vehicle in a certain mode, such as Bluetooth connection and NPC connection.
Further, when the running state information of the mobile terminal is a telephone notification, the automobile front windshield displays the mobile terminal dynamic icon representing the telephone notification on the transparent automobile front windshield in an augmented reality mode in a superposed manner, and a user can perform gesture or screen click to perform information interaction after seeing a sign signal of the telephone;
further, when the running state information of the mobile terminal is the WeChat notification, the automobile front windshield displays the mobile terminal dynamic icon representing the WeChat notification on the transparent automobile front windshield in an augmented reality mode in an overlapping mode, and a user can perform information interaction through gestures or screen clicking;
further, when the running state information of the mobile terminal is the short message notification, the automobile front windshield displays the mobile terminal dynamic icon representing the short message notification on the transparent automobile front windshield in an augmented reality mode in an overlapping mode, and a user can perform information interaction through gestures or screen clicking.
When the WeChat notification is a WeChat voice incoming call, the user can click the mobile terminal dynamic icon representing the WeChat voice notification to perform voice communication with the caller;
when the WeChat notification is a WeChat video incoming call, the user can click the mobile terminal dynamic icon representing the WeChat video notification to perform video communication with the caller.
Here, the corresponding dynamic icon is generated based on the operation state information of the mobile terminal, for example, the mobile terminal dynamic icon representing the telephone notification may be a telephone answering button icon, the mobile terminal dynamic icon representing the WeChat notification may be a microphone icon, the mobile terminal dynamic icon representing the SMS notification may be a short message icon, and the like, so that a user can directly obtain the operation state information of the mobile terminal such as a mobile phone and the like in time by superimposing the dynamic icon displayed on the transparent front windshield of the automobile, thereby avoiding frequent switching of the user's attention between the transparent front windshield of the automobile and the mobile terminal, and ensuring the driving safety of the user.
In an embodiment of the augmented reality method for an automobile front windshield, step S6, after displaying the mobile terminal dynamic icon on the transparent automobile front windshield in an augmented reality manner, further includes:
step S7, acquiring the operation gesture of the user on the mobile terminal dynamic icon;
and step S8, recognizing the operation gesture to obtain a corresponding moving end control instruction, and performing operation control on the moving end based on the moving control instruction.
Here, the present embodiment can be applied to a user to answer a call through gesture separation, as shown in fig. 6, when a driver drives a vehicle, the mobile phone obtains incoming call information, which may include information such as a phone dialing incoming call and a chat tool incoming call, and the incoming call information is received, connected to the vehicle through bluetooth, and processed by the central control processor, and displayed on the front windshield in the form of a mobile terminal dynamic icon.
The embodiment can enable the user to receive telephone notification, WeChat notification, short message notification and the like in the driving process, ensure the driving safety, and further enable the dialer to carry out video and other interactive behaviors in the conversation process with the user under the allowable condition, so that the dialer can observe the current condition of the user in a closer distance.
In this embodiment, the operation gesture of the mobile terminal dynamic icon is recognized by obtaining the operation gesture of the user to obtain the corresponding mobile terminal control instruction, and based on the operation control of the mobile terminal by the mobile control instruction, the convenient and reliable control of the driving user on the mobile terminal can be realized.
In an embodiment of the augmented reality method for the front windshield of the automobile, in step S1, the obtaining of the operating environment information of the automobile includes:
step S11, acquiring road dynamic information through a camera and a sensor of the vehicle;
step S12, obtaining road static information through a 3D map;
here, the 3D map may be, for example, a 3D navigation map;
in step S13, the running environment information of the vehicle is obtained based on the road dynamic information and the road static information.
Here, as shown in fig. 2, the dynamic road information includes working condition changes and pedestrian information, which is mainly obtained by a camera and a sensor of the vehicle, and the static road information may include 3D navigation information provided by a 3D map, so that a user can effectively perceive surrounding environment and road condition information.
Specifically, as shown in fig. 1, after the automobile is started, a user inputs a destination, the mobile phone is connected with the automobile through bluetooth, the vehicle networking information such as information collected by a camera and a sensor mounted outside the automobile body is input into the central control processor through WiFi, after the mobile phone is connected with the automobile through bluetooth, the communication information of the mobile phone is connected with the automobile through bluetooth, and when someone makes a call to the user, the incoming call information on the mobile phone can be input into the central control processor through bluetooth, so that the collection and display of the communication information are completed.
Meanwhile, more accurate running environment information of the vehicle can be obtained through the road dynamic information and the road static information.
In an embodiment of the augmented reality method for a front windshield of an automobile of the present invention, in step S11, the acquiring road dynamic information by a camera and a sensor of the automobile includes:
step S111, acquiring dynamic information of a first road through a camera and a sensor of a vehicle of the vehicle;
step S112, acquiring second road dynamic information through a camera and a sensor of other vehicles near the self vehicle which is communicated with the self vehicle;
here, the external cameras may include a camera mounted on a vehicle of the own vehicle to photograph the external road surface condition and a camera mounted on a camera of another vehicle to photograph the external road surface condition;
step S13, obtaining the operating environment information of the vehicle based on the road dynamic information and the road static information, including:
and obtaining the running environment information of the vehicle based on the first road dynamic information, the second road dynamic information and the road static information.
Specifically, as shown in fig. 2, the first road dynamics information and the second road dynamics information are generated around the own vehicle and the other vehicles respectively,
in the first road dynamic information, signals acquired by a camera and a sensor of a vehicle of the vehicle can show the situation of vehicle Positioning, pedestrian information and the like of the vehicle, and information can be transmitted through a Global Positioning System (GPS), an Intelligent Transport System (ITS), a wheel sensor and the like and input to a central control processor;
in the second road dynamic information, signals acquired by cameras and sensors of other vehicles can show the automobile Positioning condition, pedestrian information and the like of the other vehicles, and information can be transmitted through a Global Positioning System (GPS), an Intelligent Transport System (ITS), a wheel sensor and the like and input to a central control processor;
here, the own vehicle may be networked with other vehicles near the own vehicle, and the own vehicle may acquire the road dynamic information to be passed by the other vehicles near the own vehicle to expand the acquisition range of the road dynamic information of the own vehicle, for example, a ghost probe may be avoided.
As shown in fig. 3, it is a schematic diagram of the output of the visual effect of the navigation of the windshield in the travel mode, in which the dynamic icon of the vehicle end displayed on the front windshield of the vehicle can embody the characteristics of the travel mode, for example, in the travel mode, the destination of the travel, the specific traveling direction of the travel, the real-time speed of the travel and the distance to the destination can be displayed on the transparent front windshield of the vehicle, and optionally, the weather condition of the destination, the passenger flow volume of the destination, etc. can also be prompted on the transparent front windshield of the vehicle;
further, in the travel mode, the corresponding mobile terminal dynamic icon is generated based on the operation state information of the mobile terminal described in the above embodiment, and the mobile terminal dynamic icon may also be displayed in an augmented reality manner by being superimposed on a transparent automobile front windshield.
Herein, through for the user sets up windscreen navigation visual effect under the travel mode, give the user better driving experience, richen the journey impression.
As shown in fig. 4, which is a schematic view of the output of the visual effect of the windshield navigation in the night driving mode of the present invention, the vehicle-side dynamic icon displayed on the front windshield of the vehicle can embody the characteristics of the night driving mode, and the information including the driving direction, the vehicle speed, the engine speed, and the like can be displayed on the transparent front windshield of the vehicle.
Further, in the night driving mode, the corresponding mobile terminal dynamic icon is generated based on the operation state information of the mobile terminal described in the above embodiment, and the mobile terminal dynamic icon may also be displayed on the transparent front windshield of the automobile in an augmented reality manner.
Here, through setting up windshield navigation visual effect for the user under the mode of driving at night, can provide the reference of the more comprehensive driving condition of driving at night for the user, make the user more have the sense of security in the driving at night, no matter drive operation itself or drive psychology all have certain guarantee effect.
As shown in fig. 5, the schematic view of the navigation visual effect of the windshield in extreme weather according to the present invention is shown, the vehicle end dynamic icon displayed on the front windshield of the vehicle can embody the characteristics of the extreme weather mode, the hidden danger possibly generated in extreme weather is considered, the speed limit, the lane line, etc. of the vehicle can be provided on the transparent front windshield of the vehicle, and the road condition in extreme weather can be informed to the user in advance by prompting the driving restriction condition in the designated area around the vehicle on the transparent front windshield of the vehicle.
Further, in the extreme weather mode, the corresponding mobile terminal dynamic icon is generated based on the operation state information of the mobile terminal described in the above embodiment, and the mobile terminal dynamic icon may also be displayed on the transparent front windshield of the automobile in an augmented reality manner.
Here, by setting the windshield navigation visual effect for the user in the extreme weather mode, a basis reference can be provided for the driving behavior of the user in the extreme weather mode, and the user is given a real driving experience.
According to another aspect of the present invention, there is also provided an augmented reality apparatus for a front windshield of an automobile, wherein the apparatus includes:
an acquisition device for acquiring running state information and running environment information of a vehicle;
and the display equipment is used for generating a corresponding vehicle end dynamic icon based on the running state information and the running environment information of the vehicle, and displaying the dynamic icon on the transparent automobile front windshield in an augmented reality mode in an overlapping mode.
According to another aspect of the present invention, there is also provided a computing-based device, including:
a processor; and
a memory arranged to store computer executable instructions that, when executed, cause the processor to:
acquiring running state information and running environment information of a vehicle;
and generating a corresponding vehicle end dynamic icon based on the running state information and the running environment information of the vehicle, and displaying the dynamic icon on the transparent automobile front windshield in an augmented reality mode in an overlapping manner.
According to another aspect of the present invention, there is also provided a computer-readable storage medium having stored thereon computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, cause the processor to:
acquiring running state information and running environment information of a vehicle;
and generating a corresponding vehicle end dynamic icon based on the running state information and the running environment information of the vehicle, and displaying the dynamic icon on the transparent automobile front windshield in an augmented reality mode in an overlapping manner.
The details of each device embodiment of the present invention may specifically refer to the corresponding parts of each method embodiment, and are not described herein again.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.
It should be noted that the present invention may be implemented in software and/or in a combination of software and hardware, for example, as an Application Specific Integrated Circuit (ASIC), a general purpose computer or any other similar hardware device. In one embodiment, the software program of the present invention may be executed by a processor to implement the steps or functions described above. Also, the software programs (including associated data structures) of the present invention can be stored in a computer readable recording medium, such as RAM memory, magnetic or optical drive or diskette and the like. Further, some of the steps or functions of the present invention may be implemented in hardware, for example, as circuitry that cooperates with the processor to perform various steps or functions.
In addition, some of the present invention can be applied as a computer program product, such as computer program instructions, which when executed by a computer, can invoke or provide the method and/or technical solution according to the present invention through the operation of the computer. Program instructions which invoke the methods of the present invention may be stored on a fixed or removable recording medium and/or transmitted via a data stream on a broadcast or other signal-bearing medium and/or stored within a working memory of a computer device operating in accordance with the program instructions. An embodiment according to the invention comprises an apparatus comprising a memory for storing computer program instructions and a processor for executing the program instructions, wherein the computer program instructions, when executed by the processor, trigger the apparatus to perform a method and/or solution according to embodiments of the invention as described above.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned. Furthermore, it will be obvious that the term "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. A plurality of units or means recited in the apparatus claims may also be implemented by one unit or means in software or hardware. The terms first, second, etc. are used to denote names, but not any particular order.
Claims (10)
1. An augmented reality method of an automobile front windshield, wherein the method comprises the following steps:
acquiring running state information and running environment information of a vehicle;
and generating a corresponding vehicle-end dynamic icon based on the running state information and the running environment information of the vehicle, and displaying the dynamic icon on the transparent front windshield of the automobile in an augmented reality mode in an overlapping manner.
2. The method of claim 1, wherein generating a corresponding vehicle-side dynamic icon based on the operating state information and the operating environment information of the vehicle, and displaying the vehicle-side dynamic icon on a transparent automobile front windshield in an augmented reality manner in an overlapping manner comprises:
generating a corresponding running state dynamic icon based on the running state information of the vehicle;
generating a corresponding navigation dynamic icon based on the running environment information of the vehicle;
and displaying the running state dynamic icon on the transparent automobile front windshield in an augmented reality mode in an overlaid manner, and displaying the navigation dynamic icon on the transparent automobile front windshield in an overlaid manner in an augmented reality mode.
3. The method of claim 2, wherein displaying the vehicle-end dynamic icon superimposed on a transparent front windshield of the automobile in an augmented reality manner further comprises:
acquiring an operation gesture of a user on the vehicle-end dynamic icon;
and recognizing the operation gesture to obtain a corresponding vehicle end control instruction, and performing driving control on the vehicle based on the vehicle end control instruction.
4. The method of claim 1, wherein the method further comprises:
acquiring running state information of a mobile terminal;
and generating a corresponding mobile terminal dynamic icon based on the running state information of the mobile terminal, and displaying the mobile terminal dynamic icon on the transparent automobile front windshield in an augmented reality mode in an overlapping manner.
5. The method of claim 4, wherein displaying the mobile terminal dynamic icon superimposed on a transparent automobile front windshield in an augmented reality manner further comprises:
acquiring an operation gesture of a user on the mobile terminal dynamic icon;
and identifying the operation gesture to obtain a corresponding mobile terminal control instruction, and carrying out operation control on the mobile terminal based on the mobile control instruction.
6. The method of claim 1, wherein obtaining operating environment information for the vehicle comprises:
acquiring road dynamic information through a camera and a sensor of a vehicle;
acquiring static road information through a 3D navigation map;
and obtaining the running environment information of the vehicle based on the road dynamic information and the road static information.
7. The method of claim 6, wherein the running environment information of the vehicle is obtained based on road dynamic information and road static information;
acquiring first road dynamic information through a camera and a sensor of a vehicle of the vehicle;
acquiring second road dynamic information through a camera and a sensor of other vehicles near the self vehicle which is communicated with the self vehicle;
obtaining running environment information of the vehicle based on the road dynamic information and the road static information, wherein the running environment information comprises the following steps:
and obtaining the running environment information of the vehicle based on the first road dynamic information, the second road dynamic information and the road static information.
8. An augmented reality device for an automotive front windshield, wherein the device comprises:
an acquisition device for acquiring running state information and running environment information of a vehicle;
and the display equipment is used for generating a corresponding vehicle end dynamic icon based on the running state information and the running environment information of the vehicle, and displaying the dynamic icon on the transparent automobile front windshield in an augmented reality mode in an overlapping mode.
9. A computing-based device, comprising:
a processor; and
a memory arranged to store computer executable instructions that, when executed, cause the processor to:
acquiring running state information and running environment information of a vehicle;
and generating a corresponding vehicle end dynamic icon based on the running state information and the running environment information of the vehicle, and displaying the dynamic icon on the transparent automobile front windshield in an augmented reality mode in an overlapping manner.
10. A computer-readable storage medium having computer-executable instructions stored thereon, wherein the computer-executable instructions, when executed by a processor, cause the processor to:
acquiring running state information and running environment information of a vehicle;
and generating a corresponding vehicle end dynamic icon based on the running state information and the running environment information of the vehicle, and displaying the dynamic icon on the transparent automobile front windshield in an augmented reality mode in an overlapping manner.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210179099.0A CN114572112A (en) | 2022-02-25 | 2022-02-25 | Augmented reality method and system for automobile front windshield |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210179099.0A CN114572112A (en) | 2022-02-25 | 2022-02-25 | Augmented reality method and system for automobile front windshield |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114572112A true CN114572112A (en) | 2022-06-03 |
Family
ID=81770053
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210179099.0A Withdrawn CN114572112A (en) | 2022-02-25 | 2022-02-25 | Augmented reality method and system for automobile front windshield |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114572112A (en) |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20130027367A (en) * | 2011-09-07 | 2013-03-15 | 동국대학교 산학협력단 | A navigation apparatus and method for displaying course thereof |
| US20140168243A1 (en) * | 2012-12-19 | 2014-06-19 | Jeffrey Huang | System and Method for Synchronizing, Merging, and Utilizing Multiple Data Sets for Augmented Reality Application |
| CN104627078A (en) * | 2015-02-04 | 2015-05-20 | 刘波 | Automobile drive simulation system based on flexible transparent OLED and control method thereof |
| CN106314152A (en) * | 2015-07-03 | 2017-01-11 | Lg电子株式会社 | Driver assistance apparatus and vehicle including the same |
| CN106740114A (en) * | 2017-01-15 | 2017-05-31 | 上海云剑信息技术有限公司 | Intelligent automobile man-machine interactive system based on augmented reality |
| KR20170101758A (en) * | 2016-02-26 | 2017-09-06 | 자동차부품연구원 | Augmented Reality Head Up Display Navigation |
| CN108437896A (en) * | 2018-02-08 | 2018-08-24 | 深圳市赛格导航科技股份有限公司 | Vehicle drive assisting method, device, equipment and storage medium |
| CN109584596A (en) * | 2018-12-20 | 2019-04-05 | 奇瑞汽车股份有限公司 | Vehicle drive reminding method and device |
| CN110304057A (en) * | 2019-06-28 | 2019-10-08 | 威马智慧出行科技（上海）有限公司 | Car crass early warning, air navigation aid, electronic equipment, system and automobile |
| CN110672110A (en) * | 2018-07-03 | 2020-01-10 | 百度在线网络技术（北京）有限公司 | Navigation method, device and equipment of vehicle and computer readable storage medium |
| CN110920604A (en) * | 2018-09-18 | 2020-03-27 | 阿里巴巴集团控股有限公司 | Driving assistance method, driving assistance system, computing device, and storage medium |
| CN111152790A (en) * | 2019-12-29 | 2020-05-15 | 的卢技术有限公司 | Multi-device interactive vehicle-mounted head-up display method and system based on use scene |
| CN113837127A (en) * | 2021-09-28 | 2021-12-24 | 中国科学技术大学先进技术研究院 | Map and V2V data fusion model, method, system and medium |
-
2022
- 2022-02-25 CN CN202210179099.0A patent/CN114572112A/en not_active Withdrawn
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20130027367A (en) * | 2011-09-07 | 2013-03-15 | 동국대학교 산학협력단 | A navigation apparatus and method for displaying course thereof |
| US20140168243A1 (en) * | 2012-12-19 | 2014-06-19 | Jeffrey Huang | System and Method for Synchronizing, Merging, and Utilizing Multiple Data Sets for Augmented Reality Application |
| CN104627078A (en) * | 2015-02-04 | 2015-05-20 | 刘波 | Automobile drive simulation system based on flexible transparent OLED and control method thereof |
| CN106314152A (en) * | 2015-07-03 | 2017-01-11 | Lg电子株式会社 | Driver assistance apparatus and vehicle including the same |
| KR20170101758A (en) * | 2016-02-26 | 2017-09-06 | 자동차부품연구원 | Augmented Reality Head Up Display Navigation |
| CN106740114A (en) * | 2017-01-15 | 2017-05-31 | 上海云剑信息技术有限公司 | Intelligent automobile man-machine interactive system based on augmented reality |
| CN108437896A (en) * | 2018-02-08 | 2018-08-24 | 深圳市赛格导航科技股份有限公司 | Vehicle drive assisting method, device, equipment and storage medium |
| CN110672110A (en) * | 2018-07-03 | 2020-01-10 | 百度在线网络技术（北京）有限公司 | Navigation method, device and equipment of vehicle and computer readable storage medium |
| CN110920604A (en) * | 2018-09-18 | 2020-03-27 | 阿里巴巴集团控股有限公司 | Driving assistance method, driving assistance system, computing device, and storage medium |
| CN109584596A (en) * | 2018-12-20 | 2019-04-05 | 奇瑞汽车股份有限公司 | Vehicle drive reminding method and device |
| CN110304057A (en) * | 2019-06-28 | 2019-10-08 | 威马智慧出行科技（上海）有限公司 | Car crass early warning, air navigation aid, electronic equipment, system and automobile |
| CN111152790A (en) * | 2019-12-29 | 2020-05-15 | 的卢技术有限公司 | Multi-device interactive vehicle-mounted head-up display method and system based on use scene |
| CN113837127A (en) * | 2021-09-28 | 2021-12-24 | 中国科学技术大学先进技术研究院 | Map and V2V data fusion model, method, system and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP2714456B1 (en) | System and method for selectively altering content of a vehicle interface | |
| KR20140031369A (en) | Method and device for assisting a driver in performing lateral guidance of a vehicle on a carriageway | |
| JP4867463B2 (en) | Driving assistance device | |
| FI124068B (en) | A method to improve driving safety | |
| KR20120118074A (en) | Vehicle periphery monitoring device | |
| JP6037923B2 (en) | Display information generating apparatus and display information generating method | |
| WO2013090125A1 (en) | Providing a user interface experience based on inferred vehicle state | |
| US20160379422A1 (en) | Systems and methods for displaying vehicle information with see-through effect | |
| CN108099790A (en) | Driving assistance system based on augmented reality head-up display Yu multi-screen interactive voice | |
| US11694656B2 (en) | Display control device, display control method, and storage medium storage program | |
| JP6113006B2 (en) | Vehicle information display control device | |
| US11858526B2 (en) | Apparatus and method for use with vehicle | |
| CN114572112A (en) | Augmented reality method and system for automobile front windshield | |
| JP6539057B2 (en) | Vehicle display device | |
| JP5435113B2 (en) | Energy saving evaluation device, energy saving evaluation method | |
| CN114527923A (en) | In-vehicle information display method and device and electronic equipment | |
| US20240042858A1 (en) | Vehicle display system, vehicle display method, and storage medium storing vehicle display program | |
| JP2017174043A (en) | Display device | |
| JP2022041245A (en) | Vehicle display control device, method, program, and vehicle display system | |
| CN114084159A (en) | Driving assistance function reminding method, device, medium and vehicle | |
| WO2014049705A1 (en) | Display control device, display control method, program, and server device | |
| WO2022210172A1 (en) | Vehicular display system, vehicular display method, and vehicular display program | |
| JP6520212B2 (en) | Telephone call reception system, in-vehicle device, control program | |
| US20230234442A1 (en) | Method For Displaying Augmented Reality Information In Vehicles | |
| WO2022224752A1 (en) | Vehicle display system, display system, display method, and display program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20220603 |