CN108665556B - Road indication display method based on mixed reality and storage medium - Google Patents
Road indication display method based on mixed reality and storage medium Download PDFInfo
- Publication number
- CN108665556B CN108665556B CN201810484962.7A CN201810484962A CN108665556B CN 108665556 B CN108665556 B CN 108665556B CN 201810484962 A CN201810484962 A CN 201810484962A CN 108665556 B CN108665556 B CN 108665556B
- Authority
- CN
- China
- Prior art keywords
- information
- virtual
- road
- model
- display
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 49
- 238000003860 storage Methods 0.000 title claims abstract description 16
- 238000002156 mixing Methods 0.000 claims description 26
- 238000002955 isolation Methods 0.000 claims description 19
- 238000004590 computer program Methods 0.000 claims description 14
- 238000013528 artificial neural network Methods 0.000 claims description 13
- 238000013135 deep learning Methods 0.000 claims description 10
- 238000010295 mobile communication Methods 0.000 claims description 3
- 230000000694 effects Effects 0.000 abstract description 7
- 239000000203 mixture Substances 0.000 description 21
- 238000010586 diagram Methods 0.000 description 8
- 230000008569 process Effects 0.000 description 8
- 238000004891 communication Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 5
- 238000012545 processing Methods 0.000 description 5
- 230000008859 change Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 230000003190 augmentative effect Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 239000003973 paint Substances 0.000 description 2
- 210000001525 retina Anatomy 0.000 description 2
- 230000004270 retinal projection Effects 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 206010039203 Road traffic accident Diseases 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/003—Navigation within 3D models or images
-
- E—FIXED CONSTRUCTIONS
- E01—CONSTRUCTION OF ROADS, RAILWAYS, OR BRIDGES
- E01F—ADDITIONAL WORK, SUCH AS EQUIPPING ROADS OR THE CONSTRUCTION OF PLATFORMS, HELICOPTER LANDING STAGES, SIGNS, SNOW FENCES, OR THE LIKE
- E01F9/00—Arrangement of road signs or traffic signals; Arrangements for enforcing caution
- E01F9/60—Upright bodies, e.g. marker posts or bollards; Supports for road signs
- E01F9/604—Upright bodies, e.g. marker posts or bollards; Supports for road signs specially adapted for particular signalling purposes, e.g. for indicating curves, road works or pedestrian crossings
- E01F9/615—Upright bodies, e.g. marker posts or bollards; Supports for road signs specially adapted for particular signalling purposes, e.g. for indicating curves, road works or pedestrian crossings illuminated
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/05—Geographic models
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/006—Mixed reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
- G06F2203/01—Indexing scheme relating to G06F3/01
- G06F2203/012—Walk-in-place systems for allowing a user to walk in a virtual environment while constraining him to a given position in the physical environment
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2200/00—Indexing scheme for image data processing or generation, in general
- G06T2200/04—Indexing scheme for image data processing or generation, in general involving 3D image data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30248—Vehicle exterior or interior
- G06T2207/30252—Vehicle exterior; Vicinity of vehicle
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Graphics (AREA)
- Computer Hardware Design (AREA)
- Remote Sensing (AREA)
- Geometry (AREA)
- Radar, Positioning & Navigation (AREA)
- Human Computer Interaction (AREA)
- Architecture (AREA)
- Civil Engineering (AREA)
- Structural Engineering (AREA)
- Processing Or Creating Images (AREA)
- Traffic Control Systems (AREA)
Abstract
A road indication display method and a storage medium based on mixed reality comprise the following steps of establishing three-dimensional model information of real space scanning, receiving virtual road mixed information, loading the virtual road mixed information, and displaying the virtual road mixed information and the real space information in a mixed mode. The technical effect of auxiliary display of the virtualized display road information can be achieved.
Description
Technical Field
The invention relates to the field of mixed reality, in particular to the field of road indication sign display applying mixed reality.
Background
Mixed Reality (MR), which includes both augmented reality and augmented virtual, refers to a new visual environment created by merging the real and virtual worlds. Physical and digital objects coexist in the new visualization environment and interact in real time. The system is generally characterized by:
it combines virtual and real; projecting a virtual in a real three-dimensional space; and (5) running in real time.
The implementation of Mixed Reality (MR) is required in an environment that can interact with real world things. All things are virtual is the VR domain. If the virtual information displayed can be simply superimposed on the real thing, the AR is the virtual information. The key point of MR is to interact with the real world and acquire information in time.
In the existing road driving environment, the isolation belt is required to be pulled up in reality, the guideboard is established, the lane paint is brushed and the like, so that resources are wasted, if virtual display of road traffic facilities can be achieved through the existing computer-aided display technology, a large amount of physical cost can be saved, and the road indication sign which is virtually displayed cannot be damaged even if the road indication sign is 'bumped' by a vehicle, so that the safety of the driving environment is improved.
Disclosure of Invention
For this reason, it is necessary to provide a new auxiliary display method capable of virtualizing the road information.
In order to achieve the above object, the present inventors provide a road indication display method based on mixed reality, which includes the steps of establishing three-dimensional model information of real space scanning, receiving virtual road mixing information, loading the virtual road mixing information, and displaying the virtual road mixing information and real space information in a mixed manner.
Specifically, the virtual road mixing information comprises virtual mixing model information and coordinate information; the virtual hybrid model information includes two-dimensional hybrid information or three-dimensional hybrid information.
Further, the virtual hybrid model information is used for playing a role in road traffic indication, including road signs, traffic signboards, lane lines, straight/turning sign marks, guardrails, isolation belts, triangular pyramids or safety islands.
Preferably, the method further comprises the steps of receiving first virtual road mixture information, receiving second virtual road mixture information, receiving a switching display instruction of the virtual road mixture information, and after receiving the switching display instruction, replacing the first virtual road mixture information originally displayed with the second virtual road mixture information for display by the display device.
A mixed reality-based road indication display storage medium storing a computer program which, when executed, performs the steps of creating three-dimensional model information for real space scanning, receiving virtual road mix information, loading the virtual road mix information, and displaying the virtual road mix information mixed with real space information.
Further, the virtual road mixing information comprises virtual mixing model information and coordinate information; the virtual hybrid model information includes two-dimensional hybrid information or three-dimensional hybrid information.
Specifically, the virtual hybrid model information is used for playing a role in road traffic indication, and comprises road signs, traffic signboards, lane lines, straight-going/turning sign marks, guardrails, isolation belts, triangular pyramids or safety islands.
Preferably, the computer program further executes the steps of receiving the first virtual road mix information, receiving the second virtual road mix information, receiving a switching display instruction of the virtual road mix information, and after receiving the switching display instruction, the display device replaces the first virtual road mix information originally displayed with the second virtual road mix information to display.
Compared with the prior art, the technical scheme has the advantages that the new mark is seen through the display equipment, and meanwhile, the problem that the color of the existing entity indication is easy to fade after a long time is also solved. Moreover, all the existing road administration related signs can be replaced and displayed according to the needs, so that not only can the changed traffic lights be displayed, but also the time-limited closed lanes, traffic-forbidden signboards and the like can be displayed.
Drawings
FIG. 1 is a flow chart of a method for displaying a road indication according to an embodiment of the present invention;
FIG. 2 is a block diagram of a mixed reality road display system according to an embodiment of the invention;
FIG. 3 is a flow chart of a method for displaying a mixed reality road according to an embodiment of the invention;
FIG. 4 is a flowchart of an isolated model display method according to an embodiment of the present invention;
fig. 5 is a flowchart of a road display optimization method according to an embodiment of the present invention.
Detailed Description
In order to describe the technical content, constructional features, achieved objects and effects of the technical solution in detail, the following description is made in connection with the specific embodiments in conjunction with the accompanying drawings.
Referring to fig. 1, a schematic diagram of a method for displaying a road indication with mixed reality according to the present invention is shown, and the method can be used in any situation of road traffic, whether it is an urban road, an off-road, an expressway, etc., and the method can be used for displaying the road indication information and the mixed reality of road conditions. From the figure, we see that the method of the present invention includes the following steps, S100, creating three-dimensional model information of the scanned real space, that is, the real space environment needs to be dataized in advance, which is a conventional step that needs to be completed in general mixed reality, so as to facilitate accurate mixing of the subsequent virtual information and the real information. The real space model information is established, so that the display of the subsequent virtual road mixing information can be more accurate. However, in the prior art, there are few examples of performing hybrid display in an open space, which requires a large memory, but this is not overcome, but in an application scenario of an intersection, an area within a certain range near the intersection may be used as a limited space to perform modeling, so in the embodiment of the present invention, the step is performed, the display device receives a real space model, and next, the method further includes the step of receiving virtual road hybrid information in step S104, and loading the virtual road hybrid information. The display device is an electronic device, typically a head-mounted device, which can allow light generated or reflected by a real object to pass through, and can generate virtual and auxiliary light signals (whether a display screen is needed as a medium or direct retinal projection imaging) which can be accepted by retina, and in a preferred embodiment, the display device can even be realized by a windshield of a vehicle, and the like, and the loading virtual road mixing information is that only a model display with a certain shape is needed to generate a light shadow relative to human eyes so that people 'feel' that a certain position really exists. AR technology and AR devices that simply do the display of virtual models are not excluded here, and indeed up to now AR and MR do not have a completely defined distinction, even in most cases the difference in "purported" effect between different companies. In the scheme, the effect of mixing and displaying the virtual road mixed information and the real space model information only needs to be finally achieved. The virtual road mixing information comprises virtual mixing model information and coordinate information; the virtual mixed model information is a plurality of luminous points, and the pixel points are represented as two-dimensional mixed information in a two-dimensional plane or three-dimensional mixed information with a certain spatial structure in a virtual space. The virtual hybrid model can be used for indicating road traffic, for example, the virtual hybrid model can be a two-dimensional picture such as road sign, traffic sign, lane line on the ground, straight/turning sign and the like. The three-dimensional model can also be a three-dimensional model such as a guardrail, a separation belt, a triangular cone, a safety island and the like. The coordinate information, that is, the position information of these virtual hybrid models with respect to the aforementioned real space model may be the position information of the light spots in all models or the coordinate information of the feature points in different models. The method comprises the steps of calculating the relative positions of the virtual mixed model and the display device and displaying the virtual mixed model at the relative positions through mixed display of the virtual mixed model information and the real space model. Through the steps, a user can see the virtually generated road traffic indication mark through the mixed reality display device under the road traffic situation, the cost of the entity road mark can be saved, and the safety of road traffic is improved. More importantly, when the traffic indication mark of the road of the part of road sections needs to be changed, the virtual model can be edited or replaced on the computer, so that the scheme further comprises the steps of S106 receiving the first virtual road mixing information, receiving the second virtual road mixing information, S108 receiving a switching display instruction of the virtual road mixing information, and S110 displaying the first virtual road mixing information originally displayed after receiving the switching display instruction by the display device. If the left-most virtual traffic lane of a certain intersection is indicated as a turning lane, when the virtual traffic lane is required to be changed into a left-turning lane, a person does not need to be dispatched to paint again, and the two-dimensional mixed information is directly adjusted to be a road indication for displaying the left-turning lane. The user can easily see the new mark through the display device, and the problem that the existing entity indication is easy to fade after a long time is also solved. Moreover, all the existing road administration related signs can be replaced and displayed according to the needs, so that not only can the changed traffic lights be displayed, but also the time-limited closed lanes, traffic-forbidden signboards and the like can be displayed.
Specifically, the invention further comprises a road administration indication display storage medium based on mixed reality, wherein a computer program is stored, the computer program performs the steps of establishing three-dimensional model information of real space scanning when being executed, receiving virtual road mixing information, loading the virtual road mixing information, and displaying the virtual road mixing information and the real space information in a mixed mode.
Further, the virtual road mixing information comprises virtual mixing model information and coordinate information; the virtual hybrid model information includes two-dimensional hybrid information or three-dimensional hybrid information.
Specifically, the virtual hybrid model information is used for playing a role in road traffic indication, and comprises road signs, traffic signboards, lane lines, straight-going/turning sign marks, guardrails, isolation belts, triangular pyramids or safety islands.
Preferably, the computer program further executes the steps of receiving the first virtual road mix information, receiving the second virtual road mix information, receiving a switching display instruction of the virtual road mix information, and after receiving the switching display instruction, the display device replaces the first virtual road mix information originally displayed with the second virtual road mix information to display.
In order to be able to use the mixed reality display method of new road indications in a larger area, we have also devised a new system. As shown in fig. 2, this system includes a vehicle-mounted end 20, and a relay station 22 disposed along a road, where the road may be on a roadside, a road surface, a roadbed, or the like. The relay station is used for storing the real space model information of the part of road segments and storing the virtual mixed model information of the part of road segments. The part of road segments are used for positioning the display device 200 at the vehicle-mounted end only for convenience in general, the display device is used for positioning the position of the display device in the real space model, and the virtual hybrid model also comprises coordinate information in the real space model, so that the real-time relative position of the virtual hybrid model and the display device is obtained through calculation of coordinate transformation, and smooth display can be performed. Therefore, in some embodiments, the real space model and the virtual hybrid model may be packaged and stored, and when communication with the display device is required, for example, after the relay station receives a data request instruction of the display device, the relay station transmits the packaged data to the vehicle-mounted terminal, or the real space model information and the virtual hybrid model may be respectively transmitted, so that the method is practical in situations where the virtual hybrid model needs to be changed frequently. The requirement of road traffic indication sign mixed display in a large-scale space can be solved through the design of the relay station.
In this system, the system further includes a scanning module 202, where the scanning module is configured to scan the real space, build a real space model, and store the real space model, where the scanning module can collect the real space model and build data information, and the scanning module may be set along a road, or may store a scanning result along with the relay station collocation setting, where such a scanning result may have a certain limitation. In a preferred embodiment, as shown in fig. 2, the scanning module 202 is disposed at a vehicle-mounted end, and scans road condition information in real time during the running process of the vehicle, so as to construct a real space model in a form that the real space is divided into road segments, and then the road segments are sent to the relay stations 22 corresponding to different road segments for storage. Techniques for modeling space in an in-vehicle scanning mode have become mature. The accuracy of modeling of the road related area can be further improved by performing real-time block scanning through the moving vehicle end.
From fig. 2 we can see that the mixed reality display system further comprises a display device at the vehicle-mounted end, which is an electronic device, typically a head-mounted, preferably even a windshield designed as a vehicle, capable of transmitting light generated or reflected by real objects, while being capable of generating virtual, or auxiliary, display light signals that can be received by the retina, whether the individual display screens are needed as media or direct retinal projection imaging. The display device may be configured to perform the steps of receiving the real space model, receiving the virtual road mix information, loading the virtual road mix information, and displaying the virtual road mix information. Through the system, the mixed reality display system achieves the effect of displaying the virtualized traffic indication sign, solves the problem that in the prior art, the road guardrail, the isolation belt and the like of the entity destroy the vehicle to death and are too dangerous once being collided, and can save space even though the isolation fence is carelessly collided after being changed into virtual display, a driver only needs to go back to the mind and then turn back to the steering wheel, and the vehicle can be driven back to the correct road. The virtual road traffic indication mark can also achieve the technical effects of conveniently changing the indication mark, saving cost, being easy to replace and convenient to maintain.
In order to better achieve the management of the road traffic indication sign, the system further comprises a server 24, wherein the server comprises an editing module 240 and a communication module 242, the editing module is used for editing virtual mixed model information of a certain road section, the editing content can comprise a model to be displayed on the road section, display time, the shape and the size of each model, content displayed on the model, coordinate information of the model and the like, and a user can select, modify, add, delete, color and the like the virtual mixed model information of the specific road section on the server. Finally, the server sends the edited virtual mixed model information to a relay station through a communication module, and the relay station sends the edited virtual mixed model information to a vehicle or directly sends the edited virtual mixed model information to a vehicle-mounted terminal through a mobile communication network. In a specific embodiment, a lane at a certain intersection is a limited bus lane, and a yellow solid line is displayed in the display device from 11 to 13 midday and from 17 to 19 evening, and a white solid line is displayed in the rest period. On road traffic indication marks of certain road sections, the road traffic indication marks are single-way roads in certain time, and pedestrian streets for prohibiting motor vehicles from entering in other times such as holidays. The system can be flexibly set according to the needs of city managers, police duty scheduling needs to be sent out in the past, and the problem of human resource consumption can be solved after the system is applied.
In order to further improve the practicability of the mixed reality display system, the vehicle end is further provided with an unmanned navigation module 204, wherein the unmanned navigation module is used for receiving road condition information, processing and judging, and further sending out a control instruction of road running to the vehicle. The above functions can be implemented very mature according to the prior art, and the logic of processing judgment is not different, for example, google X of Google company, and companies such as hundred degrees and UBER have mature unmanned AI. The system has the advantages that the unmanned navigation module can receive virtual mixed model information and also serve as input of a part of road condition information, the existing road condition information generally identifies emergency in actual road conditions through optical identification, radar scanning and other modes to conduct driving navigation, and the scanning result of the actual road conditions needs to be dataized to conduct processing judgment. In our embodiment, the vehicle-mounted terminal receives the virtual hybrid model information and then sends the virtual hybrid model information to the unmanned navigation module, and the unmanned navigation module judges the position, the size, the color, the text and the like existing according to the virtual hybrid model as road condition information, so as to send a control instruction of road running to the vehicle. Thus, when the human driver with the display device and the unmanned vehicle exist on the road at the same time, the two parties can 'see' the same road condition and road traffic indication mark, thereby avoiding traffic accidents caused by information differences.
Corresponding to the system in fig. 2, we will also introduce a mixed reality road display method, as shown in fig. 3, further including the step of S302 that the vehicle-mounted terminal receives virtual mixed model information or real space model information stored in the relay station; locating a position of the vehicle in the real space model; the virtual hybrid model further comprises coordinate information of the virtual hybrid model in the real space model, the real-time relative position of the virtual hybrid model and the vehicle is obtained through calculation of coordinate transformation according to the coordinate information S304, and the virtual hybrid model is displayed according to the relative position in display equipment at the vehicle end. The method can solve the requirement of road traffic indication sign mixed display in a large-scale space, so that vehicles can refer to mixed reality display anytime and anywhere.
In a specific embodiment, the method further comprises the step that the vehicle-mounted terminal sends a data request instruction; and the relay station receives the data request instruction and returns real space model information or virtual mixed model information according to the request instruction. The steps are to realize the transmission and sharing between the storage content of the relay station and the vehicle-mounted terminal, so that the vehicle can always know the specific condition of the road in front or the road section where the vehicle is located in the driving process, the communication modes between the vehicle-mounted terminal and the relay station can be various, and the communication modes can be freely selected according to the prior communication technology. However, it is not necessary for the vehicle-mounted terminal to send a data request instruction, for example, the relay station always broadcasts the real space model information or the virtual hybrid model information outwards, which can achieve the same effect.
In other embodiments, the method further comprises the step of scanning the real space by the scanning module, establishing a real space model and storing the scanning result in the relay station. The accuracy of modeling of the road related area can be further improved by performing real-time block scanning through the moving vehicle end. In a preferred embodiment, the building of the real space model may be that the real space model is built within a preset space range, and the real space models in different spaces are sent to the corresponding relay stations for storage after the building is completed.
In order to better provide different virtual mixed model information for display, the method further comprises S300 of editing virtual mixed model information of a specific road section before the mixed model or the real space model is sent by a relay station, wherein the virtual mixed model information comprises a model to be displayed on the road section, display time, the shape and the size of the model, content displayed on the model or coordinate information of the model; and sending the edited virtual mixed model information to the relay station. The editing content can comprise a model to be displayed on the road section, display time, the shape and size of each model, content displayed on the model, coordinate information of the model and the like, and a user can select, modify, add, delete, color and the like the virtual mixed model information of the specific road section on the server. By means of artificial design, labor cost can be better saved, and emergency response level to road facilities can be improved.
In a further embodiment, the method further includes the steps that the unmanned navigation module at the vehicle-mounted end receives road condition information, processes and judges according to the road condition information and the virtual hybrid model information, and the unmanned navigation module needs to 'regard' all the virtual hybrid model information as existing physical entities and sends a control instruction of road running to the vehicle based on the logic. The adaptability of the method in the unmanned field can be further improved by processing and responding the virtual mixed model information as road condition information.
In some more advanced examples, we will see how to solve the problem of road congestion with new virtual hybrid models.
Let us consider a scenario in which a certain direction vehicle entering a traffic light intersection has been set up as a long dragon, but the opposite lane of the road is empty, which is very common in modern cities, the number of available lanes of the intersection represents the upper limit of the traffic flow of the road section, and in a specific period, the number of vehicles entering the road section is higher than the number of vehicles leaving the road section, so that congestion is naturally caused, in our scheme, as shown in fig. 4, the steps are included, and when congestion is detected at the intersection of the current road section, step is performed, the display position of the isolation model in the virtual mixed model information is changed, and the road width of the direction entering the intersection is widened. The method for detecting whether the traffic jam is caused by the prior art can be used as a reference, for example, whether the traffic jam is caused by the traffic flow detection is judged through a threshold value; or receiving the vehicle positioning, judging that the vehicle positioning is jammed when the average time of the vehicle positioning trapped on a certain road section exceeds a certain threshold value, and the like, wherein the technology is used more in each map software; in a more preferred embodiment, the data of the urban monitoring network of each metropolitan area can be directly called, and the congestion condition of the current road can be directly judged through image recognition in the monitoring of each road section. The isolation model herein is a two-dimensional or three-dimensional virtual model for separating opposite lanes, such as guardrails, hard-pads, greenbelts, double yellow lines, and the like. The display position is changed to be a position for widening one lane, and the opposite lane is narrowed, but one lane is added in the direction of waiting for traffic lights, so that the length of a traffic light of a traffic jam can be effectively reduced, one more team is allowed to enter the intersection at the same time when the road is allowed to enter the intersection, and the upper limit of the traffic flow of the road section can be increased, and the congestion problem of the current road section is effectively relieved.
In the new method, the road politics management is carried out by changing the display position of the virtual isolation model, but if the change is too sudden, the driving accident can be caused, the method can carry out a means of linking with the traffic signal lamp in order to avoid the problem, and when the straight-going vehicles and the left-turning vehicles of other lanes are limited by the traffic signal lamp and can not enter the road section, the change of the display position of the isolation model is carried out, and the method can also carry out the steps of displaying the virtual warning mark to prompt the avoidance of the vehicles in the lanes needing to be changed, and detecting whether the vehicles exist in the lanes possibly changed, wherein the detected possibly changed lanes comprise the lanes with certain width and the limitation in certain length. The road section with the length of 100 meters can be divided into 10 sections, the lane which can be changed is the width of a single lane, whether vehicles exist in the preset road section with the length of 10 meters of the single lane or not is detected each time, if not, the step is carried out, and the display position of the isolation model of the road section with the length of 10 meters is changed, so that the number of lanes entering the intersection is increased. Or when the road is not congested, the display position of the isolation model is changed by reverse operation, so that the number of lanes entering the intersection is reduced. And detecting the road section of the next 10 meters, and repeating the steps until the isolation models in all the preset road sections can obtain the change of the display positions.
Therefore, in the embodiment with stronger comprehensiveness, the scheme includes the steps as shown in fig. 5, in which S500 records congestion conditions of different roads and display contents of the virtual hybrid model, specifically including time of starting congestion and time of ending congestion, the virtual hybrid model displays duration of different contents, and congestion information statistics results respectively classified according to week, date and time are determined according to the congestion information statistics results, and adjustment periods of optimal isolation model display positions of different road sections are determined. In a general embodiment, the statistical result may be analyzed, for example, an average value is used for determining an optimal adjustment period for the time when congestion starts and the time when congestion ends in different road segments, so that the display position of the isolation model is changed in the congestion period, and thus, the road surface is widened, and the like. In some other embodiments, step S502 is further performed to construct a deep learning neural network, where an input layer of the deep learning neural network includes a date, a week, a time of day, whether the input layer is a holiday, and a road congestion condition, and an output layer includes display position information of the virtual hybrid model, and display contents of the virtual hybrid model may be trained according to the statistical result, so that the trained deep learning neural network capable of automatically adjusting the virtual hybrid display contents according to the time and the road condition can be finally obtained. S504, the trained deep learning neural network is used, the current time and date and road congestion conditions are collected at each preset time, and the neural network can obtain virtual mixed model information to be displayed at the optimal preset time. Of course, the processing in the application of the neural network can be not limited to the scope of the display of the isolation model, and the display and the change of all the virtual hybrid models can be completed through the neural network. The problem of intelligent optimization of virtual mixed model display is solved through the scheme.
It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the statement "comprising … …" or "comprising … …" does not exclude the presence of additional elements in a process, method, article or terminal device comprising the element. Further, herein, "greater than," "less than," "exceeding," and the like are understood to not include the present number; "above", "below", "within" and the like are understood to include this number.
It will be appreciated by those skilled in the art that the various embodiments described above may be provided as methods, apparatus, or computer program products. These embodiments may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. All or part of the steps in the methods according to the above embodiments may be implemented by a program for instructing related hardware, and the program may be stored in a storage medium readable by a computer device, for performing all or part of the steps in the methods according to the above embodiments. The computer device includes, but is not limited to: personal computers, servers, general purpose computers, special purpose computers, network devices, embedded devices, programmable devices, intelligent mobile terminals, intelligent home devices, wearable intelligent devices, vehicle-mounted intelligent devices and the like; the storage medium includes, but is not limited to: RAM, ROM, magnetic disk, magnetic tape, optical disk, flash memory, usb disk, removable hard disk, memory card, memory stick, web server storage, web cloud storage, etc.
The embodiments described above are described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a computer device to produce a machine, such that the instructions, which execute via the processor of the computer device, create means for implementing the functions specified in the flowchart block or blocks and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer device-readable memory that can direct a computer device to function in a particular manner, such that the instructions stored in the computer device-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer apparatus to cause a series of operational steps to be performed on the computer apparatus to produce a computer implemented process such that the instructions which execute on the computer apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While the embodiments have been described above, other variations and modifications will occur to those skilled in the art once the basic inventive concepts are known, and it is therefore intended that the foregoing description and drawings illustrate only embodiments of the invention and not limit the scope of the invention, and it is therefore intended that the invention not be limited to the specific embodiments described, but that the invention may be practiced with their equivalent structures or with their equivalent processes or with their use directly or indirectly in other related fields.
Claims (8)
1. The road indication display method based on mixed reality is characterized by comprising the following steps of establishing three-dimensional model information of real space scanning, and receiving virtual road mixing information, wherein the virtual road mixing information comprises virtual mixing model information and coordinate information; the virtual mixed model information is used for playing a role in road traffic indication, and comprises road signs, traffic signboards, lane lines, straight-going/turning sign marks, guardrails, isolation belts, triangular pyramids or safety islands;
editing virtual mixed model information of a specific road section, wherein the virtual mixed model information comprises a model to be displayed on the road section, display time, shape and size of the model, content displayed on the model or coordinate information of the model; transmitting the edited virtual mixed model information to a relay station or directly transmitting the edited virtual mixed model information to a vehicle-mounted terminal through a mobile communication network;
recording congestion conditions of different roads and display contents of a virtual mixed model, wherein the congestion conditions comprise time for starting congestion and time for ending congestion, the duration of displaying different contents by the virtual mixed model, and determining adjustment time intervals of optimal isolation model display positions of different road sections according to congestion information statistical results respectively classified according to weeks, dates and times;
and loading the virtual road mixing information and mixing and displaying the virtual road mixing information and the real space information.
2. The mixed reality-based road indication display method of claim 1, wherein the virtual mixed model information comprises two-dimensional mixed information or three-dimensional mixed information.
3. The road indication display method based on mixed reality according to claim 1, further comprising the step of constructing a deep learning neural network, wherein an input layer of the deep learning neural network comprises date, week, time of day, holiday, road congestion status, an output layer comprises display position information of a virtual mixed model and display content of the virtual mixed model, training is performed according to the statistical result, and finally the trained deep learning neural network capable of automatically adjusting the virtual mixed display content according to time and road status can be obtained.
4. The mixed reality-based road indication display method according to claim 1, further comprising the steps of receiving first virtual road mix information, receiving second virtual road mix information, receiving a switching display instruction of the virtual road mix information, and after receiving the switching display instruction, the display device replacing the first virtual road mix information originally displayed with the second virtual road mix information to display.
5. A mixed reality-based road indication display storage medium, characterized in that a computer program is stored, which when executed performs the steps of establishing three-dimensional model information of a real space scan, receiving virtual road mix information, the virtual road mix information including virtual mix model information and coordinate information; the virtual mixed model information is used for playing a role in road traffic indication, and comprises road signs, traffic signboards, lane lines, straight-going/turning sign marks, guardrails, isolation belts, triangular pyramids or safety islands;
editing virtual mixed model information of a specific road section, wherein the virtual mixed model information comprises a model to be displayed on the road section, display time, shape and size of the model, content displayed on the model or coordinate information of the model; transmitting the edited virtual mixed model information to a relay station or directly transmitting the edited virtual mixed model information to a vehicle-mounted terminal through a mobile communication network;
recording congestion conditions of different roads and display contents of a virtual mixed model, wherein the congestion conditions comprise time for starting congestion and time for ending congestion, the duration of displaying different contents by the virtual mixed model, and determining adjustment time intervals of optimal isolation model display positions of different road sections according to congestion information statistical results respectively classified according to weeks, dates and times; and loading the virtual road mixing information and mixing and displaying the virtual road mixing information and the real space information.
6. The mixed reality-based road indication display storage medium of claim 5, wherein the virtual mixed model information comprises two-dimensional mixed information or three-dimensional mixed information.
7. The mixed reality-based road indication display storage medium of claim 5, wherein the computer program when executed further performs the steps of constructing a deep learning neural network, the input layer of the deep learning neural network includes date, week, time of day, holiday, road congestion status, the output layer includes display position information of the virtual hybrid model, display content of the virtual hybrid model, training is performed according to the statistical result, and finally a trained deep learning neural network capable of automatically adjusting the virtual hybrid display content according to time and road status can be obtained.
8. The mixed reality-based road indication display storage medium of claim 5, wherein the computer program when executed further performs the steps of receiving first virtual road mix information, receiving second virtual road mix information, receiving a switch display instruction for the virtual road mix information, and the display device, after receiving the switch display instruction, replacing the first virtual road mix information that is originally displayed with the second virtual road mix information for display.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810484962.7A CN108665556B (en) | 2018-05-20 | 2018-05-20 | Road indication display method based on mixed reality and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810484962.7A CN108665556B (en) | 2018-05-20 | 2018-05-20 | Road indication display method based on mixed reality and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108665556A CN108665556A (en) | 2018-10-16 |
| CN108665556B true CN108665556B (en) | 2024-03-01 |
Family
ID=63776229
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810484962.7A Active CN108665556B (en) | 2018-05-20 | 2018-05-20 | Road indication display method based on mixed reality and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108665556B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114915906A (en) * | 2019-02-19 | 2022-08-16 | 福州市极化律网络科技有限公司 | Mixed reality road isolation fence adjusting method |
| CN109881605B (en) * | 2019-02-19 | 2022-08-09 | 福州市极化律网络科技有限公司 | Mixed reality road isolated column adjustment vehicle |
| CN111580516A (en) * | 2020-05-09 | 2020-08-25 | 上海木木聚枞机器人科技有限公司 | Scheduling control method, intelligent mobile device and storage medium |
| CN112037636A (en) * | 2020-09-07 | 2020-12-04 | 合肥清雅装饰工程有限公司 | Intersection road structure |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101673463A (en) * | 2009-09-17 | 2010-03-17 | 北京世纪高通科技有限公司 | Traffic information predicting method based on time series and device thereof |
| CN103473939A (en) * | 2013-10-09 | 2013-12-25 | 招商局重庆交通科研设计院有限公司 | Road signal control method and system |
| CN103489314A (en) * | 2013-09-25 | 2014-01-01 | 广东欧珀移动通信有限公司 | Method and device for displaying real-time road conditions |
| CN106251406A (en) * | 2016-08-04 | 2016-12-21 | 合肥景昇信息科技有限公司 | City based on virtual reality technology roadway location system |
| CN106652437A (en) * | 2017-03-05 | 2017-05-10 | 赵莉莉 | Comprehensive real-time command intelligent traffic management and control system |
| CN106710360A (en) * | 2016-02-03 | 2017-05-24 | 北京易驾佳信息科技有限公司 | Intelligent driving training system and method based on augment virtual reality man-machine interaction |
| CN107228681A (en) * | 2017-06-26 | 2017-10-03 | 上海驾馥电子科技有限公司 | A kind of navigation system for strengthening navigation feature by camera |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8195386B2 (en) * | 2004-09-28 | 2012-06-05 | National University Corporation Kumamoto University | Movable-body navigation information display method and movable-body navigation information display unit |
| US10040394B2 (en) * | 2015-06-17 | 2018-08-07 | Geo Semiconductor Inc. | Vehicle vision system |
-
2018
- 2018-05-20 CN CN201810484962.7A patent/CN108665556B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101673463A (en) * | 2009-09-17 | 2010-03-17 | 北京世纪高通科技有限公司 | Traffic information predicting method based on time series and device thereof |
| CN103489314A (en) * | 2013-09-25 | 2014-01-01 | 广东欧珀移动通信有限公司 | Method and device for displaying real-time road conditions |
| CN103473939A (en) * | 2013-10-09 | 2013-12-25 | 招商局重庆交通科研设计院有限公司 | Road signal control method and system |
| CN106710360A (en) * | 2016-02-03 | 2017-05-24 | 北京易驾佳信息科技有限公司 | Intelligent driving training system and method based on augment virtual reality man-machine interaction |
| CN106251406A (en) * | 2016-08-04 | 2016-12-21 | 合肥景昇信息科技有限公司 | City based on virtual reality technology roadway location system |
| CN106652437A (en) * | 2017-03-05 | 2017-05-10 | 赵莉莉 | Comprehensive real-time command intelligent traffic management and control system |
| CN107228681A (en) * | 2017-06-26 | 2017-10-03 | 上海驾馥电子科技有限公司 | A kind of navigation system for strengthening navigation feature by camera |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108665556A (en) | 2018-10-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108520635B (en) | Mixed reality road display system | |
| CN108665556B (en) | Road indication display method based on mixed reality and storage medium | |
| US11840239B2 (en) | Multiple exposure event determination | |
| CN107643073B (en) | Rear camera intersection detection | |
| US11248925B2 (en) | Augmented road line detection and display system | |
| DE102005053461B4 (en) | Method and device for traffic control | |
| DE112014006919T5 (en) | A vehicle projection control system and method for controlling image projection | |
| DE102012214988A1 (en) | Advanced reality front and rear seat vehicle gaming system for entertaining and informing passengers | |
| CN108711298B (en) | Mixed reality road display method | |
| CN111951548B (en) | Vehicle driving risk determination method, device, system and medium | |
| DE102021124913A1 (en) | METRIC BACKPROPAGATION FOR EVALUATION OF SUBSYSTEMS PERFORMANCE | |
| CN110956838A (en) | Intelligent driving method, vector map generation method, vehicle-mounted device and storage medium | |
| CN116597690B (en) | Highway test scene generation method, equipment and medium for intelligent network-connected automobile | |
| US20230159052A1 (en) | Method for processing behavior data, method for controlling autonomous vehicle, and autonomous vehicle | |
| DE112017008214T5 (en) | Shuttle route planning system | |
| CN110675476A (en) | Method and device for visually conveying definition of automatic driving scene | |
| CN112418081A (en) | Method and system for air-ground joint rapid investigation of traffic accidents | |
| CN114492022A (en) | Road condition sensing data processing method, device, equipment, program and storage medium | |
| CN108648480B (en) | Road isolation model display method based on mixed reality and storage medium | |
| CN117930673A (en) | Simulation test method and device for automatic driving vehicle, storage medium and electronic equipment | |
| EP1489552A1 (en) | Method and apparatus for improving the identification and/or re-identification of objects in image processing | |
| CN108765580B (en) | Mixed reality road display optimization method and storage medium | |
| DE102021119871B4 (en) | Method and processor circuit for operating an automated driving function with an object classifier in a motor vehicle, and motor vehicle | |
| CN116977971A (en) | Image processing method based on image recognition model and related product | |
| WO2023119462A1 (en) | Management assistance system, management assistance method, and recording medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |