CN110298912A - Reproducing method, system, electronic device and the storage medium of three-dimensional scenic - Google Patents
Reproducing method, system, electronic device and the storage medium of three-dimensional scenic Download PDFInfo
- Publication number
- CN110298912A CN110298912A CN201910392509.8A CN201910392509A CN110298912A CN 110298912 A CN110298912 A CN 110298912A CN 201910392509 A CN201910392509 A CN 201910392509A CN 110298912 A CN110298912 A CN 110298912A
- Authority
- CN
- China
- Prior art keywords
- information
- unmanned plane
- scene information
- instruction sequence
- current scene
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 31
- 238000003860 storage Methods 0.000 title claims abstract description 15
- 238000013528 artificial neural network Methods 0.000 claims description 33
- 238000013527 convolutional neural network Methods 0.000 claims description 16
- 230000015654 memory Effects 0.000 claims description 16
- 238000012549 training Methods 0.000 claims description 12
- 238000007781 pre-processing Methods 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 9
- 230000000306 recurrent effect Effects 0.000 claims description 5
- 230000008569 process Effects 0.000 claims description 3
- 210000004218 nerve net Anatomy 0.000 claims 1
- 238000012545 processing Methods 0.000 abstract description 8
- 238000004326 stimulated echo acquisition mode for imaging Methods 0.000 abstract description 7
- 230000006870 function Effects 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 230000015572 biosynthetic process Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 230000008878 coupling Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 230000010354 integration Effects 0.000 description 2
- 230000014759 maintenance of location Effects 0.000 description 2
- 230000006399 behavior Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 210000005036 nerve Anatomy 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 230000006403 short-term memory Effects 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/10—Simultaneous control of position or course in three dimensions
- G05D1/101—Simultaneous control of position or course in three dimensions specially adapted for aircraft
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Biomedical Technology (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Mathematical Physics (AREA)
- Biophysics (AREA)
- Health & Medical Sciences (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Aviation & Aerospace Engineering (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Automation & Control Theory (AREA)
- Electrically Operated Instructional Devices (AREA)
- User Interface Of Digital Computer (AREA)
- Processing Or Creating Images (AREA)
Abstract
The invention discloses a kind of reproducing method of three-dimensional scenic, system electronic device and storage mediums, it is related to technical field of image processing, for reappearing indoor three-dimensional scenic, solve the problems, such as that STEAM education utensil in the prior art cannot reappear the scene in three-dimensional space, comprising: control unmanned plane acquires indoor current scene information；Current scene information is pre-processed, pretreatment information is obtained；The pre-generated sequence of information input will be pre-processed and generate model, generate present instruction sequence；Client sends control instruction to unmanned plane using client according to present instruction sequence；After unmanned plane receives control instruction, indoor two-dimensional scene information is acquired, and two-dimensional scene information is sent to client；The scene in three-dimensional space is reconstructed and is reappeared according to two-dimensional scene information in client；And after client is to the reconstruct and reproduction of the scene in three-dimensional space, the true scene experience of learner and good learning experience can be given.
Description
Technical field
The present invention relates to technical field of image processing more particularly to a kind of reproducing methods of three-dimensional scenic, system electronic dress
It sets and storage medium.
Background technique
In recent years, STEAM education is gradually risen on the market, and STEAM is a kind of discipline of education, represents science
(Science), technology (Technology), engineering (Engineering), artistic (Arts), mathematics (Mathematics), and
STEAM itself and the English acronym of above-mentioned five kinds of representatives.STEAM education is exactly collection science, technology, engineering, skill
Art, the integrated education of mathematics Multidisciplinary Integration.
Education utensil in current STEAM education classroom, the robot product mainly artificially represented with the high machine of pleasure.
These education utensil increasingly Highgrade integrations, user can carry out visualization behaviour by the terminal interface provided in education utensil
Make, operating result will be transferred to education implement device in a manner of control instruction, and education implement device is further according to receiving
It operates control instruction and carries out corresponding scene operation.
But the mode of operation of existing education utensil, it simply facilitates and is operated using the people of education utensil, it can not be to reality
Scene in the three-dimensional space of border is reappeared, therefore cannot give the true scene experience of learner and good learning experience.
Summary of the invention
The main purpose of the present invention is to provide a kind of reproducing method of three-dimensional scenic, system electronic device and storages to be situated between
Matter, it is intended to solve STEAM in the prior art education utensil the scene in practical three-dimensional space cannot be reappeared and cannot
The technical issues of giving the true scene experience of learner and good learning experience.
To achieve the above object, first aspect present invention provides a kind of reproducing method of three-dimensional scenic, comprising: controls nobody
Machine acquires indoor current scene information, and the current scene information is what the sensor on unmanned plane was acquired in current point in time
Sensing data；The current scene information is pre-processed, pretreatment information is obtained；The pretreatment information input is pre-
The sequence first generated generates model, generates present instruction sequence, and the present instruction sequence is sent to client；Client
After receiving the present instruction sequence, control is sent to unmanned plane using client according to the present instruction sequence and is referred to
It enables；After unmanned plane receives the control instruction, indoor two-dimensional scene is acquired using unmanned plane according to the control instruction and is believed
Breath, and the two-dimensional scene information is sent to client；In client according to the two-dimensional scene information in three-dimensional space
Scene be reconstructed and reappear.
Further, it includes: preparatory control and using in unmanned plane collection room that the sequence, which generates the generation method of model,
Scene information, obtain historic scenery information, the historic scenery information is that the sensor on unmanned plane is adopted in time in the past point
The sensing data of collection, and unmanned plane is recorded at this moment by the received sample instruction sequence of control institute；The historic scenery is believed
Breath is pre-processed, and obtains pretreated historic scenery information, and use pretreated historic scenery information training convolutional nerve
Network, wherein the pretreatment includes standardization and vectorization；By the sample instruction sequence carry out vectorization, and use to
The sample instruction sequence training first circulation neural network of quantization；Using second circulation neural network by the convolutional neural networks
And the first circulation neural network combines, so that after the historic scenery information and the sample instruction sequence match,
It obtains sequence and generates model, the sequence generates model and is used for after receiving the current scene information, current scene is believed
Breath matches with historic scenery information, and using the sample instruction sequence to match with historic scenery information as the current scene
The present instruction sequence that information matches.
Further, described to pre-process the scene information, obtaining pretreatment information includes: to believe the scene
Breath is standardized, and obtains standardized information；The standardized information is subjected to vectorization, obtains pretreatment information.
Further, it is described control and using unmanned plane acquire indoor current scene information include: using unmanned plane from
First scene of band obtains sensor and acquires indoor first current scene information；It is obtained using the second scene of unmanned plane carry
Sensor acquires indoor second current scene information.
Second aspect of the present invention provides a kind of recurrence system of three-dimensional scenic, comprising: current scene information acquisition module is used
Indoor current scene information is acquired in control unmanned plane, the current scene information is the sensor on unmanned plane when current
Between put the sensing data of acquisition；Preprocessing module, the current scene for being acquired to the current scene information acquisition module
Information is pre-processed, and pretreatment information is obtained；Instruction sequence generation module, the pre- place for obtaining the preprocessing module
Reason information is transmitted to sequence generating module, and instruction sequence generation module is sent according to the instruction sequence that pretreatment information generates
To client；Instruction sending module, instruction sequence for being generated in client according to described instruction sequence generating module is to nothing
Man-machine transmission control instruction, to control unmanned plane；Two-dimensional scene acquisition module, for using unmanned plane to be sent according to described instruction
The indoor two-dimensional scene information of instruction acquisition that module is sent, and two-dimensional scene information is sent to client；Recurrent modules are used
The scene in three-dimensional space is reconstructed and is reappeared in the two-dimensional scene information acquired according to the two-dimensional scene acquisition module.
Further, the sequence generating module includes: historic scenery information acquisition unit, for controlling and using in advance
Unmanned plane acquires indoor scene information, and the historic scenery information is what the sensor on unmanned plane was acquired in time in the past point
Sensing data obtains historic scenery information, and records unmanned plane at this moment by the received sample instruction sequence of control institute；Pre- place
Unit is managed, the historic scenery information for acquiring the historic scenery information acquisition unit is pre-processed, pre-processed
Historic scenery information, and use pretreated historic scenery information training convolutional neural networks, wherein the pretreatment includes
Standardization and vectorization；Vectorization unit, sample instruction sequence for recording the historic scenery information acquisition unit into
Row vector, and use the sample instruction sequence of vectorization training first circulation neural network；Series model generation unit, is used for
Using second circulation neural network by the first circulation neural network of the convolutional neural networks of pretreatment unit and vectorization unit
It is combined, so that the historic scenery information in the acquisition of historic scenery information acquisition unit matches with the sample instruction sequence
Afterwards, it obtains sequence and generates model, the sequence generates model and is used to receive the current scene information acquisition module acquisition
Current scene information after, current scene information and historic scenery information are matched, and will match with historic scenery information
The present instruction sequence that matches as the current scene information of sample instruction sequence.
Further, the preprocessing module includes: Standardisation Cell, is used for the current scene information acquisition module
The current scene information of acquisition is standardized, and obtains standardized information；Vectorization unit, for obtaining the Standardisation Cell
The standardized information arrived carries out vectorization, obtains pretreatment information.
Further, the current scene information acquisition module includes: the first current scene information acquisition unit, for making
Sensor, which is obtained, with the first scene that unmanned plane carries acquires indoor first current scene information；Second current scene information is adopted
Collect unit, acquires indoor second current scene information for using the second scene of unmanned plane carry to obtain sensor.
Third aspect present invention provides a kind of electronic device, comprising: memory, processor and is stored on the memory
And the computer program that can be run on the processor, when the processor executes the computer program, realize among the above
Any one the method.
Fourth aspect present invention provides a kind of computer readable storage medium, is stored thereon with computer program, the meter
When calculation machine program is executed by processor, any one the method among the above is realized.
The present invention provides the reproducing method, system electronic device and storage medium of a kind of three-dimensional scenic, and beneficial effect is:
The current scene information for acquiring unmanned plane pre-processed obtain pretreatment information after, mould can be generated by sequence
Type, the position of Lai Dingwei unmanned plane in three dimensions enable a client to generate present instruction sequence according to instruction
Sequence sends control instruction to unmanned plane, so that unmanned plane can acquire indoor two-dimensional scene letter based on current location
Breath, and client then can reconstruct and reproduction according to two-dimensional scene information realization to the scene in three-dimensional space, so as to
Enough give the true scene experience of learner and good learning experience.
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
Some embodiments of invention for those skilled in the art without creative efforts, can also basis
These attached drawings obtain other attached drawings.
Fig. 1 is the schematic process flow diagram of the reproducing method of three-dimensional scenic of the embodiment of the present invention；
Fig. 2 is the structural schematic block diagram of the recurrence system of three-dimensional scenic of the embodiment of the present invention；
Fig. 3 is the structural schematic block diagram of electronic device of the embodiment of the present invention.
Specific embodiment
In order to make the invention's purpose, features and advantages of the invention more obvious and easy to understand, below in conjunction with the present invention
Attached drawing in embodiment, technical scheme in the embodiment of the invention is clearly and completely described, it is clear that described reality
Applying example is only a part of the embodiment of the present invention, and not all embodiments.Based on the embodiments of the present invention, those skilled in the art
Member's every other embodiment obtained without making creative work, shall fall within the protection scope of the present invention.
Referring to Fig. 1, being a kind of reproducing method of three-dimensional scenic, comprising: S1, control unmanned plane acquisition indoor work as front court
Scape information, current scene information are the sensing data that the sensor on unmanned plane is acquired in current point in time；S2, front court will be worked as
Scape information is pre-processed, and pretreatment information is obtained；S3, the pre-generated sequence generation model of information input will be pre-processed, it is raw
Client is sent at present instruction sequence, and by present instruction sequence；S4, client after receiving present instruction sequence,
Control instruction is sent to unmanned plane using client according to present instruction sequence；After unmanned plane receives control instruction, according to control
System instruction acquires indoor two-dimensional scene information using unmanned plane, and two-dimensional scene information is sent to client；S5, in client
End is reconstructed and reappears to the scene in three-dimensional space according to two-dimensional scene information.
The generation method that sequence generates model includes: preparatory control and acquires indoor scene information using unmanned plane, is obtained
To historic scenery information, historic scenery information is the sensing data that the sensor on unmanned plane is acquired in time in the past point, and
Unmanned plane is recorded at this moment by the received sample instruction sequence of control institute；Historic scenery information is pre-processed, pre- place is obtained
The historic scenery information of reason, and use pretreated historic scenery information training convolutional neural networks, wherein pretreatment includes mark
Standardization and vectorization；Sample instruction sequence is subjected to vectorization, and uses the sample instruction sequence of vectorization training first circulation
Neural network；Convolutional neural networks and first circulation neural network are combined using second circulation neural network, so that in history
After scene information and sample instruction sequence match, obtains sequence and generate model, sequence generates model and is used to receive currently
After scene information, current scene information and historic scenery information are matched, and will be matched with historic scenery information sample
The present instruction sequence that instruction sequence matches as current scene information.
When controlling the unmanned plane indoor historic scenery information of acquisition, in the present embodiment, by using client indoors
End sends instruction sequence to manipulate unmanned plane, such as operation unmanned plane is using moving back and forth forward or backward, or upwards to
Under move back and forth etc. and to generate sensor data information, and instruction sequence is preserved with corresponding sensor data information, and
Instruction sequence at this time is denoted as sample instruction sequence；In other embodiments, unmanned plane is changed by hand-held unmanned plane
Position so that the sensor on unmanned plane generates sensor data information, and is demarcated sensor data information, i.e.,
The corresponding relationship of calibration sensor data information and instruction sequence, and instruction sequence at this time is denoted as sample instruction sequence.
In the present embodiment, sequence is generated using first circulation neural network convolutional neural networks and second circulation neural network
Column-generation model；First circulation neural network and second circulation neural network are Recognition with Recurrent Neural Network, Recognition with Recurrent Neural Network English
Literary fame is known as Long Short-Term Memory Networks, and English name is abbreviated as LSTM；The English of convolutional neural networks
Entitled Convolutional Neural Networks, English name are abbreviated as CNN；It is carried out in advance to historic scenery information
When processing, historic scenery information is exported in CNN and carries out feature extraction, is carried out after being encoded sample instruction sequence data
Word is embedded in vectorization, and is output in first circulation neural network and extracts, and finally mentions CNN and first circulation neural network
The feature taken is input in second circulation neural network and is learnt after being attached matching, enables second circulation neural network
It is enough that condition generates instructions corresponding thereto sequence based on sensor data information, so that obtaining sequence generates model, sequence is raw
The sensor data information represented in the current scene information that can be acquired based on unmanned plane at model generates driving unmanned plane
Present instruction sequence.
In other embodiments, model is generated by confrontation neural network formation sequence；Fight neural network English name
For generative adversarial nets, English name is abbreviated as GAN, and specifically, confrontation neural network uses
DiscoGAN；During generating model using confrontation neural network formation sequence, the biography that first represents historic scenery information
Sensor data information is mapped to image, obtains sensor image, is mapped as image after then being encoded sample instruction sequence,
Instruction image is obtained, then will be learnt in sensor image and instruction image input confrontation neural network, so that confrontation
Neural network can be from completion from sensor image to the mapping instruction image, so that formation sequence generates model, sequence
It generates the sensor data information represented in the current scene information that model can be acquired based on unmanned plane and generates driving unmanned plane
Present instruction sequence.
In other embodiments, if the sensor on indoor scene more dullness and unmanned plane is TOF sensor, make
The TOF generated when executing instruction sequence to unmanned plane with the time series feature extracting method on data-signal processing technique away from
Extraction from the characteristic point that delta data carries out crest value and valley value, and the characteristic point extracted is screened accordingly
Filtration treatment is finally completed so that is instructed executes the judgement of item number and unmanned plane during flying trend by TOF sensor data
Driving generates the task of unmanned instruction sequence, and specifically, TOF sensor is time-of-flight sensor.
Scene information is pre-processed, obtaining pretreatment information includes: to be standardized scene information, obtains standard
Change information；Standardized information is subjected to vectorization, obtains pretreatment information.
Controlling and acquiring indoor current scene information using unmanned plane includes: to be obtained using the first scene that unmanned plane carries
Sensor is taken to acquire indoor first current scene information；It is obtained in sensor collection room using the second scene of unmanned plane carry
The second current scene information.
In the present embodiment, due in a practical situation, the number of sensors that unmanned plane itself carries be it is fixed, in order to
Unmanned plane is enabled to acquire more galore current scene information, then the carry sensor on unmanned plane, unmanned plane itself is certainly
The sensor of band is denoted as the first scene and obtains sensor, and sensor of the carry on unmanned plane is denoted as the second scene and obtains sensing
Device；And for the ease of obtaining the data between sensor, the second scene acquisition sensor, unmanned plane and client to the first scene
Transmission, the carry wireless relay module on unmanned plane, client, unmanned plane are wirelessly connected with wireless relay module, and second
Scape obtains sensor and is electrically connected with wireless relay module, and when carrying out data transmission, instruction sequence is sent to wirelessly by client
Instruction sequence is forwarded to unmanned plane by trunk module, wireless relay module, so that the flight of unmanned plane is controlled, and on unmanned plane
First scene obtains sensor and the second scene obtains sensor, then respectively works as the first current scene information of acquisition and second
Preceding scene information is sent to client by wireless relay module, and the status information of unmanned plane is sent also by wireless relay module
To client.
Specifically, the software in client can need to carry out according to scene the application development of respective version, therefore
Client can be the application program on computer and computer, can also be the APP software in mobile device and mobile device；In wireless
Have ap mode (Access Point) and Station mode, i.e. offer wirelessly access after module, permission is other wirelessly to be set
Standby access, while itself also may be coupled to the AP of other equipment, must have certain data processing function, that is, have
Therefore wireless relay module selects ESP8266 or ESP32 model to have WIFI function to the communication interfaces such as serial ports either IIC or SPI
The wireless module and its extension facility of energy and data processing function；Unmanned plane has the api interface used for secondary development, tool
Standby ap mode radio network functions, therefore unmanned plane is the small drone of Tello model；The sensor of carry on unmanned plane,
That is the second scene, which obtains sensor, to be had and can be transmitted the data of itself by communication modes such as serial ports either IIC, or SPI
To the interface of wireless relay module, therefore it is TOF sensor that the second scene, which obtains sensor,.
Referring to Fig. 2, the embodiment of the present application also provides a kind of recurrence system of three-dimensional scenic, which includes: to work as front court
Scape information acquisition module, preprocessing module, instruction sequence generation module, instruction sending module, two-dimensional scene acquisition module and multiple
Existing module；Current scene information acquisition module acquires indoor current scene information, current scene information for controlling unmanned plane
The sensing data acquired for the sensor on unmanned plane in current point in time；Preprocessing module is for adopting current scene information
The current scene information of collection module acquisition is pre-processed, and pretreatment information is obtained；Instruction sequence generation module for that will locate in advance
The pretreatment information that reason module obtains is transmitted to sequence generating module, and instruction sequence generation module is raw according to pretreatment information
At instruction sequence be sent to client；Instruction sending module is used for the finger generated in client according to instruction sequence generation module
Sequence is enabled to send control instruction to unmanned plane, to control unmanned plane；Two-dimensional scene acquisition module is used for using unmanned plane according to finger
The indoor two-dimensional scene information of instruction acquisition for enabling sending module send, and two-dimensional scene information is sent to client；Reproduction
The two-dimensional scene information that module is used to be acquired according to two-dimensional scene acquisition module is reconstructed the scene in three-dimensional space and answers
It is existing.
Sequence generating module includes: historic scenery information acquisition unit, pretreatment unit, vectorization unit and series model
Generation unit；Historic scenery information acquisition unit, for controlling in advance and acquiring indoor scene information, history using unmanned plane
Scene information is the sensing data that the sensor on unmanned plane is acquired in time in the past point, obtains historic scenery information, and remember
Unmanned plane is recorded at this moment by the received sample instruction sequence of control institute；Pretreatment unit is used for historic scenery information collection list
The historic scenery information of member acquisition is pre-processed, and obtains pretreated historic scenery information, and use pretreated history field
Scape information training convolutional neural networks, wherein pretreatment includes standardization and vectorization；Vectorization unit is used for history field
The sample instruction sequence of scape information acquisition unit record carries out vectorization, and uses the sample instruction sequence of vectorization training first
Recognition with Recurrent Neural Network；Series model generation unit, for using second circulation neural network by the convolutional Neural of pretreatment unit
The first circulation neural network of network and vectorization unit is combined, so that going through in the acquisition of historic scenery information acquisition unit
It after history scene information and sample instruction sequence match, obtains sequence and generates model, sequence generates model and is used to work as receiving
After the current scene information of preceding scene information acquisition module acquisition, current scene information and historic scenery information are matched, and
The present instruction sequence that the sample instruction sequence to match with historic scenery information is matched as current scene information.
Preprocessing module includes: Standardisation Cell and vectorization unit；Standardisation Cell, for adopting current scene information
The current scene information of collection module acquisition is standardized, and obtains standardized information；Vectorization unit is used for Standardisation Cell
Obtained standardized information carries out vectorization, obtains pretreatment information.
Current scene information acquisition module includes: that the first current scene information acquisition unit and the second current scene information are adopted
Collect unit；First current scene information acquisition unit is used to obtain in sensor collection room using the first scene that unmanned plane carries
The first current scene information；Second current scene information acquisition unit is used to be obtained using the second scene of unmanned plane carry and be passed
Sensor acquires indoor second current scene information.
3 are please referred to, the embodiment of the present application also provides a kind of electronic device, which includes: memory 601, processing
Device 602 and it is stored in the computer program that can be run on memory 601 and on processor 602, processor 602 executes the calculating
When machine program, realize it is aforementioned described in three-dimensional scenic reproducing method.
Further, electronic device further include: at least one input equipment 603 and at least one output equipment 604.
Above-mentioned memory 601, processor 602, input equipment 603 and output equipment 604, are connected by bus 605.
Wherein, input equipment 603 concretely camera, touch panel, physical button or mouse etc..Output equipment
604 concretely display screens.
Memory 601 can be high random access memory body (RAM, Random Access Memory) memory,
It can be non-labile memory (non-volatile memory), such as magnetic disk storage.Memory 601 is for storing one
Group executable program code, processor 602 are coupled with memory 601.
Further, the embodiment of the present application also provides a kind of computer readable storage medium, the computer-readable storages
Medium can be in the electronic device being set in the various embodiments described above, which can be in aforementioned
Memory 601.It is stored with computer program on the computer readable storage medium, which realizes when being executed by processor 602
The reproducing method of three-dimensional scenic described in previous embodiment.
Further, the computer can storage medium can also be USB flash disk, mobile hard disk, read-only memory 601 (ROM,
Read-Only Memory), RAM, the various media that can store program code such as magnetic or disk.
In several embodiments provided herein, it should be understood that disclosed device and method can pass through it
Its mode is realized.For example, the apparatus embodiments described above are merely exemplary, for example, the division of the module, only
Only a kind of logical function partition, there may be another division manner in actual implementation, such as multiple module or components can be tied
Another system is closed or is desirably integrated into, or some features can be ignored or not executed.Another point, it is shown or discussed
Mutual coupling, direct-coupling or communication connection can be through some interfaces, the INDIRECT COUPLING or logical of device or module
Letter connection can be electrical property, mechanical or other forms.
The module as illustrated by the separation member may or may not be physically separated, aobvious as module
The component shown may or may not be physical module, it can and it is in one place, or may be distributed over multiple
On network module.Some or all of the modules therein can be selected to realize the mesh of this embodiment scheme according to the actual needs
's.
It, can also be in addition, each functional module in each embodiment of the present invention can integrate in a processing module
It is that modules physically exist alone, can also be integrated in two or more modules in a module.Above-mentioned integrated mould
Block both can take the form of hardware realization, can also be realized in the form of software function module.
If the integrated module is realized in the form of software function module and sells or use as independent product
When, it can store in a computer readable storage medium.
It should be noted that for the various method embodiments described above, describing for simplicity, therefore, it is stated as a series of
Combination of actions, but those skilled in the art should understand that, the present invention is not limited by the sequence of acts described because
According to the present invention, certain steps can use other sequences or carry out simultaneously.Secondly, those skilled in the art should also know
It knows, the embodiments described in the specification are all preferred embodiments, and related actions and modules might not all be this hair
Necessary to bright.
In the above-described embodiments, it all emphasizes particularly on different fields to the description of each embodiment, there is no the portion being described in detail in some embodiment
Point, it may refer to the associated description of other embodiments.
The above are to a kind of reproducing method of three-dimensional scenic provided by the present invention, system electronic device and storage medium
Description, for those skilled in the art, thought according to an embodiment of the present invention, in specific embodiments and applications
It will change, to sum up, the contents of this specification are not to be construed as limiting the invention.
Claims (10)
1. a kind of reproducing method of three-dimensional scenic characterized by comprising
It controls unmanned plane and acquires indoor current scene information, the current scene information is the sensor on unmanned plane current
The sensing data of time point acquisition；
The current scene information is pre-processed, pretreatment information is obtained；
The pre-generated sequence of the pretreatment information input is generated into model, generates present instruction sequence, and will be described current
Instruction sequence is sent to client；
Client sends out unmanned plane using client after receiving the present instruction sequence, according to the present instruction sequence
Send control instruction；After unmanned plane receives the control instruction, indoor two are acquired using unmanned plane according to the control instruction
Scene information is tieed up, and the two-dimensional scene information is sent to client；
The scene in three-dimensional space is reconstructed and is reappeared according to the two-dimensional scene information in client.
2. the reproducing method of three-dimensional scenic according to claim 1, which is characterized in that
The generation method that the sequence generates model includes:
It controls in advance and acquires indoor scene information using unmanned plane, obtain historic scenery information, the historic scenery information
For the sensing data that the sensor on unmanned plane is acquired in time in the past point, and records unmanned plane and controlled at this moment and received
Sample instruction sequence；
The historic scenery information is pre-processed, and uses pretreated historic scenery information training convolutional neural networks,
Wherein, the pretreatment includes standardization and vectorization；
The sample instruction sequence is subjected to vectorization, and uses the sample instruction sequence of vectorization training first circulation nerve net
Network；
The convolutional neural networks and the first circulation neural network are combined using second circulation neural network, so that in institute
It states historic scenery information and after the sample instruction sequence matches, obtains sequence and generate model, the sequence generates model and uses
In after receiving the current scene information, current scene information and historic scenery information are matched, and will be with history field
The present instruction sequence that the sample instruction sequence that scape information matches matches as the current scene information.
3. the reproducing method of three-dimensional scenic according to claim 1, which is characterized in that
Described to pre-process the scene information, obtaining pretreatment information includes:
The scene information is standardized, standardized information is obtained；
The standardized information is subjected to vectorization, obtains pretreatment information.
4. the reproducing method of three-dimensional scenic according to claim 1, which is characterized in that
The control simultaneously acquires indoor current scene information using unmanned plane and includes:
Sensor, which is obtained, using the first scene that unmanned plane carries acquires indoor first current scene information；
Sensor, which is obtained, using the second scene of unmanned plane carry acquires indoor second current scene information.
5. a kind of recurrence system of three-dimensional scenic characterized by comprising
Current scene information acquisition module acquires indoor current scene information, the current scene letter for controlling unmanned plane
Breath is the sensing data that the sensor on unmanned plane is acquired in current point in time；
Preprocessing module, the current scene information for acquiring to the current scene information acquisition module are pre-processed, are obtained
To pretreatment information；
Instruction sequence generation module, the pretreatment information for obtaining the preprocessing module are transmitted to sequence generating module,
And instruction sequence generation module is sent to client according to the instruction sequence that pretreatment information generates；
Instruction sending module, for being sent out according to the instruction sequence that described instruction sequence generating module generates to unmanned plane in client
Control instruction is sent, to control unmanned plane；
Two-dimensional scene acquisition module, the instruction acquisition indoor two for using unmanned plane to be sent according to described instruction sending module
Scene information is tieed up, and two-dimensional scene information is sent to client；
Recurrent modules, the two-dimensional scene information for being acquired according to the two-dimensional scene acquisition module is to the scene in three-dimensional space
It is reconstructed and reappears.
6. the recurrence system of three-dimensional scenic according to claim 5, which is characterized in that
The sequence generating module includes:
Historic scenery information acquisition unit, for controlling in advance and acquiring indoor scene information, the history using unmanned plane
Scene information is the sensing data that the sensor on unmanned plane is acquired in time in the past point, obtains historic scenery information, and remember
Unmanned plane is recorded at this moment by the received sample instruction sequence of control institute；
Pretreatment unit, the historic scenery information for acquiring the historic scenery information acquisition unit are pre-processed, are obtained
To pretreated historic scenery information, and use pretreated historic scenery information training convolutional neural networks, wherein described pre-
Processing includes standardization and vectorization；
Vectorization unit, the sample instruction sequence for recording the historic scenery information acquisition unit carry out vectorization, and
Use the sample instruction sequence training first circulation neural network of vectorization；
Series model generation unit, for using convolutional neural networks and vector of the second circulation neural network by pretreatment unit
The first circulation neural network for changing unit is combined, so that in the historic scenery information of historic scenery information acquisition unit acquisition
After matching with the sample instruction sequence, obtain sequence and generate model, the sequence generate model be used for receive it is described
After the current scene information of current scene information acquisition module acquisition, current scene information and historic scenery information are matched,
And the present instruction sequence that the sample instruction sequence to match with historic scenery information matches as the current scene information
Column.
7. the recurrence system of three-dimensional scenic according to claim 5, which is characterized in that
The preprocessing module includes:
Standardisation Cell, the current scene information for acquiring the current scene information acquisition module are standardized, obtain
To standardized information；
Vectorization unit, the standardized information for obtaining the Standardisation Cell carry out vectorization, obtain pretreatment information.
8. the recurrence system of three-dimensional scenic according to claim 5., which is characterized in that
The current scene information acquisition module includes:
First current scene information acquisition unit, it is indoor that the first scene for using unmanned plane included obtains sensor acquisition
First current scene information；
Second current scene information acquisition unit, it is indoor for using the second scene of unmanned plane carry to obtain sensor acquisition
Second current scene information.
9. a kind of electronic device, comprising: memory, processor and be stored on the memory and can transport on the processor
Capable computer program, which is characterized in that when the processor executes the computer program, realize in Claims 1-4
Any one the method.
10. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the computer program
When being executed by processor, any one the method in Claims 1-4 is realized.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910392509.8A CN110298912B (en) | 2019-05-13 | 2019-05-13 | Reproduction method, reproduction system, electronic device and storage medium for three-dimensional scene |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910392509.8A CN110298912B (en) | 2019-05-13 | 2019-05-13 | Reproduction method, reproduction system, electronic device and storage medium for three-dimensional scene |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110298912A true CN110298912A (en) | 2019-10-01 |
| CN110298912B CN110298912B (en) | 2023-06-27 |
Family
ID=68026893
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910392509.8A Active CN110298912B (en) | 2019-05-13 | 2019-05-13 | Reproduction method, reproduction system, electronic device and storage medium for three-dimensional scene |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110298912B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112146660A (en) * | 2020-09-25 | 2020-12-29 | 电子科技大学 | Indoor map positioning method based on dynamic word vector |
| CN112529248A (en) * | 2020-11-09 | 2021-03-19 | 北京宇航系统工程研究所 | Data-driven intelligent flight space-ground mirror image system of carrier rocket |
| CN114078347A (en) * | 2021-11-24 | 2022-02-22 | 武汉小绿人动力技术股份有限公司 | Novel juvenile STEAM education system and method |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107707900A (en) * | 2017-10-17 | 2018-02-16 | 西安万像电子科技有限公司 | Processing method, the device and system of content of multimedia |
| WO2018119676A1 (en) * | 2016-12-27 | 2018-07-05 | 深圳前海达闼云端智能科技有限公司 | Display data processing method and apparatus |
| WO2018119889A1 (en) * | 2016-12-29 | 2018-07-05 | 深圳前海达闼云端智能科技有限公司 | Three-dimensional scene positioning method and device |
| CN109407679A (en) * | 2018-12-28 | 2019-03-01 | 百度在线网络技术（北京）有限公司 | Method and apparatus for controlling pilotless automobile |
-
2019
- 2019-05-13 CN CN201910392509.8A patent/CN110298912B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018119676A1 (en) * | 2016-12-27 | 2018-07-05 | 深圳前海达闼云端智能科技有限公司 | Display data processing method and apparatus |
| WO2018119889A1 (en) * | 2016-12-29 | 2018-07-05 | 深圳前海达闼云端智能科技有限公司 | Three-dimensional scene positioning method and device |
| CN107707900A (en) * | 2017-10-17 | 2018-02-16 | 西安万像电子科技有限公司 | Processing method, the device and system of content of multimedia |
| CN109407679A (en) * | 2018-12-28 | 2019-03-01 | 百度在线网络技术（北京）有限公司 | Method and apparatus for controlling pilotless automobile |
Non-Patent Citations (2)
| Title |
|---|
| 李康等: "基于无人机的大场景序列图像自动采集和三维建模", 《西北大学学报(自然科学版)》 * |
| 闫阳阳等: "三维激光点云联合无人机影像的三维场景重建研究", 《测绘通报》 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112146660A (en) * | 2020-09-25 | 2020-12-29 | 电子科技大学 | Indoor map positioning method based on dynamic word vector |
| CN112529248A (en) * | 2020-11-09 | 2021-03-19 | 北京宇航系统工程研究所 | Data-driven intelligent flight space-ground mirror image system of carrier rocket |
| CN112529248B (en) * | 2020-11-09 | 2024-06-04 | 北京宇航系统工程研究所 | Data-driven intelligent flying space mirror image system of carrier rocket |
| CN114078347A (en) * | 2021-11-24 | 2022-02-22 | 武汉小绿人动力技术股份有限公司 | Novel juvenile STEAM education system and method |
| CN114078347B (en) * | 2021-11-24 | 2024-03-22 | 武汉小绿人动力技术股份有限公司 | Teenager STEAM education system and method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110298912B (en) | 2023-06-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105912980B (en) | Unmanned plane and UAV system | |
| KR102288280B1 (en) | Device and method to generate image using image learning model | |
| CN110298912A (en) | Reproducing method, system, electronic device and the storage medium of three-dimensional scenic | |
| CN109685202A (en) | Data processing method and device, storage medium and electronic device | |
| WO2018054330A1 (en) | Method and device for data processing and storage medium | |
| US11615516B2 (en) | Image-to-image translation using unpaired data for supervised learning | |
| CN110349232B (en) | Image generation method and device, storage medium and electronic equipment | |
| CN106485956A (en) | Method and device for demonstrating functions of electronic equipment and intelligent terminal | |
| CN110751721B (en) | Furniture layout drawing generation method and device, computer equipment and storage medium | |
| CN109035415B (en) | Virtual model processing method, device, equipment and computer readable storage medium | |
| CN105074615A (en) | Virtual sensor systems and methods | |
| CN106161939A (en) | A kind of method, photo taking and terminal | |
| CN112562019A (en) | Image color adjusting method and device, computer readable medium and electronic equipment | |
| CN110321761A (en) | A kind of Activity recognition method, terminal device and computer readable storage medium | |
| CN110427100A (en) | A kind of movement posture capture system based on depth camera | |
| CN108900788A (en) | Video generation method, video-generating device, electronic device and storage medium | |
| CN110149551A (en) | Media file playing method and device, storage medium and electronic device | |
| CN114360018B (en) | Rendering method and device of three-dimensional facial expression, storage medium and electronic device | |
| CN111124902A (en) | Object operating method and device, computer-readable storage medium and electronic device | |
| CN109819001A (en) | A kind of communication device of UAV Communication method, unmanned plane and unmanned plane | |
| CN109407826A (en) | Ball game analogy method, device, storage medium and electronic equipment | |
| CN115690592B (en) | Image processing method and model training method | |
| CN110443852A (en) | A kind of method and relevant apparatus of framing | |
| CN110827394A (en) | Facial expression construction method and device and non-transitory computer readable recording medium | |
| CN109858380A (en) | Expansible gesture identification method, device, system, gesture identification terminal and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |