CN210093397U

CN210093397U - City intelligent virtual imaging device and imaging system thereof

Info

Publication number: CN210093397U
Application number: CN201920928498.6U
Authority: CN
Inventors: 李少峰
Original assignee: Jinling Institute of Technology
Current assignee: Jinling Institute of Technology
Priority date: 2019-06-20
Filing date: 2019-06-20
Publication date: 2020-02-18
Anticipated expiration: 2029-06-20

Abstract

The utility model discloses a virtual imaging device of city intelligence and imaging system thereof belongs to three-dimensional virtual formation of image field. The equipment comprises a shooting device, an image processor and a projection device, wherein the shooting device comprises a rotating holder, a CCD (charge coupled device) camera unit arranged below the rotating holder, and a reflector which is arranged on the rotating holder and forms an included angle of 45 degrees with the lens direction of the CCD camera unit. An imaging system consisting of a data acquisition module, an image processing module, an intelligent fusion module and a holographic imaging module is established on the basis of the equipment. The utility model discloses select single camera ring scan's mode and adopt the rotation that rotatory reflector replaced the camera, acquire 360 panoramic picture, improved greatly and obtained original image in every cyclic scanning period and had the advantage that high resolution, all directions are to distortionless, the amount of calculation of the later stage image processing that reduces greatly.

Description

City intelligent virtual imaging device and imaging system thereof

Technical Field

The utility model belongs to city three-dimensional scene field of modelling, especially a virtual imaging device of city intelligence and imaging system thereof.

Background

The virtual imaging technology is a method for human interaction with a computer and complex data thereof, the core of the virtual imaging technology lies in modeling and drawing of a real virtual environment consisting of three-dimensional scenes, and the problem of modeling around the virtual scenes in the world mainly adopts the following two modes: firstly, the method is based on geometric modeling and drawing on the basis of traditional computer graphics, and has higher requirements on hardware and algorithm of a computer; secondly, a scene is established by image acquisition of a three-dimensional environment, so that the calculation processing process is intentionally weakened, and the requirements of the shooting process of the three-dimensional image are more emphasized.

In the second mode in virtual scene modeling, a super-large scale staring imaging device is required to be adopted for acquiring a three-dimensional image to realize imaging of a large visual angle, the optical system can realize a normal imaging observation function only by matching with a series of image processing technologies such as image distortion correction, light intensity correction, sub-pixel resolution and the like, and a large amount of image data processing amount still has higher performance requirements on an image processor and software. When the virtually imaged image is enlarged to a scene in an oversized area such as a city or a scenic spot, the problem is further enlarged by a huge image data processing amount.

SUMMERY OF THE UTILITY MODEL

Utility model purpose: the urban intelligent virtual imaging equipment and the imaging system thereof are provided to solve the problems of high hardware cost, large software data processing calculation amount, poor instantaneity and the like in the existing modeling method.

The technical scheme is as follows: a city intelligent virtual imaging device, comprising: the device comprises a shooting device, an image processor and a projection device.

The shooting device is used for scanning by adopting a single CCD camera adopting a fisheye lens, and comprises a rotating cloud platform fixedly installed on an unmanned aerial vehicle, a helicopter, a designated area or an on-vehicle platform, a CCD camera unit vertically placed below the rotating cloud platform, and a reflector fixedly installed on the rotating cloud platform and forming a 45-degree included angle with the lens direction of the CCD camera unit.

The image processor is mainly a computer processor and comprises a high-performance computer used for the initial template of the first scene modeling image, a common PC terminal used for the later image processing, and a WiFi communicator.

Projection arrangement, for the VR glasses of immersive experience, it includes the glasses support of laminating mutually with person's of the wearer's face surface, fixes two convex lens lenses on the glasses support, with the smart mobile phone support that leaves predetermined clearance between the glasses support, be located bottom junction between glasses support and the smart mobile phone support, be used for placing in the cell-phone standing groove of the smart mobile phone on the smart mobile phone support, and connect between glasses support and the smart mobile phone support, be used for adjusting the regulation hasp of reserving the distance between the two.

In a further implementation, the rotating head comprises: the robot comprises a frame, a chassis, a motor mounting seat, a servo motor, a crankshaft, a motor shaft, a motor mounting seat, a servo motor, a crank shaft and a motor shaft, wherein the frame consists of a plurality of sectional materials and is fixed with the unmanned aerial vehicle, the helicopter, a designated area or a vehicle-mounted platform; with the connecting rod that the bent axle is connected, fixed mounting be in the first fixed frame of motor mount pad both sides, through round pin axle with the fixed frame of second of connecting is rotated to the bottom of first fixed frame, fixed mounting in the fixed frame of second, and run through the gag lever post of connecting rod, and with connecting rod fixed connection's vacuum chuck. The reflector is fixedly arranged on the vacuum chuck, and the fixed position of the reflector is located at the intersection point of the extension line of the CCD camera unit and the rotating shaft.

In a further implementation process, the projection device can also be a screen type VR projection, which can be a desktop computer or a large screen display, wherein a panorama for sightseeing experience can be a projection hall, and the panorama comprises a projection room, a holographic projection film covering the inside of the wall of the projection room, and a spherical multi-angle projection unit arranged at the center of the projection room in the morning; wherein the projection unit is provided with at least six projectors which respectively face six surfaces of the projection chamber.

A city intelligent virtual imaging system, comprising: the system comprises a data acquisition module, an image processing module, an intelligent fusion module and a holographic imaging module.

The data acquisition module comprises an aerial long-range camera device for acquiring multi-angle inclined multi-angle images of a city to be detected by using an unmanned aerial vehicle or a helicopter, and a close-range camera device fixedly distributed in a specified area of a street and acquiring peripheral ground images by using vehicle-mounted movement.

The image processing module comprises an image splicing module which adopts a series of image splicing technologies of correcting a rotation angle, cylindrical projection and translation alignment, an image initial template manufactured by the image splicing module, an image storage unit for comparing the image initial template, an image processing module and an image processing module, wherein the image information is obtained by the rotation shooting of the far-view camera device and the near-view camera device in a relatively static state in a first scanning period, the image splicing module adopts the image splicing technology of correcting the rotation angle, cylindrical projection and translation alignment, the image initial template manufactured by the image splicing module is used for acquiring real-time new image information through the near-view camera device fixed in a specified area of a street, the same feature vector is compared with the image initial template for.

The intelligent fusion module comprises a scene simulation device for special effect information of three-dimensional terrain and weather information fused and constructed for real-time images, a close-range shooting device arranged inside a key building, an internal building imaging device fused to the real-time images through the image processing module, and an information model construction device for configuring the real-time images, geographic information, position information and space category information, and the information model construction device is used for storing audio information.

The holographic imaging module can be specifically three modes of immersion-type experience VR glasses, desktop VR images or screen VR projections.

Has the advantages that: the utility model relates to a virtual imaging device of city intelligence, imaging system and modeling method thereof has following advantage:

1) in the aspect of shooting equipment, a single camera annular scanning mode is selected to obtain a 360-degree panoramic image, and an original image obtained in each cycle scanning period has the characteristics of high resolution and no distortion in all directions; the rotating reflector replaces the rotation of the camera, so that the problems of stability and data transmission when the camera rotates are effectively solved, the advantages of the camera are further enlarged, the calculated amount of later image processing is greatly reduced, the camera is fixed, better protection can be achieved, the camera is not easy to damage in the using process, and the service life is prolonged;

2) in the aspect of projection equipment, VR glasses are combined with a smart phone, immersion experience is optimized through smart phone screen playing and voice explanation, sharing degree of modeling data is greatly improved, and the embarrassment of 'less wolf meat' in the actual experience process is avoided;

3) in the aspect of system software, an initial image template is established between a far-view camera device and a near-view camera device and is stored in a small computer, and a series of related characteristic points are extracted based on the initial image template in the post-modeling manufacture to replace a small amount of image data, so that the calculated amount of image processing can be greatly reduced;

4) in the aspect of system software, a scene simulation device is added, and the weather conditions under different conditions are simulated by adopting the light shielding characteristic of the space environment, so that the weather conditions are closer to reality, and the user experience is improved;

5) according to actual requirements, the virtual imaging image can be designed into multiple experience modes, and the immersive VR glasses can provide simultaneous experience for multiple users by taking a network as a hub; the desktop VR image can provide the most direct spatial distance sense for some urban construction planners, and is matched with other software to directly design on a three-dimensional image; the screen VR projection may be used for tourist sightseeing.

Drawings

Fig. 1 is a schematic diagram of an imaging system of the present invention.

Fig. 2 is a schematic structural view of the rotary tripod head of the present invention.

Fig. 3 is a schematic diagram of the movement of the shooting device of the present invention.

Fig. 4 is a schematic structural diagram of the VR glasses of the present invention.

The reference signs are: the device comprises a CCD camera 1, a rotating holder 2, a reflective mirror 3, a glasses support 4, a lens 5, a smart phone support 6, a mobile phone placing groove 7, an adjusting lock catch 8, a chassis 201, a motor mounting seat 202, a servo motor 203, a crankshaft 204, a connecting rod 205, a first fixing frame 206, a second fixing frame 207, a limiting rod 208, a vacuum chuck 209, a long-range shooting device 9, a short-range shooting device 10, an image splicing module 11, an image storage unit 12, a scene simulation device 13, an information model construction device 14, VR glasses 15 and a screen type VR projection 16.

Detailed Description

In the following description, numerous specific details are set forth in order to provide a more thorough understanding of the present invention. It will be apparent, however, to one skilled in the art, that the present invention may be practiced without one or more of these specific details. In other instances, well-known features have not been described in order to avoid obscuring the invention.

In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings, and are only for convenience of description and simplicity of description, but do not indicate or imply that the device or element being referred to must have a particular orientation, be constructed and operated in a particular orientation, and thus, should not be construed as limiting the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.

As shown in fig. 1, a city intelligent virtual imaging device includes: the device comprises a shooting device, an image processor and a projection device.

The imaging device includes: the rotary tripod head 2, the CCD camera unit and the reflector 3. Including rotatory cloud platform 2 on fixed mounting unmanned aerial vehicle, helicopter, specified area or the vehicle-mounted platform, vertically place with rotatory cloud platform 2 below the CCD camera unit, and fixed mounting be in rotatory cloud platform 2 go up, and with CCD camera unit's camera lens direction becomes reflector 3 of 45 contained angles. Wherein the rotating head 2 comprises: the robot comprises a frame, a chassis 201, a motor mounting seat 202, a servo motor 203, a crankshaft 204, a motor shaft, a motor mounting seat 202, a servo motor 203, a crank shaft and a crank shaft, wherein the frame consists of a plurality of sectional materials and is fixed with the unmanned aerial vehicle, the helicopter, a designated area or a vehicle-mounted platform, the chassis 201 is fixedly mounted on the frame, the motor mounting seat 202 is welded on the chassis 201, the servo motor 203 is arranged in the motor mounting seat 202, the; with connecting rod 205 that the bent axle 204 is connected, fixed mounting be in the first fixed frame 206 of motor mount pad 202 both sides, through the round pin axle with the fixed frame 207 of second of connecting is rotated to the bottom of first fixed frame 206, fixed mounting in the fixed frame 207 of second, and run through the gag lever post 208 of connecting rod 205, and with connecting rod 205 fixed connection's vacuum chuck 209. The reflector 3 is fixedly installed on the vacuum chuck 209, and the fixed central point is located at the intersection point of the extension line of the CCD camera unit and the rotating shaft. The CCD camera shooting unit is a CCD camera 1 adopting 15 heads of fish VR glasses. The servo motor 203 drives the rotating shaft to rotate eccentrically, the connecting rod 205 is provided with a vacuum chuck 209 to rotate along a preset inclination angle, the first fixing frame 206 and the second fixing frame 207 limit and fix the position of the connecting rod 205, and the running stability of the reflector 3 in the vertical direction is ensured. The reflector 3 reflects specific surrounding street scenery to the CCD camera shooting unit, the rotation of the CCD camera 1 is replaced by rotating the reflector 3, the weight of the reflector 3 is light, the problems of stability and data transmission when the CCD camera 1 rotates are effectively solved, the environment-friendly type full-view image acquisition device further amplifies the environment-friendly type full-view image acquisition device which adopts the single CCD camera 1 to perform annular scanning to acquire 360-degree full-view images, the original images obtained in each cycle scanning period have the characteristics of high resolution and no distortion in all directions, the calculated amount of later-stage image processing is greatly reduced, the CCD camera 1 is fixed, better protection can be obtained, the situation that damage is difficult to occur in the using process is guaranteed, and the service life is prolonged.

The image processor is mainly a computer processor and comprises a high-performance computer used for modeling an initial template of an image for a first scene, or the high-performance computer is used for calculating for a long time to obtain the initial image template, and the image template can be updated once in 1-2 years; the common computer processor for later image processing only needs to replace a small amount of information and adopts a common computer in the market and a WiFi communicator for data transmission.

The projection device is VR glasses 15 for immersive experience. It includes: the intelligent mobile phone comprises a VR glasses 15 support 4, two convex lens lenses 5 and a smart mobile phone support 6, wherein the VR glasses 15 support 4 is attached to the outer surface of the face of a wearer, the two convex lens lenses 5 are fixed on the VR glasses 15 support 4, a preset gap is reserved between the VR glasses 15 support 4 and the smart mobile phone support 6, the bottom of the VR glasses 15 support 4 is connected with the smart mobile phone support 6 through a mobile phone placing groove 7, and a preset distance is reserved on the upper portion of a mobile phone to be placed on a smart mobile phone; the reserved distance between the two sides and the upper part of the VR glasses 15 support 4 and the smart phone support 6 is adjusted through the adjusting lock catch 8, and the adjusting lock catch 8 is detachably fixed on a plurality of clamping grooves which are formed in the VR glasses 15 support 4 and keep the preset distance, and is used for fixing the smart phone; and a plurality of rubber bands, NFC pastes, magic pastes and plastic plates which play a role in auxiliary supporting, filling and fixing. Along with the popularity and the popularization of smart mobile phones, most of look all to have in the screen player and divide the screen function, can carry out wireless connection through wiFi between smart mobile phone and the image processor, a server can correspond a plurality of mobile phone user sides simultaneously, through smart mobile phone screen broadcast and pronunciation explanation, optimizes and immerses experience and feels, improves the sharing degree of modeling data greatly, avoids the embarrassment of "wolf more meat is few" at the actual experience in-process. Above-mentioned immersive VR glasses 15 can use the network as the pivot, provides multi-user experience simultaneously, can apply with the sight spot sightseeing aspect, can experience the person of experience not enough family, can experience the nature scene beyond the thousand miles.

As a preferable scheme, the projection device may also be a screen VR projection 16, and a spherical picture can be converted into a normal plane view, which can be converted into a normal plane view suitable for human eyes by being corrected by correction software. The screen type VR projection 16 may be specifically a desktop computer and a large screen display, where the desktop computer is mainly used for design by city builders and planners, provides the most direct sense of spatial distance, and is directly designed and edited on a three-dimensional image in cooperation with software such as Sketch UP, 3dMax, V-Ray renderer, and the like. The large-screen projection is mainly applied to a panoramic exhibition hall for urban sightseeing exhibition, in particular to a projection hall, and specifically comprises a projection room, a holographic projection film covered in the wall of the projection room, and a spherical multi-angle projection unit arranged at the center of the projection room in the morning; wherein the projection unit is provided with at least six projectors which respectively face six surfaces of the projection chamber.

And the data acquisition module comprises a long-range camera 9 and a short-range camera 10. Wherein, long shot camera device 9 utilizes the shooting device on fixed unmanned aerial vehicle or the helicopter to acquire the aerial image of the slope multi-angle of the city that awaits measuring, and close shot camera device 10 utilizes the shooting device who is fixed in the appointed region in street and utilizes on-vehicle removal, acquires the ground image of city street view.

And the image processing module comprises an image splicing module 11 and an image storage unit 12. The long-range view camera device 9 and the short-range view camera device 10 are adopted to shoot different image information as much as possible in a rotating mode in a relatively static state in a first scanning period, a plurality of pictures at the same place are spliced to form a hologram, then an image splicing module 11 adopts a series of image splicing technologies of correcting a rotating angle, cylindrical projection and translation alignment to form an electronic sand table by the plurality of holograms, a manufactured image initial template is stored in an image storage unit 12, then real-time new image information is obtained through the short-range view camera device 10 fixed in a specified area of a street, feature points in the same area are compared with the image initial template to be used as comparison, all pixel points in different feature point areas are replaced, and a new real-time image is updated and synthesized. The holographic picture of the local key position is updated in real time through the camera fixed in the key area, and the three-dimensional image of the city is updated in time only by small calculation processing, so that the problem of poor real-time performance in the existing modeling method is solved.

The intelligent fusion module comprises a scene simulation device 13 and an information model construction device 14; the scene simulation device 13 is special effect information of three-dimensional terrain and weather information which are fused and constructed by real-time images, adopts the light shielding characteristic of space environment, simulates a plurality of pictures including smoke, rainwater, lightning and sky, adjusts the light shadow according to specific time and actual weather conditions, and sets the sky texture, the fog color, the position, the density and the moving speed of clouds. The information model construction device 14 is used for configuring the information model construction device 14 of real-time image configuration geographic information, position information and space category information, and the information model construction device is stored as audio information. The device can be used as a explaining device in VR experience, and can explain in real time and experience the humanity scene of a city.

The holographic imaging module can be specifically three modes of immersion-type experience VR glasses 15, desktop-type VR images or screen-type VR projection 16.

A modeling method of an urban intelligent virtual imaging system comprises the following specific modeling methods:

step one, a long-range view annular hologram of an urban designated area is completed by adopting a long-range view camera device 9 carried by an unmanned aerial vehicle or a helicopter and is used as a background frame;

secondly, carrying a close-range camera device 10 by vehicle-mounted movement, carrying out high-precision modeling on specific street view of the city, and manufacturing a three-dimensional electronic sand table which is used as an initial image template;

refining and optimizing the three-dimensional electronic sand table image according to close-range cameras 10 fixedly distributed in the designated area of the street, and realizing real-time updating of the image in the designated area;

fourthly, the scene simulation device 13 simulates a plurality of pictures including smog, rainwater, lightning and sky by adopting the light shielding characteristic of the space environment, adjusts the light shadow according to specific time and actual weather conditions, and sets the sky texture, the fog color, the position, the density and the moving speed of the cloud;

step five, the information model building device 14 builds geographical information, position information and space category information of the relevant scene positions and stores the geographical information, the position information and the space category information as audio information;

step six, the user places the smart phone on VR glasses 15 to wear VR glasses 15, transmit image information to the smart phone through the WiFi setting, play the image through the VR screen function on the smart phone, realize the broadcast of image.

For the shooting and projection device for VR imaging in the design, another calculation method with a small calculation amount different from the prior art is provided to realize that the image processing module processes and splices the holograms at the same place, and the specific method is as follows:

step one, a CCD camera unit collects a full light function in a light field of a street view of a designated area, and the full light function comprises the following specific steps:

；

wherein the content of the first and second substances,

the actual position coordinate of any point in the light field;

the included angle of any point of a target scene and the CCD camera unit in the east-west direction and the south-north direction is respectively; gamma is the wavelength of the light, t is the time from any point of the target scene to the CCD camera unit, and the distance between any point of the target scene and the CCD camera unit can be further calculated;

secondly, the CCD camera units are positioned at the same position, and are used for shooting images at different angles, generating corresponding images and transmitting information to the image processing module;

thirdly, the image processing module divides each image into a plurality of image elements along the transverse direction at equal distance to obtain an image element array, and standard weights (values are 1, 0.50, 0.33, 0.25 and 0.20 … …) of each image element are determined by comparing the rotation angle of each CCD camera unit with the contact ratio between a plurality of adjacent images to obtain a weight array of each image;

step four, when the reflector 3 rotates at a constant speed in the vertical direction, the offset of the image in the horizontal direction and the vertical direction is ignored, and the standard weight array is a correction weight array;

and step five, multiplying the transposed array of the correction weight array and the image element array to obtain a new pixel point array, accumulating and combining new image element arrays of different images to form a new image element, and finally taking the new image as the annular hologram.

When the method is used for processing and splicing the hologram, because the reflector 3 always rotates at a constant speed at an angle of 45 degrees with the horizontal direction, the distortion in the horizontal direction and the vertical direction can be almost ignored, and the pre-processing calculated amount of image processing and calculation, specifically the calculated amount of the procedures of image rotation alignment, cylindrical surface projection and the like, is greatly reduced.

It should be noted that the various features described in the above embodiments may be combined in any suitable manner without departing from the scope of the invention. The invention is not described in detail in order to avoid unnecessary repetition.

Claims

1. A virtual imaging device of city intelligence, comprising:

the shooting device adopts a CCD camera which adopts a fisheye lens for scanning and comprises a rotating tripod head which is fixedly installed on an unmanned aerial vehicle, a helicopter, a designated area or a vehicle-mounted platform, a CCD camera unit which is vertically arranged below the rotating tripod head, and a reflector which is fixedly installed on the rotating tripod head and forms an included angle of 45 degrees with the lens direction of the CCD camera unit;

the image processor is mainly a computer processor and comprises a high-performance computer used for the first scene modeling image initial template, a common PC terminal used for the later image processing and a WiFi communicator;

2. The city intelligent virtual imaging device of claim 1, wherein the rotating pan-tilt comprises: the robot comprises a frame, a chassis, a motor mounting seat, a servo motor, a crankshaft, a motor shaft, a motor mounting seat, a servo motor, a crank shaft and a motor shaft, wherein the frame consists of a plurality of sectional materials and is fixed with the unmanned aerial vehicle, the helicopter, a designated area or a vehicle-mounted platform; with the connecting rod that the bent axle is connected, fixed mounting be in the first fixed frame of motor mount pad both sides, through the round pin axle with the fixed frame of second of connecting is rotated to the bottom of first fixed frame, fixed mounting in the fixed frame of second, and run through the gag lever post of connecting rod, and with connecting rod fixed connection's vacuum chuck, wherein, reflector fixed mounting is in on the vacuum chuck, the fixed position is in the nodical of CCD camera unit extension line and pivot.

3. The city intelligent virtual imaging apparatus according to claim 1, wherein the projection device is also a screen VR projection, in particular a desktop computer or a large screen display; the large screen display for sightseeing experience is a panoramic VR projection auditorium and specifically comprises a projection room, a holographic projection film covering the interior of the wall of the projection room, and a spherical multi-angle projection unit arranged at the center of the projection room in the morning; the projection unit is at least provided with six projectors which face six surfaces of the projection chamber respectively.

4. A city intelligent virtual imaging system is characterized by comprising

The data acquisition module is based on the shooting device in the urban intelligent virtual imaging equipment of claim 1 or 2; the system comprises an aerial long-range image pickup device for acquiring multi-angle inclined multi-angle images of a city to be detected by using an unmanned aerial vehicle or a helicopter, and a close-range image pickup device fixedly distributed in a street designated area and acquiring surrounding ground images by using vehicle-mounted movement;

an image processing module based on the image processor in the urban intelligent virtual imaging device of claim 1; the method comprises the steps that the long-range view camera device and the close-range view camera device are adopted to shoot image information in a first scanning period in a rotating mode in a relatively static state, an image splicing module of a series of image splicing technologies of rotation angle correction, cylindrical projection and translation alignment is adopted, an image initial template manufactured by the image splicing module is adopted, an image storage unit used for comparing the image initial template is used, real-time new image information is obtained through the close-range view camera device fixed in a specified region of a street, the same feature vector is compared with the image initial template to be used as comparison, all pixel points in different feature vector regions are replaced, and a new real-time image is synthesized;

the intelligent fusion module is based on the image processor in the urban intelligent virtual imaging device of claim 1; the system comprises a scene simulation device for special effect information of three-dimensional terrain, people/traffic flow and weather information fused and constructed for real-time images, a close-range shooting device arranged in a key building, an internal building imaging device fused to the real-time images through an image processing module, and an information model construction device for configuring the real-time images and configuring geographic information, position information and space category information, wherein the information model construction device is used for storing audio information;

a holographic imaging module based on the projection device in the urban intelligent virtual imaging equipment of claim 1 or 3; specifically, three modes of immersive VR glasses, desktop VR image or screen VR projection.