WO2018113011A1

WO2018113011A1 - Scene generation device, robot and unmanned driving device

Info

Publication number: WO2018113011A1
Application number: PCT/CN2016/112865
Authority: WO
Inventors: 李军明
Original assignee: 李军明
Priority date: 2016-12-21
Filing date: 2016-12-29
Publication date: 2018-06-28
Also published as: CN206271053U

Abstract

A scene generation device, consisting of an optical imaging unit (1), an image processing unit (2), a control unit (3) and a storage unit (4), the units being connected sequentially. The optical imaging unit (1) comprises a first ultra-wide angle negative lens module (11), a second ultra-wide angle negative lens module (12), a first steering module (13), a second steering module (14), a relay module (17), an imaging module (18) and an image sensor (19). The present invention provides a scene generation device which is low in cost, has a small volume, may be handheld and which may achieve 360-degree acquisition of visual field without blind spots, and provides a robot and an unmanned driving device which have the scene generation device installed.

Description

Description Title: A scene generation device, a robot and an unmanned device

[0001] The present invention relates to the field of virtual reality scene generation technologies, and in particular, to a scene generation device, a robot, and an unmanned device of an ultra-wide field of view.

Background technique

[0002] Virtual reality systems have been widely used in urban planning, simulation training, publicity and display, and entertainment. Compared with the application of virtual reality scenes, the construction of virtual reality scenes usually uses computer modeling to generate three-dimensional images. It often requires a lot of manpower and material resources, and the cycle is long and the cost is huge, which restricts the development of the market.

technical problem

[0003] At present, the use of optical devices for panoramic image acquisition to construct a virtual reality scene has many advantages such as high efficiency and low cost. The single-lens panoramic optical device adopts a wide-angle lens and an imaging device for image acquisition, which is small in size and low in cost, and occupies a large market share. However, since the lens has limited field of view, only a panoramic image can be generated, so it cannot be truly A virtual reality scene with no dead angle in the 360-degree field of view. The dual-lens or multi-lens panoramic optical device uses two-lens or multi-lens with multiple imaging devices for image acquisition, and then spliced into a panoramic image through software, which can achieve a 360-degree field of view without a dead angle virtual reality scene, but Due to the large number of devices used, the corresponding processing circuits are also complicated, and the device is bulky and costly.

Problem solution

Technical solution

[0004] The technical solution adopted by the present invention to solve the technical problem is as follows:

[0005] A scene generating device, comprising two super wide-angle negative lens modules, two steering modules, two switching modules, a relay module, an imaging module, an image sensor, a data processing circuit, and a control Unit, storage unit composition. The control unit is connected to the switching module, the data processing circuit and the storage unit. The image field range of the super wide-angle negative lens module can be more than 180 degrees, and the two super wide-angle negative lens modules are coaxially arranged at a position where a complete 360-degree viewing angle can be acquired. The steering module is set to receive the picture completely after the super wide-angle negative lens module The position of the image, the gate module is set between the super wide-angle negative lens and the image module to block any position of the light path, and the relay module is disposed behind the steering module to completely receive two super wide-angle negative lens modules. The position of the image is collected, and the imaging module is disposed between the relay module and the image sensor and allows the image sensor to completely receive the position of the image acquired by the two super wide-angle negative lens modules. The control unit controls the turning on and off of the switching modules so that the light of the respective optical paths alternately enters the imaging module and the image sensor. The data processing circuit corrects the image acquired by the image sensor, and splices the two image data in the adjacent period under the coordinated control of the control unit to generate a complete 360-degree panoramic image and transmits it to the storage unit for storage.

[0006] A scene generation method corresponding to the foregoing apparatus includes the following steps:

[0007] 1. The control unit issues an initial command.

[0008] 2. The A-light path is turned off, and the B-path is turned off to make the A-path image enter the imaging module, and the image sensor collects the A-path image.

[0009] 3. The B-light path is turned off, and the A-light path is turned off to make the B-path image enter the imaging module, and the image sensor collects the B-path image.

[0010] 4. The data processing circuit corrects the acquired image to remove severely distorted portions.

[0011] 5. The data processing circuit splices an A-path image and a B-path image of the adjacent period according to the feature points to generate a complete panoramic image.

[0012] 6. Save the complete image to the storage unit.

[0013] 7. Repeat steps 2 through 6 until the control unit issues a stop command.

Advantageous effects of the invention

Beneficial effect

Compared with the prior art, the present invention provides a virtual reality scene generating device, which comprises two sets of separated super wide-angle negative lens modules, a steering module, a switching module, and a relay module. The imaging module divides the 360-degree panoramic image into the same image sensor, and performs image correction and splicing through the data processing circuit under the coordination of the control unit, and saves the final image to the storage unit. Since the 360-degree panoramic image is received by the same sensor, only one sensor and data processing circuit can be used to collect and process the panoramic image, effectively reducing the system size and manufacturing cost, making the whole device compact and compact, and can be used in handheld The device can also be used for personal information terminals. Brief description of the drawing

DRAWINGS

1 is a schematic structural diagram of a scene generation apparatus according to the present invention.

2 is a schematic diagram of an image acquisition and synthesis image of a scene generation device according to the present invention.

3 is a schematic flow chart of a scene generation method according to the present invention.

Embodiments of the invention

[0018] The solution of the present invention will be described in detail below in conjunction with the preferred embodiments thereof. The drawings are for illustrative purposes only and are not to be construed as limiting the scope of the invention; in order to better illustrate the embodiments, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the dimensions of the actual product; It will be understood by persons skilled in the art that certain known structures and their description may be omitted. The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

1 is a schematic structural diagram of a scene generating apparatus according to the present invention, which is composed of an optical imaging unit 1, a data processing unit 2, a control unit 3, and a storage unit 4, and each unit is sequentially connected; the optical imaging unit 1 includes: An ultra wide-angle negative lens module 11 , a second super wide-angle negative lens module 12 , a first steering module 13 , a second steering module 14 , a first switching module 15 , a second switching module 16 , Following the module 17, the imaging module 18 and the image sensor 19; wherein the control unit 3 is connected to the first switching module 15, the second switching module 16, the data processing unit 2 and the storage unit 3. The first super wide-angle negative lens module 11 and the second super wide-angle negative lens module 12 are optical lenses or lens groups which can be collected by optical glass and can have an image field of view greater than 180 degrees, and the positions of the lenses can be set. It can be used to capture a complete 360-degree view without requiring a special position. As a better example, the first super wide-angle negative lens module 11 and the second super wide-angle negative lens module 12 are coaxially arranged at a position where a full 360-degree viewing angle can be acquired, which can have a better effect. The first steering module 13 and the second steering module 14 are prisms, and may be respectively disposed in the form of a combination of plane mirrors, respectively, after the two super wide-angle negative lens modules 11 and 12 can completely receive the position of the image, the first mode The group 15 and the second bypass module 16 are disposed between the first super wide-angle negative lens module 11 and the second super wide-angle negative lens module 12 and the imaging module 19 to block any position of the optical path.

[0020] The first switching module 15 and the second switching module 16 in this embodiment use a mirror that can be controlled to deflect and block the optical path, and are respectively disposed in the first steering module. 13. After the second steering module 14, At the same time, the first shut-off module 15 and the second shut-off module 16 can adopt an electronically controlled shutter to realize on-off control of the respective optical paths. The relay module 17 can be completely received by the first steering module 13 and the second steering module 14 by using a quadric surface element with high reflection characteristics or a plurality of mirrors combined with the lens. The negative lens module 11 and the second super wide-angle negative lens module 12 capture the position of the image, and the imaging module 18 is disposed between the relay module 17 and the image sensor 19 by using an imaging lens or a lens group having an imaging function. The image sensor 19 is completely received by the positions of the images acquired by the two super wide-angle negative lens modules. The image sensor 19 may be a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) device. The data processing unit 2 includes an integrated circuit chip and can perform image correction and stitching functions.

[0021] The control unit 3 controls the opening and closing of the first switching module 15 and the second switching module 16 so that the light of the respective optical paths alternately enters the imaging module 18 and the image sensor 19. The data processing circuit 2 corrects the image acquired by the image sensor 19, removes the severely distorted portion, and splices the two image data in the adjacent period under the coordinated control of the control unit 3 to generate a complete 360-degree panoramic image. After the transfer to the storage unit 4 is saved. The details will be described below with reference to FIG. 2.

2 is a schematic diagram of an image acquisition and synthesis image of the scene generation device of the present invention. For convenience of description, one of the optical paths corresponding to the first super wide-angle negative lens module 11 and the second super wide-angle negative lens module 12 is now called A light path, the other is B light path. During the acquisition period tla, the control unit 3 activates the first switching module 15 of the A optical path, and simultaneously closes the second switching module 16 of the B optical path, so that the light of the A optical path enters the imaging module 18, and the image sensor 19 Receiving the image of the A optical path, the data processing unit 2 corrects and generates the image P1a; in the adjacent acquisition period tlb, the control unit 3 activates the second switching module 16 of the B optical path, and simultaneously closes the first pass of the A optical path. The module 1 5 causes the light of the B optical path to enter the imaging module 18, the image sensor 19 receives the image of the B optical path, and the data processing unit 2 corrects the generated image Plb. Next, the data processing circuit 110 images the image P1a, P1 according to the feature point. Splicing is performed to generate a complete panoramic image Cl. In the following collection periods t2a, t2b, t3a, t3b, the above process is repeated, and the image sensor 19 separately collects the A light path, and the image of the light path is corrected by the data processing unit 2 to generate images P2a, P2b, P3a, P3b, and finally Synthesize panoramic images C2, C3.

3 is a schematic flowchart of a method for generating a scene according to the present invention, which includes the following steps:

[0024] S101: using at least one position point as the feature point on the item whose parameters are known (the selection of the feature point may be randomly selected according to a straight line or a designed curve), and is started by the control unit 3 instruction [0025] S102: 幵 A A light path is turned off, and the B light path is turned off, so that the A light path image enters the imaging module, and the image sensor collects the A light path image. The scene image is obtained by the first super wide-angle negative lens module 11 , and the A light path is formed by the optical signal transmission path of the first steering module 13 and the first switching module 15 and the relay module 17; The A light path enters the imaging module 18, and the image path sensor 19 collects the A light path image to obtain the first scene image; and performs distortion correction according to the perspective relationship to remove the severe distortion portion of the image.

[0026] S103: The B side optical path is turned off, and the A side optical path is turned off, so that the B optical path image enters the imaging module, and the image sensor collects the B optical path image. Obtaining a scene picture by the second super wide-angle negative lens module 12, and forming a B optical path through the optical signal transmission path of the second steering module 14 and the second switching module 16 and the relay module 17; The B optical path enters the imaging module 18, and the B optical path image is acquired by the image sensor 19 to obtain a second scene image. The data processing unit corrects the acquired image to remove severely distorted parts.

[0027] S104: The data processing unit splices feature points of the first scene image and the second scene image of the adjacent period to generate a complete 360-degree field of view image. Finding a feature point common to the first scene image and the second scene image, and scaling, rotating, and panning the first scene image and the second scene image according to the size and position of the feature point, so that the feature points in the two images coincide, Complete image stitching, record zoom, rotate, and pan data as stitching parameters.

[0028] S105: Store the image to the storage unit.

[0029] S106: S102 to S105 are repeated until the control unit 3 issues a stop command.

[0030] The present invention also provides a robot for mounting the virtual reality scene generating device on a head, mainly for monitoring or patrolling.

[0031] The present invention also provides an unmanned device for installing the virtual reality scene generating device on the top of the vehicle; for automatically driving, better acquiring information about the surrounding environment, and assisting the driving system to appropriately according to the acquired panoramic image; Ground treatment.

The above-mentioned embodiments are merely illustrative of the embodiments of the invention, and the description thereof is more specific and detailed, but is not to be construed as limiting the scope of the invention. It should be noted that a number of variations and modifications may be made by those skilled in the art without departing from the spirit and scope of the invention. Therefore, the scope of protection of the utility model patent should be The appended claims shall govern.

Industrial applicability

The invention comprises an optical imaging unit, a data processing unit, a control unit and a storage unit, wherein the units are connected in sequence; the optical imaging unit comprises: a first super wide-angle negative lens module, a second super wide-angle negative lens module, and a first steering mode The first steering module, the second steering module, the relay module, the imaging module and the image sensor; the first super wide-angle negative lens module and the second super wide-angle negative lens module are used for acquiring a scene picture; The second steering module is respectively disposed at a position where the image is completely received after the first super wide-angle negative lens module and the second super wide-angle negative lens module; the imaging module is disposed between the relay module and the image sensor, so that The image sensor completely receives the received image of the first steering module and the second steering module, and uses two or more lenses to cooperate with multiple imaging devices for image acquisition, and then splicing into a panoramic image through software, which can achieve 360. The virtual reality scene with no dead angle in the field of view has industrial applicability.

Claims

Claim

A scene generating device is composed of an optical imaging unit (1), a data processing unit (2), a control unit (3), and a storage unit (4), wherein the units are sequentially connected; wherein the optical imaging unit (1) The method includes: a first super wide-angle negative lens module (11), a second super wide-angle negative lens module (12), a first steering module (13), a second steering module (14), and a relay module ( 17), an imaging module (18) and an image sensor (19); the first super wide-angle negative lens module (11) and the second super wide-angle negative lens module (12) are used to acquire a scene picture; The first steering module (13) and the second steering module (14) are respectively disposed on the first super wide-angle negative lens module (11) and the second super wide-angle negative lens module (12) The position at which the image can be completely received; the imaging module (18) is disposed between the relay module (17) and the image sensor (19) such that the image sensor

(19) Completely receiving the received image of the first steering module (13) and the second steering module (14).

The scene generating apparatus according to claim 1, wherein the control unit (3) is respectively connected to the first switching module (15) and the second switching module (16); The closing module (15) is disposed between the first super wide-angle negative lens module (11) and the imaging module (18) to block any position of the optical path; the second switching module (16) is disposed at the The position of the optical path can be blocked between the two super wide-angle negative lens module (12) and the imaging module (18).

The scene generating device according to claim 1 or 2, wherein the first super wide-angle negative lens module (11) and the second super wide-angle negative lens module (12) are coaxially arranged to collect a complete 360. The position of the angle of view image.

A robot, characterized in that the robot head mount comprises the scene generating device according to any one of claims 1-3.

An unmanned device, characterized in that the top of the unmanned device is mounted by the scene generating device according to any one of claims 1-3.