TWI679889B - A panoramic camera augmented reality system - Google Patents

Abstract

The invention relates to a panoramic camera augmented reality system, comprising: a casing, a circuit board disposed in the casing; a first image acquisition device and a second image acquisition device fixed on the casing and electrically connected to the circuit board. The parameters of the first image acquisition device are different from those of the second image acquisition device, and the first image acquisition device and the second image acquisition device are respectively used to capture a main scene image and An auxiliary scene image; and an image receiving unit and an image processing unit which are disposed in the casing and are electrically connected to the circuit board; the image receiving unit is configured to receive the first image acquisition device and the image receiving unit; The main scene image and the auxiliary scene image collected by the second image acquisition device at the same time; the image processing unit is configured to perform the same operation on the first image acquisition device and the second image acquisition device at the same time The collected main scene image and auxiliary scene image are corrected and stitched to obtain a panoramic image.

Description

Panoramic camera augmented reality system

The present invention relates to the field of cameras, and in particular, to a panoramic camera augmented reality system.

Panoramic camera augmented reality system can provide users with a full range of panoramic images for seamless monitoring, so it replaces multiple ordinary surveillance cameras and is widely used in banks, government agencies, prisons and other places or for filming , Especially when used in movie scenes or selfies, often need to be supported by the background to highlight a scene. However, the existing panoramic camera augmented reality system can only realize a 360-degree or 720-degree panoramic image at most, and it cannot reflect the primary and secondary angles of the image. Like, is not conducive to improving the user experience.

Therefore, it is necessary to provide a panoramic camera augmented reality system capable of solving the above problems.

A panoramic camera augmented reality system includes a casing and an image acquisition device fixed on the casing. The image acquisition device includes a first image acquisition device and a second image acquisition device. An image acquisition device and the second image acquisition device are independently controlled, and the first image acquisition device and the second image acquisition device are controlled independently. At least one of the angle of view, the focal length, the zoom factor, and the resolution is different, so that the first image acquisition device and the second image acquisition device are respectively used to capture images of different specifications; and are disposed on the casing An image receiving unit and an image processing unit inside and electrically connected to the first image acquisition device and the second image acquisition device; the image receiving unit is configured to receive the first image acquisition Images captured by the device and the second image acquisition device at the same time; the image processing unit is configured to process images acquired by the first image acquisition device and the second image acquisition device at the same time Corrective stitching is performed to obtain a panoramic image.

In a preferred embodiment, the panoramic camera augmented reality system further includes a storage module, and the storage module stores images and / or images from external sources, and the image processing unit is further configured to retrieve the images from the storage module. The group extracts the image or video from an external source and attaches it to a predetermined position of the panoramic image.

In a preferred embodiment, the panoramic camera augmented reality system further includes a communication module and a controller provided in the casing, and the communication module is configured to receive a remote control instruction sent by a mobile terminal. The instruction includes the image acquisition device to be controlled and the type of operation that the image acquisition device to be controlled and / or the type of operation that the image processing unit needs to perform. The controller is configured to analyze the remote control instruction, and to As a result, the corresponding image acquisition device is controlled to perform the operation.

In a preferred embodiment, the types of operations that the image acquisition device needs to perform include one of advancing the image acquisition device, pulling in the image acquisition device, adjusting the focal length, and adjusting the aperture; the image processing unit needs to perform Operation types include image stitching or image fitting.

In a preferred embodiment, the parameter that determines the photographing specification is any one or more of a viewing angle, a focal length, an optical zoom factor, a size and a resolution of an image sensor.

In a preferred embodiment, a tripod is connected below the casing, and a rotation mechanism is provided on the tripod.

In a preferred embodiment, the stand is made of a material with high thermal conductivity.

In a preferred embodiment, the second image acquisition device is provided on the tripod.

In a preferred embodiment, when the panoramic camera augmented reality system is switched to a manual mode, the image acquisition device is manually controlled to perform the operation.

In a preferred embodiment, the image acquisition device is connected to a power cord, and an end of the power cord far from the image acquisition device protrudes from the casing and is connected to a battery.

In a preferred embodiment, the casing is a symmetric structure or an asymmetric structure, and a solar cell panel is fixed on the casing. The solar cell panel is configured to absorb sunlight and convert the absorbed sunlight into electrical energy. , And then output the electric energy to the battery to charge the battery.

In a preferred embodiment, the casing is further provided with one or more of an HDMI output interface, a USB interface, and a TF card slot.

Compared with the prior art, since the panoramic camera augmented reality system includes a first image acquisition device and a second image acquisition device, the perspectives of the first image acquisition device and the second image acquisition device, At least one of the focal length, the zoom factor, and the resolution is different, so that the first image acquisition device and the second image acquisition device are respectively used to capture images of different specifications, so that the obtained panoramic image can realize the main The sub-angles are helpful to improve the user experience.

1, 3, 4‧‧‧‧ panoramic camera augmented reality system

2‧‧‧ mobile terminal

10‧‧‧chassis

20‧‧‧Circuit Board

30‧‧‧The first image acquisition device

40‧‧‧Second image acquisition device

50‧‧‧Storage Module

60‧‧‧Image receiving unit

31‧‧‧first optical lens

41‧‧‧second optical lens

42‧‧‧Second image sensor

32‧‧‧The first image sensor

70‧‧‧Image processing unit

80‧‧‧ communication module

90‧‧‧ Controller

110‧‧‧ Battery

120‧‧‧solar panel

130‧‧‧Tripod

200‧‧‧Power cord

201‧‧‧USB interface

202‧‧‧winding structure

140‧‧‧Rotating mechanism

FIG. 1 is a schematic structural diagram of a panoramic camera augmented reality system according to a first embodiment of the present invention.

FIG. 2 is a module architecture diagram of the panoramic camera augmented reality system shown in FIG. 1.

FIG. 3 is a schematic structural diagram of a panoramic camera augmented reality system according to a second embodiment of the present invention.

FIG. 4 is a schematic structural diagram of a panoramic camera augmented reality system according to a third embodiment of the present invention.

First embodiment

Please refer to FIG. 1 to FIG. 2, a preferred embodiment of the present invention provides a panoramic camera augmented reality system 1. The panoramic camera augmented reality system 1 includes a casing 10, a circuit board 20 disposed in the casing, and a first image acquisition device 30 fixed on the casing 10 and electrically connected to the circuit board 20. And second image acquisition device 40.

In this embodiment, the casing 10 is substantially ellipsoidal. In another embodiment, the casing 10 may also have other shapes, such as various symmetrical structures or asymmetric structures such as square, rectangle, and sphere.

The number of the first image acquisition devices 30 is one or more. The number of the second image acquisition devices 40 is one or more. In this embodiment, description is made by using a first image acquisition device 30 and a second image acquisition device 40 as an example. The first image acquisition device 30 and the second image acquisition device 40 are oppositely disposed on the casing 10.

The first image acquisition device 30 and the second image acquisition device 40 are independently controlled. The parameters of the first image acquisition device 30 for determining the photographing specifications are different from the parameters of the second image acquisition device 40 for determining the photographing specifications. The first image acquisition device 30 and the second image acquisition device 40 It is used to capture the main scene image and the auxiliary scene image. The auxiliary scene image can be used to highlight the main scene image in the future. The parameters determining the photographing specifications are any one or more of the angle of view, the focal length, the optical zoom factor, the size and resolution of the image sensor, and these parameters can determine the specifications of the photographed image.

The first image acquisition device 30 includes a first optical lens 31 and a first image sensor 32 located behind the first optical lens 31. The second image acquisition device 40 includes a second optical lens 41 and a second image sensor 42 located behind the second optical lens 41.

The first optical lens 31 and the second optical lens 41 are respectively used to collect light signals of a scene within its shooting range. For example, the first optical lens 31 is a wide-angle lens, and the first optical lens 31 The angle of view of the lens 31 is 205 degrees; the second optical lens 41 is also a wide-angle lens, and the angle of view of the second optical lens 41 is 205 degrees. The binocular lens is symmetrically arranged on the casing 10, and the first optical lens 31 and the second optical lens 41 are each responsible for the double hemisphere, that is, the 360-degree shooting of the scene is achieved on the radial circumferential surface. The 360-degree vertical shooting of the scene is also achieved on the axial circumferential surface of the sphere, so that the two lenses are combined into a sphere, so that the panoramic camera augmented reality system 1 can cover a 360-degree or 720-degree field of view. , FOV).

Of course, it can be understood that the first image acquisition device 30 and the second image acquisition device 40 can also be arranged asymmetrically on the casing 10 by providing the first image acquisition device 30 and the second image acquisition device with different viewing angle ranges. The image acquisition device 40 so that they can take pictures of different viewing angle ranges to obtain 360-degree or 720-degree panoramic field-of-view images. The shooting angle of the first image capture device 30 may be 8 °, 28 °, 45 °, 94 °, 120 °, 152 °, etc. In short, when the shooting angle or zoom factor of the first image capture device 30 is changed The shooting angle of view or the zoom factor of the second image acquisition device 40 can be selected whether the combination changes or remains unchanged.

The first optical lens 31 may further be connected with a fish-eye lens to increase its field of view. The resolution of the first optical lens 31 may be 7680 × 4320 (90-120p), and the zoom magnification may be 14, 22, 40, 76, 101 times.

The first image sensor 32 and the second image sensor 42 are respectively used to receive light signals collected by the corresponding first optical lens 31 and the second optical lens 41 and form images.

The panoramic camera augmented reality system 1 further includes a storage module 50. The storage module 50 stores an image or an image from an external source. Subsequently, the image processing unit 80 is further configured to remove the image from the storage module 50. Extract an image or image from an external source and attach it to a predetermined position of the panoramic image. The predetermined position may be a position where the image is stitched. For example, when the stitching position of the original picture is incomplete or it is found that the image or picture of the stitching position is unclear, an external picture or image may be pasted here to cover the unclear part of the stitching position. The attached external image or video may be a static image or a dynamic image. In another embodiment, the panoramic camera enhanced display system 1 may further include a A display module (not shown). The display module is connected to the output display terminal. The output display terminal includes an LED display, a projection display, a mobile terminal display, and a virtual reality glasses display.

Of course, it can be understood that the fitting can be performed before the image stitching, at the same time as the image stitching, and after the image stitching.

An image receiving unit 60 electrically connected to the circuit board 20 is also provided in the casing 10. The image receiving unit 60 is configured to receive a main scene image and an auxiliary scene image collected from the first image sensor 32 and the second image sensor 42 at the same time.

There is also one or more image processing units 70 (only one is shown in the figure) in the casing 10, and the image processing units 70 are electrically connected to the circuit board 20. The image processing unit 70 is configured to obtain a main scene image and an auxiliary scene image respectively captured by the first image sensor 32 and the second image sensor 42 from the image receiving unit 60, and convert the image The main scene image and the auxiliary scene image are corrected and stitched to form a panoramic image.

The panoramic camera augmented reality system 1 can switch between a remote control mode and a manual mode. When the panoramic camera augmented reality system 1 is in a remote control mode, the first optical lens 31 and the second optical lens 41 can be independently controlled by a remote control instruction to perform corresponding operations.

The user can send the remote control instruction to the panoramic camera augmented reality system 1 through a mobile terminal 2 through wireless transmission methods such as WIFI, Bluetooth, etc., so as to control the first optical lens 31 to perform corresponding operations. The mobile terminal 2 may be a remote controller, a smartphone, or a tablet computer.

A communication module 80 is provided on the casing 10, and the communication module 80 is configured to receive a remote control instruction sent by the mobile terminal 2. The remote control instruction includes the type of operation performed by the first optical lens 31 and / or the second optical lens 41 and the optical lens to be controlled and / or the type of operation that the image processing unit 60 needs to perform.

The types of operations that the first and second image acquisition devices 30 and 40 need to perform include advancing the image acquisition device (ZOOM IN), pulling in the image acquisition device (ZOOM OUT), adjusting the focal length, One of adjusting the aperture; the types of operations that the image processing unit 60 needs to perform include image stitching or image fitting.

A controller 90 is also provided in the casing 10, and the controller 90 is electrically connected to the communication module 80. The controller 90 is configured to analyze the remote control instruction, and control the corresponding first optical lens 31 or the second optical lens 41 to perform the operation according to the analysis result. That is, if the user wants to control the first optical lens 311 or the second optical lens 41 to adjust the focal length, he can press the corresponding key of the mobile terminal 2 to send the remote control instruction. In this way, the controller 90 can control the first optical lens 311 or the second optical lens 41 to adjust the focal length.

After receiving the remote control instruction, the panoramic camera augmented reality system 1 can cause the first optical lens 31 and the second optical lens 41 to be controlled to perform corresponding operations, thereby avoiding the inconvenience of manual control in some occasions, saving Time and effort saves the user experience.

When the first optical lens 31 and the second optical lens 41 are in a manual mode, the first optical lens 31 and the second optical lens 41 may be further controlled manually, that is, the first optical lens 31 and the first optical lens 31 The advancement, zooming, zooming, aperture adjustment, image stitching, and image fitting of the two optical lenses 41 can be achieved through manual control.

Specifically, the casing 10 may be provided with several mechanical buttons, and the user may not control the panoramic camera augmented reality system 1 through the mobile terminal 2, but control the panoramic camera by pressing a specific mechanical button. The real system 1 performs a specific operation.

Alternatively, the casing 10 is a touch screen, so that the user can directly operate the panoramic camera augmented reality system 1 through gesture sensing.

Of course, the first optical lens 31 and the second optical lens 41 may be further controlled by voice, which are not described in detail here.

The material of the casing 10 may be a metal with a high thermal conductivity, such as aluminum or copper, so as to improve the heat dissipation effect. A heat sink (such as graphene) can be attached to the casing 10 to further improve the heat dissipation effect.

In another embodiment, as shown in FIG. 1, the first image acquisition device 30 and the second image acquisition device 40 can still be connected to a power line 200, and the power line 200 is far from the first image acquisition device 30. And an end of the second image acquisition device 40 extends out of the casing 10. A USB interface 201 is provided at the end of each power cable 200, and each USB interface 201 is used to connect a battery 110. The battery 110 is configured to provide power to the panoramic camera augmented reality system 1 when the panoramic camera augmented reality system 1 is in operation. In this embodiment, the power cords 200 of different image acquisition devices 20 are entangled with each other to form a winding structure 202. Since the battery 110 is disposed outside the casing 10, it is convenient to remove or replace the battery. More specifically, the battery 110 may be a rechargeable battery.

A solar cell panel 120 is fixed on the casing 10, and the solar cell panel 120 is configured to absorb sunlight and convert the absorbed sunlight into electrical energy, and then output the electrical energy to the battery 110 to the battery 110. The battery 110 is charged.

In the above embodiment, the first image acquisition device 30 and the second image acquisition device 40 each have a battery 110, or the first image acquisition device 30 and the second image acquisition device 40 share one battery 110.

The first optical lens 31 and the second optical lens 41 can also be switched between a normal mode and a night vision mode. Specifically, an infrared filter is disposed on the first optical lens 31 and the second optical lens 41. When the first optical lens 31 and the second optical lens 41 are in the normal mode, the filter is turned on to block infrared or far-infrared rays from entering the first and second image sensors 32 and 42, that is, The first and second image sensors 32 and 42 can only sense visible light. When the first optical lens 31 and the second optical lens 41 are in the night vision mode, the infrared filter stops working, so that infrared or far infrared can enter the first and second image sensors 32 , 42 and imaging.

A tripod 130 is connected below the cabinet 10, and the bracket 130 is used for supporting the cabinet 10 and can be used to adjust the height of the cabinet 10. The tripod 130 can be made of a material with high thermal conductivity, so as to dissipate heat generated by various components in the casing 10 to achieve heat dissipation. Alternatively, the tripod 130 has a heat conducting tube inside, for example, the heat conducting tube has a heat conducting fluid inside, so as to dissipate heat generated by the image acquisition device 20, the image processing unit 40, or the controller 90 and other components to achieve heat dissipation .

The casing 10 is also provided with an HDMI output interface (not shown), a TF card slot (not shown), and the like; the HDMI output interface, the USB interface 201, and the TF card slot are electrically connected to the circuit board 10. Set the HDMI output interface, USB interface 201, TF card slot, etc. to facilitate the camera to connect with the host computer, mobile terminal and other equipment to achieve the transfer and exchange of data. Of course, it can be understood that the external interface on the casing 10 is not limited to the above-mentioned types of interfaces, and other interfaces that can implement data transmission with a host computer and a mobile terminal can also meet the requirements of the present invention.

The control method of the panoramic camera augmented reality system 1 includes the following steps:

Step S1: Start the panoramic camera augmented reality system 1 to control the panoramic camera augmented reality system 1 to enter a remote control mode.

Step S2: Select at least one first optical lens 31 and / or second optical lens 41 to be controlled by the mobile terminal 2.

Step S3: selecting, by the mobile terminal 2, the types of operations to be performed by the first optical lens 31 and the second optical lens 41 to be controlled and / or the types of operations that the image processing unit 60 needs to perform, thereby triggering all The mobile terminal 2 generates a remote control instruction and sends the remote control instruction to the panoramic camera augmented reality system 1. Wherein, the remote control instruction includes the types and / or operations of the first and / or second image acquisition devices 30 and 40 to be controlled and the first and / or second image acquisition devices to be controlled. The types of operations that the image processing unit 60 needs to perform are described. The types of operations that the image acquisition device needs to perform include one of advancing the image acquisition device, pulling in the image acquisition device, adjusting the focal length, or adjusting the aperture; the types of operations that the image processing unit needs to perform include image stitching or Image fit. That is, actions such as shooting, image stitching, and image bonding can be performed by sending a remote control instruction from the mobile terminal 2.

Step S4: The controller 90 obtains the remote control instruction through the communication module 80, analyzes the remote control instruction, and controls the corresponding image acquisition device and / or image processing unit to perform the operation according to the analysis result.

Of course, according to different requirements, the order of the above steps can be changed, and some steps can be omitted or combined.

Second embodiment

The panoramic camera augmented reality system 3 provided by the second embodiment is basically the same as the panoramic camera augmented reality system 1 provided by the first embodiment, except that the panoramic camera augmented reality system 3 provided by this embodiment includes two first The image acquisition device 30 and two second image acquisition devices 40, and the two first image acquisition devices 30 and two second image acquisition devices 40 are respectively disposed at the ends of the long axis of the ellipsoidal casing 10 And the position of the short axis end, in other embodiments, the positions of the two first image acquisition devices 30 and two second image acquisition devices 40 in the casing 10 are not limited, and can be based on actual shooting scenarios Decide.

The angles of view, focal lengths, zoom factors, and resolution parameters of the plurality of first image acquisition devices 30 are the same.

The angles of view, focal lengths, zoom factors, and resolution parameters of the plurality of second image acquisition devices 40 may be the same or different.

The number of the first image acquisition device 30 and the second image acquisition device 40 is multiple, and according to actual needs, a remote control instruction can be sent to the panoramic camera augmented reality system 1 through the mobile terminal 2 or can be operated by manual operation. The panoramic camera augmented reality system 1 changes the angle of view, focal length, and zoom factor of the first image acquisition device 30; or changes the angle of view, focal length, and zoom factor of the second image acquisition device 40; to obtain photos of different specifications.

Third embodiment

The panoramic camera augmented reality system 4 provided by the third embodiment is basically the same as the panoramic camera augmented reality system 1 provided by the first embodiment. The difference is that the casing 10 of the panoramic camera augmented reality system 4 provided by this embodiment is Two first image acquisition devices 30 and six second image acquisition devices 40 are provided. Two first image acquisition devices 30 are disposed at the ends of the long axis of the ellipsoid, and four second acquisition devices 40 are disposed near the two sides of the first image acquisition device 30, respectively. Two second image acquisition devices are arranged symmetrically at the top positions, only one of which is shown in the figure, but the understanding is not affected.

In addition, the tripod 130 is provided with a rotation mechanism 140. The rotation mechanism 140 may be any one of a wheel axle, a pivot structure, or a ball bearing, so that the tripod 130 can be relatively rotated for adjusting the The angle between the chassis 10 and the tripod 130, so that the chassis 10 and the tripod 140 can be folded to open or close. Of course, a detachable assembly structure may also be provided between the casing 10 and the tripod 130, which is not repeated here.

Furthermore, a second image processing device 40 is further provided on the tripod 130 to shoot and compose images of the scene below the panoramic camera 3.

In other embodiments, the numbers of the first image acquisition device 30 and the second image acquisition device 40 are determined according to actual needs, which are not described in detail here.

In summary, since the panoramic camera augmented reality systems 1, 3, and 4 include at least a first image acquisition device 30 and at least a second image acquisition device 40, the first image acquisition device 30 and At least one of the angle of view, the focal length, the zoom factor, and the resolution of the second image acquisition device 40 is different, so that a remote control instruction can be sent to the panoramic camera augmented reality system 1 through the mobile terminal 2 or can be manually operated. Operating the panoramic camera augmented reality system 1 to change the angle of view, focal length, and zoom factor of the first image acquisition device 30; or changing the angle of view, focal length, and zoom factor of the second image acquisition device 40 to capture images of different specifications, Therefore, the obtained panoramic image can realize the distinction between the primary and secondary angles, which is conducive to improving the user's Augmented Reality experience.

In summary, the present invention has indeed met the requirements for an invention patent, and a patent application was filed in accordance with the law. However, the above are only preferred embodiments of the present invention, and the scope of patent application in this case cannot be limited by this. For example, those who are familiar with the skills of this case and equivalent modifications or changes made in accordance with the spirit of the present invention should be covered by the following patent applications.

Claims (11)

A panoramic camera augmented reality system includes a casing, a circuit board disposed in the casing, and an image acquisition device fixed on the casing and electrically connected to the circuit board. The image acquisition device includes The first image acquisition device and the second image acquisition device, the first image acquisition device and the second image acquisition device are independently controlled, and the first image acquisition device determines parameters and The second image acquisition device has different parameters for determining a photographing specification, and the first image acquisition device and the second image acquisition device are respectively used to capture a main scene image and an auxiliary scene image; a storage module The storage module stores an image and / or an image from an external source; and an image receiving unit and an image processing unit disposed in the casing and electrically connected to the circuit board; the image The receiving unit is configured to receive the main scene image and the auxiliary scene image collected by the first image acquisition device and the second image acquisition device at the same time; and the image processing unit is configured to compare the first image image The main scene image and the auxiliary scene image acquired by the acquisition device and the second image acquisition device at the same time are corrected and stitched to obtain a panoramic image; the image processing unit is further configured to retrieve the image from the storage module. Extract images and / or images from external sources and attach them to a predetermined position of the panoramic image. The panoramic camera augmented reality system according to claim 1, wherein the panoramic camera augmented reality system further includes a communication module and a controller disposed in the casing, and the communication module is configured to receive a mobile terminal The remote control instruction sent includes the image acquisition device to be controlled and the type of operation that the image acquisition device to be controlled needs to perform and / or the type of operation that the image processing unit needs to perform, and the controller is configured to analyze The remote control instruction, and controlling the corresponding image acquisition device to perform the operation according to the analysis result. The panoramic camera augmented reality system according to claim 2, wherein the types of operations that the image acquisition device needs to perform include one of advancing the image acquisition device, pulling in the image acquisition device, adjusting the focal length, and adjusting the aperture; The types of operations that the image processing unit needs to perform include image stitching or image bonding. The panoramic camera augmented reality system according to claim 1, wherein the parameter determining the photographing specification is any one or more of a viewing angle, a focal length, an optical zoom multiple, a size and a resolution of an image sensor. The panoramic camera augmented reality system according to claim 1, wherein a tripod is connected below the casing, and a rotation mechanism is provided on the tripod, and the panoramic camera augmented reality system is disposed on the rotation mechanism. The panoramic camera augmented reality system according to claim 5, wherein the stand is made of a material with high thermal conductivity. The panoramic camera augmented reality system according to claim 5, wherein the second image acquisition device is provided on the tripod. The panoramic camera augmented reality system according to claim 3, wherein when the panoramic camera augmented reality system is switched to a manual mode, the image acquisition device is manually controlled to perform the operation. The panoramic camera augmented reality system according to claim 1, wherein the image acquisition device is connected with a power cord, and the end of the power cord extending away from the image acquisition device extends out of the casing and is connected to a battery . The panoramic camera augmented reality system according to claim 9, wherein the casing is a symmetric structure or an asymmetric structure, and a solar panel is fixed on the casing, and the solar panel is used to absorb sunlight and The absorbed sunlight is converted into electrical energy, and then the electrical energy is output to the battery to charge the battery. The panoramic camera augmented reality system according to claim 1, wherein the casing further includes one or more of an HDMI output interface, a USB interface, and a TF card slot.