JP7062386B2 - Camera and shooting system and image processing method - Google Patents

Description

The present invention relates to a camera, a shooting system using the camera, and a method of processing captured images, and in particular, synthesizes image data having different angles of view and image quality captured through a plurality of lenses. It relates to a camera, a shooting system, and an image processing method.

In recent years, cameras capable of acquiring wide-angle image (including video) data, such as wearable cameras and drive recorders, have become widespread. Most of them employ a technique as disclosed in Patent Document 1 in which a plurality of image data obtained by photographing different imaging ranges using a plurality of imaging means are combined to create one image data. ..

As another method, as disclosed in Patent Document 2, a so-called omnidirectional image having a horizontal angle of view of 360 degrees and a vertical angle of view of 180 degrees is acquired by a camera using a fisheye lens, and the omnidirectional image is obtained as a plane. Techniques such as converting to an image or cutting out a part of the image and converting it are known.

Further, in the field of image composition, there is also a technique disclosed in Patent Document 3 as a technique for synthesizing and displaying an image acquired through a fisheye lens and an image acquired through a so-called standard lens. The technique disclosed in Patent Document 3 is a technique of displaying an image in which a shooting range and a landscape around the shooting range are combined in order to determine an angle of view when shooting with a digital camera. Specifically, the omnidirectional image is acquired by a sub camera equipped with a fisheye lens, and the image of the shooting range by the main camera equipped with a standard lens is synthesized at the center of the image developed in a plane with the apex as the base point. be.

Japanese Unexamined Patent Publication No. 2016-92814 Japanese Unexamined Patent Publication No. 2012-147310 Japanese Patent No. 5733588

According to each technique disclosed in Patent Documents 1 and 2, it is possible to acquire wide-angle image data that cannot be obtained by shooting with a standard lens alone. However, in the technique disclosed in Patent Document 1, since a plurality of imaging means are fully operated to acquire image data, if an attempt is made to improve the image quality, the data capacity will be increased. ..

Further, in the technique disclosed in Patent Document 2, due to the characteristics of the fisheye lens, the distortion of the acquired image becomes larger toward the peripheral portion. For this reason, the correction ratio when flattening the image data is also large, and the deterioration of the image quality is large. Such deterioration of image quality is not a problem in normal use, but there is a problem that preferable image data cannot be obtained when it is desired to cut out a partial detailed image from image data.

Therefore, the present invention solves the above-mentioned problems, and a camera, a photographing system, and a photographing system capable of acquiring wide-angle image data using a wide-angle lens such as a fisheye lens and acquiring detailed image data of an image quality deterioration portion as needed. It is an object of the present invention to provide an image processing method.

The camera of the present invention for achieving the above object employs a wide-angle lens, and is arranged and positioned so as to cover the entire circumference of the camera with the center of the lens as the starting point, and the wide-angle lens. Also, a lens having a long focal distance is adopted, and the straight line passing through the lens center of the lens intersects with the straight line passing through the lens center of the wide-angle lens, and the focal distance is part of the photographing range of the first imaging means. The distortion is corrected by performing distortion correction for displaying the circular omnidirectional image data captured by the first imaging means and the second imaging means arranged so as to include the imaging range with a long lens. A process of synthesizing the second image data obtained by correcting the distortion of the image data captured by the second image pickup means to the portion where the image deterioration caused in the first image data obtained by the correction becomes large. In the compositing process by the arithmetic processing means, correction is performed in consideration of the difference in the amount of distortion between the first image data and the second image data, the difference in the image angle, and the difference in the shooting angle. A camera that features that.
Further, the camera having the above-mentioned characteristics may be provided with storage means for recording the first image data and the second image data, respectively.
Further, in a camera having the above-mentioned characteristics, the image data captured between the first image pickup means and the storage means and between the second image pickup means and the storage means can be obtained. A DSP that performs primary recording conversion may be provided.

Further, in a camera having the above-mentioned characteristics, the number of pixels of the image pickup element in the second image pickup means may be smaller than the number of pixels of the image pickup element in the first image pickup means. The second image pickup means has a narrower angle of view than the first image pickup means. Therefore, by having such a feature, it is possible to acquire image data having a higher resolution than the first image data while reducing the recording capacity of the image data. In addition, the load on the arithmetic processing means can be reduced and heat generation can be suppressed.

Further, in a camera having the above-mentioned characteristics, the arithmetic processing means manually or automatically selects at least one of shooting and recording by the first imaging means and shooting and recording by the second imaging means. It is good to shoot and record. According to having such a feature, when the first image data which is wide-angle shot is unnecessary or the second image data which can shoot the details is unnecessary depending on the shooting situation, the shooting format can be selected. , It is possible to reduce the processing load in the arithmetic processing means.

Further, in the case where the camera having the above-mentioned characteristics is provided with at least one of an acceleration sensor and a temperature sensor, and the photographing and recording by the first imaging means and / or the second imaging means are automatically selected. When a signal from the acceleration sensor or the temperature sensor is input, the arithmetic processing means may use the signal as a trigger. With such a feature, it is possible to automatically perform shooting and recording based on a signal from the sensor, and it is possible to adopt a shooting format according to the shooting environment.

Further, the camera having the above-mentioned characteristics includes an acceleration sensor, and when the arithmetic processing means determines that the signal from the acceleration sensor is a sudden state change exceeding a predetermined range, the acceleration sensor. It is also possible to perform a process of saving the recorded image for a predetermined time before and after the input of the signal from the acceleration sensor as a separate file by using the signal from. According to these characteristics, for example, when the camera is used as a drive recorder and the signal from the accelerometer is output due to an accident, the time before and after the signal is output is used as a reference. It is possible to leave a recorded image in the band. As described above, the image file recorded by the signal from the acceleration sensor as a trigger may be subjected to a process for prohibiting overwrite recording. By performing such processing, it is possible to reliably leave an image file in the event of an emergency such as an accident.

Furthermore, the camera having the above-mentioned characteristics includes a temperature sensor, and when the arithmetic processing means determines that the camera temperature exceeds a predetermined temperature range based on the signal from the temperature sensor, the acceleration. When the process of setting the pause mode to stop the functions other than the functions contributing to the sensor is performed and it is determined that the signal from the acceleration sensor is a sudden state change exceeding a predetermined range, the signal from the acceleration sensor is transmitted. As a trigger, it is also possible to perform a process of returning the function from the hibernation mode to the recording mode. With such a feature, it is possible to stabilize the operating state of the camera and at least record the image data immediately after the accident in the event of an accident or the like.

Further, the photographing system according to the present invention for achieving the above object is equipped with a camera having the above characteristics, and has a cloud having a driving means for changing the photographing direction by the first imaging means and the second imaging means. The arithmetic processing means includes a table, and the arithmetic processing means detects a specific subject from the image data taken by the first image pickup means, and the shooting range by the second image pickup means includes the subject. It is preferable that the control signal for changing the shooting direction by the camera is output to the driving means.

Further, in the image processing method according to the present invention for achieving the above object, a wide-angle lens is adopted, and a first imaging means arranged so as to cover the entire circumference of the camera with the center of the lens as the starting point is used. The first image data to be displayed in a plane by performing distortion correction on the circular omnidirectional image data to be photographed is acquired, and a lens having a longer focal distance than the wide-angle lens is adopted, and a straight line passing through the lens center of the lens is used. A second image pickup means arranged so that a straight line passing through the lens center of the wide-angle lens intersects and a part of the image pickup range of the first image pickup means includes the image pickup range of the lens having a long focal distance. This is an image processing method in which the second image data obtained by correcting the distortion of the image data captured by the lens is combined with a part of the first image data, and the second image data is the first image data. The shooting range is defined in the portion where the deterioration caused by the distortion correction performed at the time of acquisition becomes large, and the process of synthesizing the second image data with respect to the first image data is the distortion of the first image data and the second image data. The feature is that correction is performed in consideration of the difference in amount, the angle of view, and the difference in shooting angle .

Further, in the image processing method having the above-mentioned characteristics, the first image data and the second image data may be acquired by using the first image pickup means and the second image pickup means as separate cameras. .. Having such a feature makes it possible to use a hand-held wide-angle lens camera or a standard lens camera in carrying out the image processing method according to the invention.

According to a camera having the above-mentioned characteristics, a shooting system, and an image processing method, it is possible to acquire wide-angle image data using a wide-angle lens and, if necessary, acquire detailed image data of an image quality deterioration portion. become able to. Further, when a shooting system is used, it is possible to acquire a detailed image of a specific subject after shooting.

It is a block diagram which shows the structure of the camera which concerns on 1st Embodiment. It is a figure which shows the image | imaging range by the 1st image pickup means, and the range where the distortion (image deterioration) of an image is large. It is a figure which shows the relationship between the range where image deterioration is large when the image pickup range by the 1st image pickup means is expanded in a plane, and the image pickup range by a 2nd image pickup means. It is a block diagram which shows the structure of the camera which concerns on 2nd Embodiment. It is a side view which shows the example of the structural structure of the camera which concerns on 1st and 2nd Embodiment. It is a block diagram which shows the structure of the photographing system which concerns on 3rd Embodiment. It is a figure for demonstrating the relationship between the photographing range by a 1st image pickup means, the image pickup range by a 2nd image pickup means, and the movement of a camera.

Hereinafter, embodiments relating to the camera, the photographing system, and the image processing method of the present invention will be described in detail with reference to the drawings. In addition, the embodiment shown below is an example of the configuration necessary for obtaining the effect of the present invention, and the same effect can be obtained even if the combination, addition, and deletion of each element are performed. If so, it can be regarded as a part of the present invention.

[First Embodiment]
First, the camera according to the first embodiment will be described with reference to FIGS. 1 to 3. Note that FIG. 1 is a block diagram showing the configuration of the camera according to the first embodiment. Further, FIG. 2 is a diagram showing a shooting range by the first imaging means constituting the camera according to the embodiment and an image deterioration distribution, and FIG. 3 is a diagram showing image data taken in the range shown in FIG. It is a figure which shows the image deterioration distribution in the case of plane development.

The camera 10 according to the present embodiment includes at least a first image pickup means 12, a second image pickup means 14, an arithmetic processing means 16, and a storage means 18.
The first imaging means 12 is an imaging means that employs a wide-angle lens such as a fisheye lens as a lens for photographing. A specific configuration other than the lens can be a general configuration, and may include an image pickup element (not shown), a color filter, a microlens, an optical filter, and the like.

The second imaging means 14 is an imaging means that employs a lens (standard lens) having a longer focal distance (narrower angle of view) than the fisheye lens in the first imaging means 12 as a lens for photographing. The specific configuration other than the lens can be the same as that of the first image pickup means 12 described above. Here, in the present embodiment, it is desirable that the image pickup element in the second image pickup means 14 has a smaller number of pixels than the image pickup element in the first image pickup means 12.

The second image pickup means 14 has a narrower angle of view than the first image pickup means 12, and the distortion correction when displaying the image data is slight. Therefore, even if the number of pixels of the image sensor is smaller than that of the first image pickup means 12, the image data (first image data) having a higher resolution than the image data (first image data) acquired by the first image pickup means 12 (first image data). This is because 2 image data) can be obtained.

Further, by reducing the number of pixels of the image sensor, it is possible to reduce the recording capacity of the image data. Further, since the amount of data for image processing is also reduced, the load on the arithmetic processing means 16 can be reduced and heat generation can be suppressed.

The arithmetic processing means 16 converts and analyzes image data (first image data, second image data) captured by the first image pickup means 12 and the second image pickup means 14, as well as synthesis processing, and to various elements. It is an element for outputting a command signal and the like. In the present embodiment, as the arithmetic processing means 16, for example, a DSP (Digital Signal Processor) 16a, 16b for processing a captured image, a CPU for reading and arithmetic processing of various programs, and a CPU for outputting a command signal ( It is equipped with a Central Processing Unit) 16c and the like.

The DSPs 16a and 16b are provided with those corresponding to the first image pickup means 12 and the second image pickup means 14, respectively. In each of the DSPs 16a and 16b, the captured image data is created as a data file in a predetermined recording format, and primary recording conversion such as adjustment of the aspect ratio of the image data and distortion correction is performed. The first image data and the second image data for which the primary recording conversion is performed by the DSPs 16a and 16b are recorded in the storage means 18, respectively.

The storage means 18 includes external memories 18a and 18b such as a memory card, as well as an internal memory 18c in which programs for various processes and operations are recorded. The external memory may be configured to record both the first image data and the second image data in a single memory, but in the case of the present embodiment, it corresponds to each of the first image data and the second image data. It is configured to include external memories 18a and 18b. By recording the image data in parallel, the time required for the recording process can be shortened, and the capacity of the recording medium can be increased.

Further, in the internal memory 18c, for example, a conversion program required for primary recording conversion by each of the DSPs 16a and 16b, a synthesis program required for a synthesis process performed via the CPU 16c whose details will be described later, and the like are recorded.

The CPU 16c is an element for outputting various processes, operations for processing, various command signals, and the like. The camera 10 according to the present embodiment mainly performs a process of synthesizing the first image data and the second image data. Specifically, first, the synthesis program recorded in the internal memory 18c is read out and expanded. After that, the first image data and the second image data are read out, the recording dates and times of both are matched, and the synthesis process is performed.

The process of synthesizing the first image data and the second image data is to synthesize the second image data with a part of the image displayed as the first image data. The composition of the second image data with respect to the first image data is performed in the portion where the image deterioration becomes large when the distortion correction for displaying the first image data in a plane is performed.

The portion where the image deterioration is large corresponds to, for example, the portion shown by the diagonal line in FIG. The first image pickup means 12 employs a fisheye lens and is configured to acquire data of an omnidirectional image. Therefore, as shown in FIG. 2, the acquired image data has a circular imaging range, and the distortion of the image increases toward the outer peripheral side indicated by the diagonal line. In FIG. 2, the portion painted in black is a so-called vignetting portion that occurs when the display range is a square. In the DSP 16a, the acquired image data is developed in the form of the Mercator projection according to a conversion program developed, for example, via the CPU 16c or by directly accessing the internal memory 18c, and image data that can be displayed in a plane is generated. In the present embodiment, the image is cut and developed along a perpendicular line L shown in the radial direction with the point P, which is the apex of the image in FIG. 2, as a base point. For this reason, corrections such as enlargement and compression of the image are applied to the portion that becomes the edge of the image in FIG. 3, particularly the portion shown by the diagonal line, and the image deterioration becomes large.

In the present embodiment, a process of synthesizing the second image data is performed, for example, in the portion surrounded by the square broken line frame D in FIG. 3 among the portions where the image deterioration due to the distortion correction becomes large. In the process of synthesizing the second image data with respect to the first image data, various correction processes are performed in consideration of the difference in the amount of distortion between the two, the angle of view, the difference in the shooting angle, and the like.

Due to the above-mentioned synthesis processing, the first image pickup means 12 and the second image pickup means 14 have a part of the image pickup range by the first image pickup means 12 and the image pickup range by the second image pickup means 14. The arrangement of lenses is defined so that it is included. More specifically, the shooting range of the second image pickup means 14 is configured to match the portion of the shooting range of the first image pickup means 12 that is the edge of the first image data. With such an arrangement relationship, it is possible to perform a synthesis process under the above conditions.

The composite image data that has been composited via the CPU 16c is recorded in the external memory. The external memory for recording the composite image data may be configured to include a dedicated memory, but is configured to be recorded in either the external memory 18a for recording the first image data or the external memory 18b for recording the second image data. It may be. In this case, it is preferable to record in the external memory 18b. The image data (first image data, second image data) recorded in the external memories 18a and 18b are overwritten in order from the oldest recorded data when the recording capacity reaches a predetermined range. Here, since the second image pickup means 14 has a small number of pixels of the image pickup element, the capacity of the second image data recorded in the external memory 18b is also small. For this reason, if the configuration is such that the old recorded data is overwritten at the same timing as the overwriting timing of the first image data, there will be a margin in the free space of the memory.

According to the composite image data created by the camera 10 having such a configuration, the part where it is difficult to display the details in the first image data acquired by the first image pickup means 12 using the fisheye lens is standard. It is possible to supplement with the second image data taken by the second image pickup means 14 using a lens and display details. As a result, for example, when the camera 10 according to the embodiment is used as a drive recorder, the number of the preceding vehicle, the description of the road sign, and the like can be confirmed from the recorded image.

In the above description, the first imaging means 12 and the second imaging means 14 have been described as if they always shoot at the same time, but the CPU 16c can also selectively output a signal for starting shooting. .. Depending on the shooting conditions, there may be cases where the first image data for wide-angle shooting is not required or the second image data for which details can be shot is not required. Therefore, by selecting the shooting format, it is possible to reduce the processing load on the arithmetic processing means 16.

[Second Embodiment]
Next, a second embodiment according to the camera of the present invention will be described with reference to FIG. The camera 10A according to the present embodiment and the camera 10 according to the first embodiment have most of the same configurations. Therefore, the same reference numerals are given to the parts having the same configuration, and detailed description thereof will be omitted.

The difference between the camera 10A according to the present embodiment and the camera 10 according to the first embodiment is the presence or absence of various sensors. Specifically, the camera 10A according to the present embodiment is provided with a sensor such as an acceleration sensor 20 and a temperature sensor 22, and is configured to be able to control a shooting format, a recording mode, and the like based on a signal from the sensor. ing.

The acceleration sensor 20 is a sensor capable of detecting acceleration / deceleration applied to an object, a change in orientation, and the like. Therefore, when the acceleration (deceleration) detected by the acceleration sensor 20 is abrupt beyond a predetermined range, it can be determined that some state change has occurred. For example, if the accelerometer 20 detects a sudden change in speed when the camera 10A is used as a drive recorder, an accident (emergency) such as a collision may have occurred.

Normally, in the case of continuous shooting regardless of the recording intention such as a drive recorder, the CPU 16c starts shooting and recording by the first imaging means 12 and the second imaging means 14 at the same time as the input of the starting power. Output a signal. The image data taken by the first image pickup means 12 and the second image pickup means 14 is temporarily stored in the internal memory 18c, and when the recording capacity reaches a predetermined value or the number of recording files reaches a predetermined number. When it reaches, the process of overwriting and saving the oldest recorded file in order is repeated.

In the camera 10A of the present embodiment, when the CPU 16c determines that the signal from the acceleration sensor has reached a range indicating a sudden change of a predetermined value or more, this is used as a trigger, and a number from several seconds before and after the time is used as a reference. It is possible to extract the image data in minutes from the internal memory 18c and record it in the external memories 18a and 18b (pre- and post-recording mode). By performing such processing, when some sudden operation change occurs in the camera 10A, it is possible to leave evidence images for a few seconds to a few minutes before and after the change in the external memories 18a and 18b.

Further, such a recording file can also be subjected to a process for prohibiting overwriting recording. By executing such a process, it becomes possible to surely leave a record file in case of an emergency such as an accident.

On the other hand, the temperature sensor 22 plays a role as a sensor for detecting heat generated by the arithmetic processing means 16 such as the CPU 16c and the DSPs 16a and 16b constituting the camera 10A. The arithmetic processing means 16 tends to increase the amount of heat generated as the load state of information processing increases. When the calorific value exceeds a predetermined range, measures are taken to reduce the processing speed, stop the processing, and prevent equipment failure due to temperature rise.

However, if all the functions are stopped when the temperature rises, the camera 10A will not play a role as a recording device. Therefore, when it is determined that the calorific value detected by the temperature sensor 22 exceeds a predetermined range, the calorific value is reduced by stopping the functions other than the functions contributing to the acceleration sensor 20 and setting the hibernation mode. Suppress and try to cool the equipment. Then, when it is determined that the signal from the acceleration sensor 20 has reached a range representing a sudden change of a predetermined value or more, the paused function is restored by using this as a trigger, and the first imaging means 12 and Shooting by the second image pickup means 14 is resumed. Here, the image data captured by the first image pickup means 12 and the second image pickup means 14 is recorded in the external memories 18a and 18b (post-recording mode).

Therefore, in the camera 10A of the present embodiment, when the CPU 16c determines that the temperature detected by the temperature sensor 22 exceeds a predetermined temperature, it is possible to automatically take a safety measure for switching from the front-back recording mode to the back-recording mode. There is. By having such a function, it is possible to stabilize (safety) the operating state of the camera 10A and to record the image data immediately after that at least in the event of an emergency.

Further, as the load reducing measure performed by the CPU 16c, the image quality of the acquired image may be deteriorated while maintaining the shooting by both the first image pickup means 12 and the second image pickup means 14.

FIG. 5 shows a specific example of the appearance configuration when the cameras 10 and 10A according to the above embodiment are used as a drive recorder. As shown in FIG. 5, in the cameras 10 and 10A according to the embodiment, the lens center L1 of the first image pickup means 12 and the lens center L2 of the second image pickup means 14 are arranged so as to intersect each other. In particular, in the example shown in FIG. 5, the lens center L1 of the first image pickup means 12 is installed so as to be a perpendicular line, and the lens center L2 of the second image pickup means 14 is a horizontal line orthogonal to the lens center L1. It is configured in. As described above, unlike a conventional camera provided with two image pickup means as disclosed in Patent Document 3, for example, the present invention is configured so that the lens center of the image pickup means faces in a completely different direction. It can be said to be one of the features of the cameras 10 and 10A.

The chain line D1 shown in FIG. 5 indicates the angle of view (shooting range) of the first imaging means 12, and the chain line D2 indicates the angle of view (shooting range) of the second imaging means 14. The lens center L1 of the first image pickup means 12 having a wide angle of view is arranged on a perpendicular line, and the lens center L2 of the second image pickup means 14 having a narrow angle of view advances in a specific direction (in the example shown in FIG. 5). By arranging it toward the direction), the following shooting is possible.

That is, the first image pickup means 12 can acquire an image of the entire periphery centered on the cameras 10 and 10A. Here, in the vicinity of the chain line D1 corresponding to the outer edge of the shooting range of the first image pickup means 12, the image deterioration becomes large due to the distortion correction of the acquired image, but the shooting range by the second image pickup means 14 arranged in a specific direction. Of these, for the range where the broken line D1 and the broken line D2 intersect, a detailed image can be obtained by image composition processing.

[Third Embodiment]
Next, the photographing system 50 including the camera 10A according to the second embodiment and the pan head 30 on which the camera 10A is mounted will be described with reference to FIG.
In the photographing system 50 according to the present embodiment, when a specific subject is transferred to the image data captured by the first imaging means 12, the subject is included in the photographing range by the second imaging means 14. The pan head 30 equipped with the camera 10A is configured to have a function of changing the direction of the camera 10A.

Therefore, in the camera 10A, a subject detection program for detecting a specific subject from the first image data, an attitude control program based on the output signal of the acceleration sensor 20, and the like are recorded in the storage means (internal memory 18c) 18. It will be. Further, the camera 10A is provided with an interface 24 for exchanging control signals with and from the pan head 30 which is an external device. Here, the specific subject may be a human face or a moving object (ball, airplane, etc.) that shows a movement different from that of other objects to be photographed, and is within a range that can be identified and recognized from the image data by a known algorithm. Anything is fine.

The pan head 30 is provided with at least a two-axis actuator (driving means) 32, and is configured to be able to move the direction of the mounted camera 10A up, down, left and right. A control signal from the camera 10A is input to the actuator 32 via the interface 34.

According to the photographing system 50 having such a configuration, the first image data, which is a wide-angle image captured by the first imaging means 12, is analyzed via the subject detection program. When it is determined by the analysis that a specific subject has moved within the shooting range of the first image data, a control signal is output to the actuator 32 of the pan head 30 via the attitude control program. ..

As the control signal, the current posture of the camera 10A (the orientation of the lens in the second image pickup means 14) is obtained from the output signal from the acceleration sensor 20 provided in the camera 10A. After that, the deviation between the position of the subject detected from the first image data and the shooting range by the current second image pickup means 14 is calculated, and the moving angle of the camera 10A and the like are obtained. Based on the obtained movement angle, a control pulse for the actuator 32 is generated and output as a control signal.

With such a configuration, in the photographing system 50 according to the present embodiment, the specific subject A is reflected in the first image data photographed by the first imaging means 12 as shown in FIG. 7 (A). At that time, as shown in FIG. 7B, the direction of the camera 10A can be changed so that the shooting range (broken line frame D) taken by the second image pickup means 14 can be adjusted to the subject A. This makes it possible to confirm the detailed image of the subject A after shooting. The method for specifying the subject A may be the detection of a moving portion as described above, the detection by a person or a face, or the detection by a specific color. Further, the face of a specific person may be detected by inputting the detection data in advance.

In the above embodiment, it has been described that the first image pickup means 12 and the second image pickup means 14 are mounted on one camera 10 and 10A, and the image data taken by the two image pickup means is synthesized. However, in implementing the image processing method according to the present invention, even if the first image pickup means 12 and the second image pickup means 14 are used as separate cameras, the first image data and the second image data are acquired. good.

Specifically, a camera equipped with a wide-angle lens such as a fisheye lens is adopted as the first imaging means 12, and a camera equipped with a standard lens, a mobile phone with a camera function, or the like is used as the second imaging means 14. Adopt a device with a camera function. Each of the two cameras sets the shooting time zone and shoots so that the shooting range of the second image pickup means 14 matches a part of the shooting range of the first image pickup means 12.

Each of the captured image data (first image data and second image data) is taken into a computer or the like (not shown) via external memories 18a and 18b, and distortion correction and composition are performed by various arithmetic processes. It's fine.

When such a method is adopted, it is possible to take a picture and perform image processing using a wide-angle lens camera or a standard lens camera on hand.

10, 10A ……… Camera, 12 ……… First image pickup means, 14 ……… Second image pickup means, 16 ……… Arithmetic processing means, 16a ……… DSP, 16b ……… DSP, 16c… …… CPU, 18 ………… Storage means, 18a ………… External memory, 18b ………… External memory, 18c ………… Internal memory, 20 ………… Acceleration sensor, 22 ………… Temperature sensor, 24 ………… Interface, 30 ……… Cloud stand, 32 ……… Actuator, 34 ……… Interface, 50 ……… Shooting system.

Claims (11)

A first imaging means that uses a wide-angle lens and is arranged so that the entire circumference of the camera is the shooting range starting from the center of the lens.
A lens having a longer focal length than the wide-angle lens is adopted, and a straight line passing through the lens center of the lens intersects with a straight line passing through the lens center of the wide-angle lens, and a part of the shooting range of the first imaging means. A second image pickup means arranged so as to include a shooting range by the lens having a long focal length.
Distortion correction is performed to display the circular omnidirectional image data captured by the first image pickup means in a plane, and the portion where the image deterioration that occurs in the first image data obtained by performing the distortion correction is large. It is provided with an arithmetic processing means for performing a process of synthesizing the second image data obtained by correcting the distortion of the image data captured by the second image pickup means .
The compositing process by the arithmetic processing means is characterized in that correction is performed in consideration of the difference in the amount of distortion, the angle of view, and the difference in the shooting angle between the first image data and the second image data . The camera according to claim 1, further comprising a storage means for recording the first image data and the second image data, respectively. A DSP for performing primary recording conversion of captured image data is provided between the first image pickup means and the storage means, and between the second image pickup means and the storage means, respectively. 2. The camera according to claim 2. The camera according to any one of claims 1 to 3, wherein the number of pixels of the image pickup element in the second image pickup means is smaller than the number of pixels of the image pickup element in the first image pickup means. The arithmetic processing means is characterized in that at least one of the shooting and recording by the first imaging means and the shooting and recording by the second imaging means is manually or automatically selected for shooting and recording. The camera according to any one of claims 1 to 4. Equipped with at least one of accelerometer and temperature sensor,
When shooting and recording by the first imaging means and / or the second imaging means are automatically selected.
The camera according to claim 5, wherein the arithmetic processing means performs the photographing and the recording by using the signal as a trigger when a signal from the acceleration sensor or the temperature sensor is input. Equipped with an accelerometer,
When the arithmetic processing means determines that the signal from the acceleration sensor is a sudden state change exceeding a predetermined range, the signal from the acceleration sensor is used as a trigger before and after input of the signal from the acceleration sensor. The camera according to claim 5, wherein a process of storing a recorded image for a predetermined time as a separate file is performed. Equipped with a temperature sensor,
Based on the signal from the temperature sensor, the arithmetic processing means performs a process of setting a pause mode in which a function other than the function contributing to the acceleration sensor is stopped when it is determined that the camera temperature exceeds a predetermined temperature range. ,
When it is determined that the signal from the acceleration sensor is a sudden state change exceeding a predetermined range, the process of returning the function from the pause mode to the recording mode is performed by using the signal from the acceleration sensor as a trigger. The camera according to claim 7. The camera according to any one of claims 1 to 8 is mounted, and a pan head having a driving means for changing the shooting direction by the first image pickup means and the second image pickup means is provided.
The arithmetic processing means detects a specific subject from the image data taken by the first image pickup means, and takes a picture with the camera so that the shooting range by the second image pickup means includes the subject. An imaging system characterized in that a control signal for changing a direction is output to the driving means. A wide-angle lens is used, and distortion correction is performed on the circular omnidirectional image data captured by the first imaging means arranged so that the entire circumference of the camera is the shooting range starting from the center of the lens and displayed in a plane. In addition to acquiring the first image data, a lens having a longer focal distance than the wide-angle lens is adopted, and the straight line passing through the lens center of the lens intersects with the straight line passing through the lens center of the wide-angle lens, and the first The second image data obtained by correcting the distortion of the image data captured by the second imaging means arranged so that the imaging range by the lens having a long focal distance is included in a part of the imaging range of the imaging means of the above. Is an image processing method for synthesizing a part of the first image data.
In the second image data, a shooting range is defined in a portion where deterioration caused by distortion correction performed when acquiring the first image data becomes large.
The process of synthesizing the second image data with respect to the first image data is characterized in that correction is performed in consideration of the difference in the amount of distortion between the first image data and the second image data, and the difference in the angle of view and the shooting angle. Image processing method. The image processing method according to claim 10, wherein the first image pickup means and the second image pickup means are used as separate cameras to acquire the first image data and the second image data.

Images