JP5464290B2 - Control device, control method, and camera system - Google Patents

Description

  The present disclosure relates to a control device, a control method, and a camera system.

  2. Description of the Related Art Conventionally, as described in Patent Document 1 below, for example, an imaging system is known that aims to quickly move the imaging direction of an imaging device in a desired direction.

JP 2007-43505 A

  The technique described in Patent Literature 1 controls to move the imaging direction of the imaging device based on a rectangular panoramic image. However, there arises a problem that the control based on the panoramic image is not intuitive only when the imaging target corresponding to the panoramic image is a rectangle.

  Therefore, there has been a demand for more intuitive control of the imaging direction of the imaging apparatus based on the panoramic image even when the imaging target is other than a rectangle.

  According to the present disclosure, an acquisition unit that acquires a camera image captured by a camera, a panoramic image display area that displays a panoramic image, and a partial image corresponding to a position specified on the panoramic image A display control unit for controlling to display a partial image display area, an input unit for inputting positional information on the panoramic image, and the panoramic image and the part based on the camera image A generation unit that generates at least one of the images, wherein the panorama image is a circular panorama image, and the display control unit is configured such that a position specified on the panorama image is positioned on a predetermined axis. Thus, a control device for adjusting the position of the panoramic image displayed in the panoramic image display area is provided.

  The circular panoramic image may be generated based on an image captured a plurality of times by the camera.

  The display control unit may switch the panorama image displayed in the panorama image display area from a circular panorama image to a rectangular panorama image in response to a panorama image switching operation.

  The display control unit may display a rectangular panoramic image together with a circular panoramic image in the panoramic image display area.

  The display control unit may be capable of moving the panoramic image display area on the screen.

  In addition, according to the present disclosure, a camera image captured by a camera is acquired, a panorama image display area for displaying a panorama image, and a partial area corresponding to a position specified on the panorama image are partially Control to display a partial image display area to be displayed as an image, input of positional information on the panoramic image, and based on the camera image, the panoramic image and the partial image The panoramic image is a circular panoramic image, and includes a part of the panoramic image display area and a part of the area corresponding to a position specified on the panoramic image. In the control of displaying a partial image display area to be displayed as an image, the path is set so that a position designated on the panoramic image is located on a predetermined axis. Adjusting the position of the panorama image displayed on the llama image display area, the control method is provided.

  Further, according to the present disclosure, the camera system includes a camera that captures an image and a control device that controls the camera, and the control device acquires an camera image captured by the camera; A display control unit for controlling to display a panoramic image display area for displaying a panoramic image, and a partial image display area for displaying a partial area corresponding to a position designated on the panoramic image as a partial image; An input unit for inputting positional information on the panoramic image, and a generation unit for generating at least one of the panoramic image and the partial image based on the camera image, A circular panoramic image, wherein the display control unit is configured to display the panoramic image display area so that a position designated on the panoramic image is located on a predetermined axis. Show camera system is provided.

  According to the present invention, it is possible to provide a control device, a control method, and a camera system that can more intuitively control the imaging direction of the imaging device on a panoramic image.

1 is a schematic diagram illustrating an overall configuration of an imaging system according to an embodiment of the present disclosure. FIG. 2 is a schematic diagram illustrating in detail the configuration of a camera terminal device and a center server in the system of FIG. 1. 6 is a schematic diagram for explaining a pan operation and a tilt operation of the camera terminal device 200. FIG. It is a schematic diagram which shows the display state of a panoramic image and a bird's-eye view image. It is a schematic diagram for demonstrating the process which produces | generates the bird's-eye view image signal DF. It is a schematic diagram for demonstrating the process which produces | generates the bird's-eye view image signal DF. It is a schematic diagram for demonstrating the process which produces | generates the bird's-eye view image signal DF. It is a schematic diagram for demonstrating the process which produces | generates the bird's-eye view image signal DF. It is a schematic diagram which shows the display screen of a monitor. It is a schematic diagram for demonstrating operation by an operation panel. FIG. 10 is a schematic diagram showing a state in which menus such as “Screen Mode”, “View Size”, “Image Codec”, and “Frame Rate” are displayed when the panel “View” is clicked. It is a schematic diagram showing a menu displayed when the panel “Camera” is clicked. It is a schematic diagram showing each menu displayed when “Preset position”, “Trigger”, “Other”, “information” is clicked. It is a schematic diagram for demonstrating the display of a bird's-eye view image. It is a schematic diagram for demonstrating the display of a bird's-eye view image. It is a schematic diagram for demonstrating the display of a bird's-eye view image. It is a schematic diagram for demonstrating the display of a bird's-eye view image. It is a schematic diagram for demonstrating the display of a bird's-eye view image. It is a schematic diagram which shows the example which displayed the panoramic image on the cylindrical curved surface instead of the normal panoramic image, and made it a reference image. It is a schematic diagram for demonstrating the effect by the display of a 1st panoramic image. It is a schematic diagram for demonstrating the effect by the display of a 1st panoramic image. It is a schematic diagram for demonstrating the effect by the display of a 1st panoramic image.

  Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In addition, in this specification and drawing, about the component which has the substantially same function structure, duplication description is abbreviate | omitted by attaching | subjecting the same code | symbol.

The description will be made in the following order.
1. First embodiment (an example of an imaging system including a camera terminal device and a server)
(1) Configuration example of imaging system (2) Configuration example of camera terminal device and center server (3) Display operation of first panorama image and second panorama image (4) Processing to generate first panorama image (5) About the GUI (6) Specifying the shooting direction on the first panoramic image

<1. First Embodiment>
(1) Configuration Example of Imaging System FIG. 1 is a schematic diagram illustrating an overall configuration of an imaging system according to an embodiment of the present disclosure. The imaging system 100 includes a camera terminal device (IP camera) 200, a center server 300, and a client 400. The plurality of camera terminal devices 200, the center server 300, and the client 400 are connected via a network 500. . Further, a proxy server 600 is connected to the network 500, and a plurality of camera terminal devices 200 are connected to the proxy server 600. A monitor 310 is connected to the center server 300. In the present embodiment, the camera terminal device 200 is a surveillance camera installed indoors or outdoors, for example, and displays an image acquired by the camera terminal device 200 on the monitor 310 of the center server 300, whereby the camera terminal device 200. You can monitor the area where is installed. The client 400 is connected to the camera terminal device 200, the center server 300, or the proxy server 600 via the network 500. Further, the client 400 may have a function of the center server 300 and a function of the monitor 310 described later. Video captured by the camera terminal device 200 is sent to the client 400 via the network 500, and a first panoramic image and a second panoramic image are displayed on the display screen of the client 400 together with a live video described later. In addition, although the IP camera is mentioned as an example of the camera terminal apparatus 200, it is not limited to this, An analog camera may be sufficient.
The panoramic image indicates a range that can be captured by the camera terminal device 200. The panoramic image has various shapes, for example, a circular panoramic image or a rectangular panoramic image as shown in FIG. 4, a ring-shaped panoramic image projected onto a cylindrical surface as shown in FIG. is there. Hereinafter, in the embodiment, a circular panorama image and a rectangular panorama image will be described as examples of the first panorama image and the second panorama image, respectively.

(2) Configuration Example of Camera Terminal Device and Center Server FIG. 2 is a schematic diagram showing in detail the configuration of the camera terminal device 200 and the center server 300 in the system of FIG. Each component of the camera terminal device 200 or the center server 300 shown in FIG. 2 can be configured by hardware (circuit or the like) or an arithmetic processing device such as a CPU and software (program) for causing it to function. The camera unit 201 of the camera terminal device 200 includes an imaging optical system, performs an imaging operation based on an imaging control signal CTa supplied from a terminal control unit 210 described later, and generates a video signal Dcam. In addition, the camera unit 201 supplies the generated video signal Dcam to the signal processing unit 202 via the bus 220. Note that a memory unit 203, an imaging direction control unit 204, a network interface unit 206, a storage unit 207, and a terminal control unit 210 are also connected to the bus 220.

  The signal processing unit 202 performs compression processing of the video signal Dcam, and stores the obtained video encoded signal DV in the memory unit 203. Also, the video signal of the first panorama image (hereinafter referred to as “first panorama image signal” DF and the image of the second panorama image is used by using the video signal Dcam obtained by sequentially moving the imaging direction of the camera unit 201. A signal DP (hereinafter referred to as “second panorama image signal”) DP is generated and stored in the storage unit 207. Note that the compression processing of the video signal Dcam, the first panorama image signal DF, or the second panorama image signal is performed. The DP is generated based on a signal processing control signal CTb supplied from the terminal control unit 210 described later.

  The imaging direction control unit 204 moves the imaging direction of the camera unit 201 in a direction instructed by a direction control signal CTc supplied from the terminal control unit 210 to be described later. Perform the calculation. Further, a drive signal MDp for performing a pan operation and a drive signal MDt for performing a tilt operation are generated based on the calculation result, and supplied to the imaging direction adjustment unit 205. In addition, the imaging direction control unit 204 generates a camera position information signal PM indicating which direction the imaging direction of the camera unit 201 is, and supplies the camera position information signal PM to the terminal control unit 210.

  The imaging direction adjustment unit 205 includes a pan operation motor that moves the camera unit 201 in the left-right direction and a tilt operation motor that moves the camera unit 201 in the up-down direction. The imaging direction adjustment unit 205 adjusts the imaging direction of the camera unit 201 to the direction instructed by the direction control signal CTc by driving the pan operation motor by the drive signal MDp and the tilt operation motor by the drive signal MDt, respectively. In addition, the imaging direction adjustment unit 205 performs an endless pan operation without limiting the operation range.

  FIG. 3 is a schematic diagram for explaining the pan operation and tilt operation of the camera terminal apparatus 200, and shows a case where the camera terminal apparatus 200 is mounted on a ceiling or the like. Here, FIG. 3B shows a pan / tilt operation of the camera terminal device 200 according to the present embodiment. On the other hand, FIG. 3A shows a pan / tilt operation of a general camera terminal device for comparison. As shown in FIG. 3A, in a general camera terminal device, pan operation is performed endlessly at 360 °, and tilt operation is performed in a range of 90 ° from the vertical direction to the horizontal direction. On the other hand, the camera terminal device 200 according to the present embodiment can perform the pan operation endlessly at 360 °, and can perform the tilt operation within a range of 220 ° centering on the vertical direction. Therefore, the video signal DF of the first panoramic image can be generated and stored in the storage unit 207 by a method described later. A first panoramic image to be described later is a circular panoramic image, and when operating the imaging direction across the center of the circle, the camera terminal device 200 according to the present embodiment is the shortest to the target imaging direction. Can be moved mechanically. On the other hand, in a general camera terminal device, the camera imaging direction cannot be mechanically moved so as to cross the center of a circle.

  The network interface unit 206 is an interface for performing communication between the camera terminal device 200 and the center server 300 via the network 500.

  The terminal control unit 210 controls the camera unit 201 by the imaging control signal CTa, controls the signal processing unit 202 by the signal processing control signal CTb, controls the imaging direction control unit 204 by the direction control signal CTc, and sets the imaging direction. Imaging is performed while sequentially moving. Then, the terminal control unit 210 generates the first panorama image signal DF and the second panorama image signal DP based on the obtained video signal Dcam and stores them in the storage unit 207.

  The imaging system 100 according to the present embodiment includes a camera terminal device that includes both a live video based on the video signal Dcam and the first panorama image or the second panorama image based on the first panorama image signal DF or the second panorama image signal DP. 200 can be transmitted to the center server 300 and displayed on the monitor 310. In the present specification, the first panorama image and the second panorama image may be referred to as a reference image. FIG. 4 is a schematic diagram illustrating a display state of the second panorama image and the first panorama image. The center server 300 can display either the first panorama image or the second panorama image together with the live video on the monitor 310 according to the user's selection. The center server 300 may be configured to display both the first panorama image and the second panorama image together with the live video. In the right diagram in FIG. 4, a circular panoramic image lacking an image near the center is used as the first panorama image. However, the present invention is not limited to this. One panoramic image may be used.

  The camera terminal device 200 is installed on a ceiling or the like, and displays a downward image of the camera terminal device 200. Therefore, as shown in FIG. 4, the first panoramic image is acquired in a range of pan 360 ° and tilt 220 ° with a vertical line extending downward from the camera terminal device 200 as a center, and its outline is circular. It becomes.

  Further, the imaging direction adjustment unit 205 is configured to be able to perform an endless pan operation as described above. Accordingly, the second panoramic image stored in the storage unit 207 is, for example, as illustrated in FIG. 4, so that one end portion has an angle difference of “+ 180 °” from the reference direction and the other end portion so that the images do not overlap. Is assumed to have an angle difference of “−180 °” from the reference direction. Further, since the operation range of the pan operation is not limited, the center position of the operation range cannot be set as the reference direction as in the case where the operation range is limited, for example. Accordingly, a reference direction is set in advance, and the second panorama image signal DP is generated so that the reference direction becomes the center position of the second panorama image. Further, the imaging direction control unit 204 generates, as the camera position information signal PM, a signal indicating, for example, an angle difference between the imaging directions with respect to a preset reference direction.

  In this way, if the reference direction is set in advance, it is not necessary to determine which direction the second panoramic image is generated by the imaging operation based on which direction. If the camera position information signal PM indicates an angle difference between the image capturing direction and the reference direction, the position in the image capturing direction based on the camera position information signal PM on the second panoramic image can be easily determined.

  Also in the tilt operation, a reference direction is set in advance, and the angle difference between the imaging direction and the reference direction is indicated by the camera position information signal PM. Thereby, the position in the imaging direction based on the camera position information signal PM can be easily determined in both the first panorama image and the second panorama image. Regarding the tilting operation, when the camera terminal device 200 is installed on the ceiling, the vertical direction can be set as the reference direction.

  The terminal control unit 210 analyzes the command signal CM supplied from the center server 300 via the network interface unit 206, and generates an imaging control signal CTa, a signal processing control signal CTb, and a direction control signal CTc. In addition, the video encoded signal DV stored in the memory unit 203, the first panorama image signal DF, the second panorama image signal DP, and the imaging direction control unit 204 stored in the storage unit 207 are supplied. The camera position information signal PM is sent to the center server 300.

  The network interface unit 301 of the center server 300 is an interface for performing communication between the camera terminal device 200 and the center server 300 via the network 500. The network interface unit 301 supplies the video encoded signal DV supplied from the camera terminal device 200 to the decompression processing unit 302. The network interface unit 301 supplies the camera position information signal PM to the position determination processing unit 303 and supplies the first panorama image signal DF and the second panorama image signal DP to the image processing unit 304.

  The decompression processing unit 302 performs decompression processing of the video encoded signal DV and supplies the obtained video signal Dcam to the display processing unit 305.

  The position determination processing unit 303 determines whether the imaging direction indicated by the camera position information signal PM corresponds to the position of the first panorama image or the second panorama image, and uses the position determination result KP as the image processing unit. 304 is supplied. Here, the camera position information signal PM generates the first panorama image signal DF or the second panorama image signal DP so that the preset reference direction is the center position of the pan operation and tilt operation as described above. Assuming that the camera position information signal PM indicates an angle difference between the reference direction and the current imaging direction, the current imaging direction is at either the overhead image or the second panoramic image based on the camera position information signal PM. It can be easily determined whether it corresponds.

  The image processing unit 304 performs image processing based on the position determination result KP, and generates a first panorama image signal DFC of the first panorama image and a second panorama image signal DPC of the second panorama image. As for the second panorama image signal DPC, the second panorama image signal DPC in which the position determined by the position determination result KP is the center position can be generated. In this case, the image processing unit 304 determines a difference amount between the determined position and the center position of the second panoramic image, and based on the determined difference amount, the determined position becomes the center position of the image. The second panoramic image is processed so that That is, the second panorama image display image frame of “± 180 °” is set with the determined position as the center position, and the second panorama image in the region corresponding to the difference amount deviating from the second panorama image display image frame is set. Is pasted into a region without an image to generate a second panoramic image signal DPC of the second panoramic image with the determined position as the center position. Even in the case of the first panorama image signal DFC, in the case of a “first panorama image rotation mode” to be described later, as in the case of the second panorama image signal DPC, the determined position and the first panorama image are detected. A difference amount from the reference position in the pan direction is determined, and the first panoramic image is processed based on the determined difference amount so that the determined position is positioned on the image. In addition, the image processing unit 304 determines the position based on the position determination result KP (the position where the optical axis of the imaging optical system of the camera unit 201 faces) in each of the first panorama image signal DFC and the second panorama image signal DPC. ) Is displayed. Further, the image processing unit 304 supplies the first panorama image signal DFC and the second panorama image signal DPC generated by performing image processing to the display processing unit 305.

  The display processing unit 305 uses the video signal Dcam supplied from the decompression processing unit 302 and the first panorama image signal DFC and the second panorama image signal DPC supplied from the image processing unit 304 to generate the display drive signal HD. Generated and supplied to the monitor 310. Further, the display processing unit 305 generates a display drive signal HD by using a GUI video signal generated based on a GUI display control signal CTg supplied from the control unit 310 described later, and supplies the display drive signal HD to the monitor 310.

  The monitor 310 drives a display element such as a liquid crystal display element, a plasma display element, or a cathode ray tube on the basis of the display drive signal HD, and the first panorama image, the second panorama image with the imaging direction as the center position, A captured image (live video) of the camera unit 201, a GUI image, and the like are displayed on the screen.

  The user interface unit 315 uses a GUI, presents information to the user using the monitor 310, and when a user operation is performed with a pointing device or a keyboard as an operation input unit based on the presented information Then, an operation signal US corresponding to a user operation is supplied to the control unit 310 to perform a desired operation or the like.

  Based on the operation signal US and the display state of the monitor 310, the control unit 310 determines what kind of processing the user has requested to select and what kind of processing is executed, and generates a control signal CTm based on the determination result. To control the operation of the center server 300. Further, a command signal CM is generated based on the determination result, supplied to the camera terminal device 200 via the network interface unit 301, and the operation of the camera terminal device 20 is controlled. Further, the control unit 310 generates a GUI display control signal CTg and supplies it to the display processing unit 305.

  In the above-described example, the camera terminal device 200 generates the first panorama image signal DF and the second panorama image signal DP from the image signal Dcam. However, the center server 300 or the client 400 performs the first panorama image signal DP. The panoramic image video signal DF and the second panoramic image video signal DP may be generated. In this case, the network interface unit 206 of the camera terminal device 200 sends the video signal Dcam output from the camera block 201 to the center server 300 or the client 400. The center server 300 or the client 400 includes a signal processing unit similar to that of the camera terminal device 200, and can generate the video signal DF of the first panoramic image and the video signal DP of the second panoramic image. The video signal DF of the first panorama image and the video signal DP of the second panorama image are stored in a memory such as a hard disk provided in the center server 300 or the client 400.

(3) Display Operation of First Panorama Image and Second Panorama Image Next, the first panorama image and second panorama image display operation will be described. At the start of operation, the control unit 310 requests the camera terminal apparatus 200 for the first panorama image signal DF, the second panorama image signal DP, the camera position information signal PM, and the video encoding signal DV. Send. When the first panorama image signal DF and the second panorama image signal DP are stored in the storage unit 207, the camera terminal device 200 reads out the first panorama image signal DF and the second panorama image signal DP. To the center server 300. In addition, when the first panorama image signal DF or the second panorama image signal DP is not stored in the image storage unit 207, the camera terminal device 200 controls the camera unit 201, the signal processing unit 202, and the imaging direction control unit 204. The image pickup operation is performed while moving the image pickup direction, and the first panorama image signal DF and the second panorama image signal DP are generated and transmitted to the center server 300. The camera terminal device 200 stores the generated first panorama image signal DF and second panorama image signal DP in the storage unit 207.

  The camera terminal apparatus 200 supplies the camera position information signal PM generated by the imaging direction control unit 204 to the center server 300. In addition, the camera terminal device 200 starts an imaging operation of the camera unit 201, compresses the obtained video signal Dcam by the signal processing unit 202, and stores the video encoded signal DV in the memory unit 203. Further, the encoded video signal DV stored in the memory unit 203 is supplied to the center server 300. As shown in FIG. 1, when the camera terminal device 200 and the network 500 are connected via a proxy server 600, information exchange between the camera terminal device 200 and the center server 300 is performed by the proxy server 600. Is done via.

  The position determination processing unit 303 of the center server 300 determines whether the imaging direction indicated by the camera position information signal PM corresponds to the position of the first panorama image or the second panorama image, and the position determination result KP. Is supplied to the image processing unit 304. Taking the second panoramic image as an example, for example, when the angle difference indicated by the camera position information signal PM is “0 °”, the imaging direction is set to the reference direction, so the position in the imaging direction is the second. This is the center position of the panoramic image.

  The image processing unit 304 performs image processing based on the position determination result KP. Here, when the position determination result KP indicates that the position in the imaging direction is the center position of the second panoramic image, the image in the imaging direction is the center of the second panoramic image. The second panorama image signal DP is supplied to the display processing unit 305 as the second panorama image signal DPC without performing the second panorama image processing. On the other hand, when the position in the imaging direction is not the center position of the second panoramic image, the difference amount between the determined position in the imaging direction and the center position of the second panoramic image is determined, and the determined difference amount is Based on this, the second panorama image is processed so that the determined position becomes the center position of the image, and the second panorama image signal DPC is generated.

  For this reason, in the second panoramic image, for example, when the angle difference indicated by the camera position information signal PM is “0 °”, it is displayed on the screen of the monitor 310 driven by the display drive signal HD from the display processing unit 305. Displays a captured image captured by the camera unit 201, a second panoramic image whose reference direction is the center position, and an image related to the GUI.

  In the case of the first panorama image, in the case of a “first panorama image rotation mode” to be described later, as in the case of the second panorama image signal DPC, the determined position and the pan direction of the first panorama image are changed. A difference amount from the reference position is determined, and based on the determined difference amount, the first panoramic image is rotated so that the determined position is positioned below the image (on a straight line L described later), 1 panoramic image signal DFC is generated. The image processing unit 304 also determines the position in the imaging direction based on the position determination result KP (the optical axis of the imaging optical system of the camera unit 201 is oriented) in each of the first panorama image signal DFC and the second panorama image signal DPC. Display position).

  The decompression processing unit 302 decompresses the video encoded signal DV and supplies the obtained video signal Dcam to the display processing unit 305. Further, the control unit 310 supplies a GUI display control signal CTg for performing GUI display to the display processing unit 305.

(4) Processing for Generating First Panorama Image Next, processing for generating the first panorama image signal DF will be described with reference to FIGS. When generating the first panorama image signal DF, an image is acquired for each region of the first panorama image. First, as shown in FIG. 5, the zoom of the camera terminal device 200 is fixed, panning and tilting are performed at certain arbitrary intervals, and a still image is acquired. The numbers shown in FIG. 5 indicate the order of image acquisition.

Next, coordinate conversion of the still image coordinate system is performed by the following procedures [1] to [3]. Here, as shown in FIG. 6, each point of the still image is converted into a relative position (x, y, z) viewed from the rotation center of the camera terminal device 200.
[1] A camera with a horizontal angle of view thh [degree] and an aspect ratio of a is set at the origin, and is directed in the -z direction. Considering the object plane (x, y, -L) at the position of distance L, the visible range is a rectangle of 2Lx × 2Ly. Here, the relationship of Lx = L · tan (thh / 2) and Ly = Lx / a is established.
[2] Although the camera posture is originally (x, y, z, p, q, r), x = y = z = 0, assuming that the viewpoint position is fixed at the origin. Now consider only (p, q, r) (p: Pan (y-axis rotation), q: Tilt (x-axis rotation), r: roll (z-axis rotation)).
[3] (2Lx) × (2Ly) point (j,
When i) is used, the following equation holds.
x = Lx × (j-Lx) / Lx
y = Ly × (i-Ly) / Ly
z = -L
[4] The following roll conversion, tilt conversion, and pan conversion formula are applied using (x, y, z).

  The obtained (x3, y3, z3) is a value represented by a coordinate system when a point (x, y, z) in the still image is viewed from the rotation center of the camera.

Next, the relative position is converted into latitude and longitude. Here, as shown in FIG. 7, (x3, y3, z3) obtained above is converted into latitude and longitude. (x3, y3,
The angle formed by the direction of z3) with respect to the -z direction is the longitude, and the angle formed by the xz plane is the latitude.
・ Latitude
La = tan ^-1
(y3 / :: sqrt (x3 * x3 + z3 * z3))
・ Longitude (Lo)
When 0.0 <z3
If 0.0 <x3 Lo = 90.0 + La
When 0.0 ≧ x3 Lo = -90.0-La
When 0.0 ≧ z3
Lo = tan ^-1 (x3 / z3) / RAD
However,
RAD = π / 180 (π: Pi ratio)

  Next, conversion into a first panoramic image is performed. Here, as shown in FIG. 8, the obtained latitude and longitude (La, Lo) are represented by a latitude axis and a longitude coordinate system in the circumferential direction.

  As described above, the first panorama image signal DF can be generated.

(5) GUI Next, the GUI will be described. FIG. 9 is a schematic diagram showing a display screen of the monitor 310. FIG. 9 shows both a normal screen and a full screen display. In either case, an operation panel (control panel) is displayed on the left side of the screen and a live image is displayed on the right side of the screen. When “Screen Mode” on the operation panel is set to “Full”, the full screen is displayed.

FIG. 10 is a schematic diagram for explaining an operation by the operation panel. The operation panel is a panel for each function, and each panel can be folded and stored. The panel classification is as follows.
・ View
Display settingsCamera control
Settings related to camera operationPanorama

Panorama display / Preset position
Settings related to presets ・ Trigger
Trigger settings-Other
Other settings

Clicking on each panel opens the panel. FIG. 10A shows a state where “View” is clicked, and a menu is displayed under the “View” column. On the other hand, FIG. 10B shows a state where “Camera control” is clicked.
A menu is displayed under the “control” column, and FIG. 10C shows a state in which the panel “Camera control” is dragged, and the panel “Camera control” can be made a floating window by dragging. it can.

FIG. 11 shows a state where menus such as “Screen Mode”, “View Size”, “Image Codec”, and “Frame Rate” are displayed when the panel “View” is clicked. “Screen
The “Mode” menu allows you to change the display mode between normal and full screen, and “View Size” allows you to specify 1/4, 1/2, 1x, Fit mode of the image. “Image Codec” to display the image accordingly.
In the menu, Codec can be changed. In the “Frame Rate” menu, the JPEG frame rate can be changed. The movie save button is used for saving a movie, and the still image save button is used for saving a still image. In addition, a microphone volume slider, a microphone mute on / off button, an audio volume slider, an audio mute on / off button, and the like are provided.

  FIG. 12 shows a menu displayed when the panel “Camera” is clicked. In the “Operation Mode” menu, the operation mode on the image can be switched. With the pan / tilt control buttons, the pan / tilt direction can be specified by operating the buttons in the arrow direction. The button located in the middle of the pan and tilt control buttons is a home position button. The zoom button is a button for designating the zoom of the camera terminal device 200, and is configured so that the boundary between the optical zoom and the digital zoom can be seen. If you continue to hold down the “W” and “T” zoom buttons, the zoom operation will continue. The focus button is used to set the focus to “far” or “near”. In addition, various buttons such as a ONE PUSH AUTO FOCUS button and an exclusive control right acquisition button are provided.

  FIG. 13 shows menus displayed when “Preset position”, “Trigger”, “Other”, and “information” are clicked. “Preset position” is a thumbnail display of images in a plurality of predetermined shooting directions. By clicking and selecting one of the thumbnail images, the camera terminal device 200 is directed in the direction of the selected thumbnail image. be able to.

(6) Specification of shooting direction on first panorama image Next, display of the first panorama image will be described with reference to FIGS. FIG. 14 is a schematic diagram showing a state where the first panorama image is displayed by clicking “Preset position” on the operation panel on the display screen of the monitor 310. A live image is displayed on the right side of the operation panel. As described above, by dragging and moving the “Preset position” column, the sub-panel on which the first panoramic image is displayed can be arranged outside the operation panel as shown in FIG. Further, the sub-panel on which the first panoramic image is displayed can be moved within the display screen.

  FIG. 16 is a schematic diagram showing in detail the sub-panel on which the first panoramic image is displayed. In the first panoramic image, the direction of the image displayed in the live video on the right side (the direction of the optical axis of the imaging optical system of the camera terminal device 200) is indicated by +. Also, as shown in FIG. 16, the first panorama image can be switched to the second panorama image by clicking the panorama switching button at the upper right of the first panorama image.

  FIG. 17 shows an operation of changing the direction of the live video using the first panoramic image. The pan / tilt direction of the camera terminal device 200 can be changed using the pan / tilt control buttons described above, but can also be performed by designating an arbitrary point in the first panoramic image. In the state of FIG. 17, the optical axis of the imaging optical system of the camera terminal device 200 is directed in the direction of the + mark, and a live image in that direction is displayed. In this state, when the portion marked with ● shown in the first panoramic image of FIG. 17 is clicked, an operation signal US corresponding to the user operation is sent to the control unit 310, and based on this, the imaging direction control of the camera terminal device 200 is performed. By controlling the unit 204, the optical axis of the imaging optical system of the camera terminal device 200 is directed in the direction of the mark ●. Accordingly, the live image displayed on the right side of the operation panel is switched to the image in the direction of the mark ●.

  Thereby, the user can designate the pan / tilt direction of the camera terminal device 200 by designating an arbitrary point in the first panoramic image. Since the first panoramic image includes all videos in the pan / tilt direction that the camera terminal device 200 can shoot, the user designates the direction of the camera terminal device 200 on the first panoramic image. Thus, a live image in a desired direction can be displayed. In addition, instead of the + mark indicating the clicked position, the direction of the optical axis may be controlled by specifying an area with a frame surrounding a predetermined range.

  When the direction of the camera terminal device 200 is designated on the first panorama image, there are a mode in which only the live video is switched without changing the display state of the first panorama image, and a mode in which the first panorama image is rotated. is there. In the mode in which the first panoramic image is rotated, as shown in FIG. 18, the position (+ mark) where the live video is currently displayed is positioned on the straight line L upward from the center of the first panoramic image. Is set to the angular position of the first panoramic image. Next, when you click the ● mark on the first panorama image, the live image displayed on the right side of the operation panel switches to the image in the direction of the ● mark. At the same time, the position of the ● mark in the first panorama image is a straight line. The first panoramic image is rotated so as to be positioned on L. Therefore, the user can immediately recognize that the direction of the live video is on the straight line L of the first panoramic image by viewing the first panoramic image.

  Similarly, in the second panorama image, the user can designate the pan direction of the camera terminal device 200 by designating an arbitrary point in the second panorama image, and based on this, the live video can be designated. The direction can be switched.

  FIG. 19 shows an example in which a panoramic image is displayed on a cylindrical curved surface as a reference image instead of the panoramic image shown in FIG. In this way, a rectangular panoramic image displayed in the range of + 180 ° to −180 ° may be displayed on the cylindrical surface. Thereby, the visibility by a user can be improved and the image | video of a desired direction can be displayed. Also in the case of FIG. 19, the user can designate the pan direction of the camera terminal device 200 by designating the “display designated point” on the curved surface, and the direction of the live video can be switched based on this. it can. Note that the above-described processing including designation of the imaging direction is performed by the display processing unit 305 based on the GUI display control signal CTg.

(7) Effect of Displaying First Panorama Image Next, the effect of displaying the first panorama image will be described with reference to FIGS. FIG. 20 shows a state in which “safe” and “table” are arranged on the floor, and the camera terminal device 200 is installed directly above the intermediate position (point O) between “safe” and “table”. It is assumed that a suspicious person approaching the “safe” can be monitored by the live video of the camera terminal device 200.

  FIG. 21 shows a state in which the first panoramic image is displayed on the monitor 310 together with the live video (FIG. 21A) and a state in which the second panoramic image is displayed on the monitor 310 together with the live video in the state of FIG. FIG. 21 (B)) is schematically shown. As described above, when an arbitrary position on the first panorama image or the second panorama image is clicked, the direction of the optical axis of the camera terminal device 200 can be changed to the clicked position, and a live video can be captured. The direction can be changed.

  As shown in FIG. 21B, in the case of the second panoramic image, since the image is distorted immediately below the camera terminal device 200, that is, between the “safe” and the “table”, the clicked position. Even if the direction of the optical axis is changed, the direction of the optical axis does not become the actual intermediate position (point O shown in FIG. 20) between the “safe” and the “table”. In other words, if you want to see the area around the center of the “safe” and “table”, if you operate on the second panorama image (rectangular panorama image), the image will be out of the location that the user originally wanted to see, and intuitively. Control is not possible. For this reason, as shown in FIG. 21B, a situation occurs in which the videos of “safe” and “table” protrude from the lower side of the live video.

  On the other hand, as shown in FIG. 21A, since the image is displayed in an overhead view, it is possible to intuitively click an intermediate position (point O shown in FIG. 20) between “safe” and “table”. it can. Further, in the case of the first panoramic image, since the distortion of the image near the point O is relatively small, the direction of the optical axis is accurately directed to the position of the point O. Therefore, even in the live video, the “safe” and “table” videos will not protrude from the display screen.

  Further, FIG. 22 shows a state in which the suspicious person is moving in the vicinity of the “safe” in the live video and the first panorama image and the second panorama image in the case of FIGS. 20 and 21. . When a suspicious person is moving in the vicinity of the “safe” in the live video, in order to track the suspicious person, it is necessary to change the direction of the optical axis of the camera terminal device 200 according to the movement of the suspicious person. is there.

  In such a case, in the case of the second panoramic image, the image is distorted at an intermediate position between the “safe” and the “table”, so that the click position is intentionally operated so as to change the moving direction immediately below the image. Need to do.

  On the other hand, in the case of the first panorama image, there is little distortion of the image between the “safe” and the “table”, so you can intuitively click according to the movement of the suspicious person while watching the live video. By going, the direction of the optical axis can be moved in accordance with the movement of the suspicious person.

Note that when the user wants to see the vicinity of the periphery rather than directly under the camera terminal device 200, the second panoramic image has less distortion and can be controlled more intuitively than the first panoramic image. In this embodiment, since it is possible to switch between the first panorama image and the second panorama image according to the situation, it is possible to improve the ease of use of the user according to the scene. Further, if there is a space in the display screen, both the first panorama image and the second panorama image are displayed, so that the switching operation becomes unnecessary and the usability can be improved.
In the above embodiment, the rectangular panoramic image is controlled so that the imaging direction becomes the center position. However, the present invention is not limited to this, and even if the imaging direction is changed, the panoramic image is not rotated and fixed. Good.

  The preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, but the present disclosure is not limited to such examples. It is obvious that a person having ordinary knowledge in the technical field to which the present disclosure belongs can come up with various changes or modifications within the scope of the technical idea described in the claims. Of course, it is understood that these also belong to the technical scope of the present disclosure.

DESCRIPTION OF SYMBOLS 100 Imaging system 200 Camera terminal device 300 Center server 301 Network interface part 305 Display processing part 310 Control part 500 Network 700 Imaging device 701 Storage part 704 Image processing part 705 Display processing part 710 Control part

Claims (7)

An acquisition unit for acquiring a camera image captured by the camera;
A display control unit for controlling to display a panoramic image display area for displaying a panoramic image, and a partial image display area for displaying a partial area corresponding to a position designated on the panoramic image as a partial image; ,
An input unit for inputting position information on the panoramic image;
A generating unit that generates at least one of the panoramic image and the partial image based on the camera image;
The panoramic image is a circular panoramic image,
The control device, wherein the display control unit adjusts a position of a panorama image displayed in the panorama image display area so that a position designated on the panorama image is located on a predetermined axis.   The control device according to claim 1, wherein the circular panoramic image is generated based on an image captured a plurality of times by the camera.   The control device according to claim 1, wherein the display control unit switches a panoramic image displayed in the panoramic image display area from a circular panoramic image to a rectangular panoramic image in response to a panoramic image switching operation.   The control device according to claim 1, wherein the display control unit displays a rectangular panoramic image together with a circular panoramic image in the panoramic image display area.   The control device according to claim 1, wherein the display control unit is capable of moving the panoramic image display area on a screen. Obtaining a camera image captured by the camera;
Controlling to display a panoramic image display area for displaying a panoramic image, and a partial image display area for displaying a partial area corresponding to a position designated on the panoramic image as a partial image;
Inputting position information on the panoramic image;
Generating at least one of the panoramic image and the partial image based on the camera image;
The panoramic image is a circular panoramic image,
In the control for displaying the panoramic image display area and a partial image display area for displaying a partial area corresponding to a position designated on the panoramic image as a partial image,
A control method of adjusting a position of a panorama image displayed in the panorama image display region so that a position designated on the panorama image is located on a predetermined axis. A camera system having a camera that captures an image and a control device that controls the camera,
The control device includes:
An acquisition unit for acquiring a camera image captured by the camera;
A display control unit for controlling to display a panoramic image display area for displaying a panoramic image, and a partial image display area for displaying a partial area corresponding to a position designated on the panoramic image as a partial image; ,
An input unit for inputting position information on the panoramic image;
A generating unit that generates at least one of the panoramic image and the partial image based on the camera image;
The panoramic image is a circular panoramic image,
The said display control part is a camera system which displays on the said panorama image display area so that the position designated on the said panorama image may be located on a predetermined axis line.

Images