JP2015228549A - Information processing device for controlling camera, method therefor, program and storage medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable easy zoom control of a camera to both telescopic direction and wide-angle direction with an operation on an image only, and to enable easy camera control with a finger operation on a touch panel.SOLUTION: An information processing device which executes the control of a camera device through a network includes: display control means for controlling processing to move the image, displayed on a display, or change a magnitude according to a user operation; and generation means for generating a control command to control the imaging position of the camera device, in association with the processing of the image movement or the magnitude change processed by the display control means.

Description

  The present invention relates to a technique for controlling panning, tilting, and zooming of an imaging apparatus in response to an enlargement / reduction operation of an image distributed from an imaging apparatus such as a network camera and displayed on an information processing apparatus.

  A recent network camera system can capture an image of a remote camera via a network, display the image on a monitor, and perform camera control such as panning, tilting, and zooming.

  In the network camera, the zoom control of the remote camera is controlled by the user performing a designation operation to enclose an arbitrary part in a rectangle from the remote image displayed on the monitor. There is a function to display a zoomed-in image on a monitor.

  Further, for example, Patent Document 1 discloses a portable terminal having a function of displaying an image of the entire shootable range of a camera and enlarging / reducing it by moving a rectangle superimposed on the image.

JP 2005-200999 A

  However, when the zoom of the camera is controlled by surrounding an arbitrary part on the image with a rectangle, only the camera control in the telephoto direction can be performed, and the zoom control of the camera in the wide angle direction cannot be performed.

  Further, in the above-described prior art, since a rectangle can be drawn only on the image, it is impossible to control the imaging direction of the camera outside the range of the image. Further, when a rectangle is drawn with a finger in a portable information processing apparatus equipped with a touch panel, it is necessary to draw a small rectangle when the camera is controlled at the telephoto end with a camera having a high zoom magnification. However, in the touch panel operation with a finger, since a portable display is small, it is difficult to specify a fine position as compared with a mouse, and there is a problem that a rectangle is hidden with a finger during drawing.

  In addition, as in Patent Document 1, when a camera is operated using a small display for a portable terminal, the camera is operated using an image having higher visibility than an image of the entire range that can be captured by the camera from the viewpoint of operability. There is a request to do.

  SUMMARY OF THE INVENTION An object of the present invention is to provide an area zoom function that enables camera control in both the telephoto direction and the wide-angle direction only by operating an image and that can be easily controlled by finger operation on a touch panel. .

  According to the present invention, there is provided an information processing apparatus that performs control of a camera device via a network, the display unit displaying an image acquired from the camera device, and the display unit displaying the image in response to a user operation. Display control means for controlling the process of moving or changing the size of the image to be performed, and shooting by the camera device corresponding to the process of moving or changing the size of the image processed by the display control means And generating means for generating a control command for controlling the position.

  According to the present invention, it is possible to easily execute zoom control of both the telephoto and wide-angle cameras only by an operation on an image. Furthermore, the camera can be easily controlled even by an operation with a finger on the touch panel.

Configuration diagram of network camera system Display example on the display of the client terminal Display example on the display of the client terminal Display example on the display of the client terminal Display example on the display of the client terminal The flowchart which shows the process of a client terminal device when a user's operation is detected with respect to a display image The flowchart which shows the process of the client terminal device 2 which produces | generates the control command of the camera corresponding to the reduction / enlargement or movement process of the image by a user The figure for demonstrating the imaging | photography angle of view of the camera apparatus before and behind control The figure for demonstrating the imaging | photography angle of view of the camera apparatus before and behind control Configuration diagram of network camera system Flow chart showing processing of client terminal device The figure showing the positional relationship between the shooting position and movable range of the camera device and the panoramic image Example of hardware configuration of client terminal device

EMBODIMENT OF THE INVENTION Below, the form for inventing is demonstrated in detail based on attached drawing.
In the first embodiment, an image from the camera device displayed on the display unit with a predetermined size is moved relative to the display area of the display unit, for example, enlarged, A system for reducing the size will be described. In the first embodiment, pan / tilt / zoom of the camera device is controlled in response to image movement / enlargement / reduction relative to a predetermined image display area on the display.

  FIG. 1 is a configuration diagram of a system according to the first embodiment.

  The camera device 1 is connected to a client terminal device 2 that is an information processing device for browsing and operating an image of the camera device 1 via a network 3.

  The camera device 1 is a camera device having a general configuration including a lens 101, an imaging unit 102, an image compression unit 103, a movable camera platform 104, a camera control unit 105, and a communication unit 106. The lens 101 has a zoom lens, and the movable camera platform 104 realizes pan control and tilt control of the imaging unit 102. The camera control unit 105 performs panning, tilting, and zooming by controlling the lens 101 and the movable camera platform 102. The image capturing unit 102 captures an image formed on the image sensor via a lens and performs AD conversion, and the image compression unit 103 compresses the digitally converted image to a data size that is easy to distribute. The communication unit 106 distributes the compressed image to the client terminal device 2 and transfers the operation command sent from the client terminal device 2 to the camera control unit 105. Further, the communication unit 106 transmits the status information of each unit received from the camera control unit 105 to the client terminal device 2. For example, information regarding the current zoom magnification value, the maximum zoom magnification value, the current pan / tilt value, and the maximum pan / tilt control range is transmitted to the client terminal device 2.

  The client terminal device 2 includes a display unit 201, a display control unit 203, an image decompression unit 204, an operation command generation / interpretation unit 205, and a communication unit 206. The display unit 201 displays a captured image received from the camera device 1 and a GUI (graphical user interface) for user operation.

  The display control unit 203 performs display image movement and size change processing by electronic image processing of an image in response to detection of a user operation, and generates a GUI for the user operation to display the display unit 201. To display. The image decompression unit 204 decompresses the compressed image from the camera device 1 and passes it to the display control unit 203. The operation command generation / interpretation unit 205 passes the state of the camera device 1 received from the communication unit 206 to the display control unit 203 and generates an operation command for communication from the operation on the camera device 1 received from the camera position calculation unit 208. .

  The operation input unit 207 receives a user's touch operation on the display of the display unit 201. Specifically, it accepts a user's touch operation for movement or enlargement / reduction by electronic image processing on a captured image displayed on the display.

  The camera position calculation unit 208 calculates the position of the camera device 1 corresponding to the image movement or enlargement / reduction by the user. Specifically, the zoom magnification when the currently displayed image is taken, and the moving position on the screen of the taken image by the user, the shooting position (shooting direction) of the camera device 1 are calculated. Further, the camera position calculation unit 208 is controlled by the camera device 1 based on the zoom magnification when the currently displayed image is taken and the ratio of enlargement or reduction of the image by the user with respect to the size of the display area. The power zoom magnification is calculated. That is, the shooting direction of the camera device 1 corresponding to the amount of movement of the image is calculated so that the subject is shot at the same position as the subject in the image moved by the image processing. Further, the zoom magnification of the camera device 1 is calculated so as to capture an image in which the size of the subject in the electronically enlarged or reduced image is the same.

  As another embodiment, an image showing the entire movable range of the camera device, such as a panoramic image, is created, and a desired part of the image is cut out from the created image and output. You may apply to the camera apparatus which performs tilt and zoom control. In that case, the camera position calculation unit 208 matches the position of the clipped image so that the image cut out from the image of the entire shooting range matches the position of the moved image and the size of the subject in the enlarged or reduced image. Calculate position and size.

  FIGS. 2 to 5 show examples of images and GUIs displayed on the display 400 of the display unit 201. The display 400 displays an image display area 410 for displaying images and buttons 420 for various camera operations. In FIG. 2, the size of the image 411 including the subject 412 transmitted from the camera device 1 is displayed in the same size and at the same position as the size of the image display area 410. The display 400 supports multi-touch operation using a touch panel. For example, the position and size of the image 411 including the subject 412 are changed by changing the positions of the tap positions 413 and 414 of the two fingers with respect to the image display area 410.

  FIG. 3 is a display example of the display 400 when a pinch-out operation for moving the tap positions 413 and 414 away from each other is performed. In FIG. 3, the distance between the tap positions 413 and 414 is increased compared to the distance between the tap positions 413 and 414 in FIG. 2. In FIG. 3, as the distance increases, the image 411 is enlarged by image processing, and as a result, the subject 412 protrudes from the image display area 410. Actually, since the image cannot be displayed outside the image display area 401, only the portion of the subject 412 within the range of the image display area 410 is displayed. In FIG. 3, since the midpoints of the tap positions 413 and 414 are the same as those in FIG. 2, the centers of the image display area 410 and the image 411 remain the same.

  Further, when the user is operating as shown in FIG. 3, indicators 430 and 431 which are not displayed in FIG. 2 are displayed. The indicator 430 corresponds to the image 411, and the indicator 431 corresponds to the image display area 410. That is, the indicators 430 and 431 indicate the relative position and size relationship between the image 411 enlarged by image processing and the image display area 410. That is, the position where the indicators 430 and 431 are displayed changes corresponding to the change in the positional relationship between the image display area 410 and the image 411.

  FIG. 4 is a display example when a pinch-in operation for moving the tap positions 413 and 414 in a direction approaching each other is performed. Compared to the distance between the tap positions 413 and 414 in FIG. 2, the distance between the tap positions 413 and 414 is reduced. In FIG. 4, image processing is performed on the image 411 in FIG. 2 by image processing in response to the user operation, and a reduced image of the image 411 is displayed. Since the midpoints of the tap positions 413 and 414 are the same as in the case of FIG. 2, the centers of the image display area 410 and the image 411 remain the same.

  Also in the case of FIG. 4, since the user is operating, indicators 430 and 431 are displayed. Compared to the case of FIG. 3, since the image 411 is displayed in the image display area 410, the size of the indicators 430 and 431 displayed is opposite to the state of FIG.

  FIG. 5 shows an example when a swipe operation for moving the tap position 414 to the right from the state of FIG. 4 is performed. In the example of FIG. 5, after releasing the finger corresponding to the tap position 413 from the state of FIG. 4, the tap position 414 is moved to the right by the user's swipe operation. In accordance with the movement of the tap position, the image 411 is moved to the right by the same amount as the movement of the tap position 414 by image processing.

  In the example shown in FIG. 5, both the position and the size of the displayed image 411 are changed from the state of FIG. 3, and the indicator 430 corresponds to the position and the size of the image 411 and the indicator 430 is on the right side. It sticks out.

  Note that not only the operation on the image 411 but also the operation on the margin part may be accepted within the image display area 410 as the operation by the user.

  Further, in the present embodiment, the moving operation of the image 411 is limited to be performed only within the range displayed in the image display area 410. For example, the image 411 is limited so that 10% or more is always displayed. This is because if the image 411 is shifted to a position where it completely protrudes from the image display area 410, the image 411 is not displayed, and the user cannot grasp where the camera apparatus 1 should be controlled. Note that the display control unit 201 temporarily moves the image to a position that completely protrudes from the image display area 411 in accordance with a user operation, and then displays the image so that a part of the image is displayed in the image display area 411. It may be corrected.

  FIG. 6 is a flowchart illustrating processing executed by the display control unit 203 of the client terminal device 2 when a user operation is detected on the display image.

  When a user input is detected by the operation input unit 207, the display magnification of the image 411 is calculated in step S101 based on the ratio between the size of the image 411 to be enlarged or reduced and the size of the image display area 410. . If the image corresponding to the image that has been enlarged or reduced cannot be captured even by controlling the zoom magnification of the camera device 1, the process proceeds to step S102.

  In S <b> 102, the display control unit 203 restricts the enlargement or reduction processing of the image 411 based on the zoom magnification of the camera device 1, that is, restricts the display magnification. Due to this limitation, an image corresponding to the enlarged or reduced image obtained by the display magnification of the image 411 can also be obtained by the zoom control of the camera device 1. For example, it is assumed that the current camera apparatus 1 has a zoom magnification of 2 and an image currently displayed by the user's operation has been doubled. In this case, the camera device 1 needs to control the zoom magnification to 2 × 2 = 4 times, but when the maximum zoom magnification of the camera device 1 is 3, the actual display magnification is changed to 1.5 times. Is done. At this time, the zoom magnification of the camera device 1 is controlled to 2 × 1.5 = 3 times. As another example, it is assumed that the zoom magnification of the current camera device 1 is three times, and the currently displayed image is reduced to ¼ by a user operation. In this case, 3 × 1/4 = 0.75, which exceeds the zoom control range on the wide-angle side, so that the actual image display magnification is limited to 1/3. At this time, the zoom magnification of the camera device 1 is controlled to 1 time. It is assumed that information regarding the zoom magnification of the camera device 1 has been acquired from the camera device 1 in advance.

  When the zoom magnification of the camera device 1 can be controlled in correspondence with the display magnification of the enlarged or reduced image 411, that is, an image corresponding to the image 411 enlarged or reduced by controlling the zoom magnification of the camera device 1 is obtained. If yes, go to step S103.

  In step S <b> 103, the display area of the image portion on which the image 411 is displayed on the display 410 is calculated by the movement process or the reduction process of the image 411 corresponding to the detection of the user operation.

  When the display area of the image 411 is less than a predetermined ratio (for example, 10%) with respect to the entire area of the image 411, the process proceeds to step S104, and the display control unit 203 displays the image 411 while maintaining the display magnification of the image 411. The display position is changed to a display position that is equal to or greater than a predetermined ratio. In step S104, information regarding the pan / tilt drive limit of the camera apparatus 1 may be acquired from the camera apparatus 1, and the display position of the image may be limited based on this information.

  FIG. 7 is a flowchart illustrating processing executed by the command generation unit / interpretation unit 205 of the client terminal device 2 that generates a camera control command corresponding to image reduction / enlargement or movement processing by the user.

  In step S201, when an operation by the user such as a long tap on the display 410 is detected by the operation input unit 207, the display control unit 203 executes enlargement / reduction or movement processing of the image 411 by image processing. At the same time, the camera position calculation unit 208 calculates the position of the camera device 1 represented by pan, tilt, and zoom values based on the current display position and display magnification of the image 411. In step S202, the operation command generation / interpretation unit 205 generates a camera control command based on the calculated camera position, and transmits the camera control command to the camera device 1 via the communication unit 206. When a captured image is received from the camera device 1 after being controlled by the camera control command, the display is switched from the display of the image that has been enlarged / reduced or moved by the image processing to the display of the received captured image.

  FIG. 8 and FIG. 9 are examples showing the camera position change when the camera apparatus 1 is controlled based on step S202.

  8 and 9, the hemisphere indicates a hemisphere centered on the viewpoint of the camera apparatus 1, the circumferential direction 505 of the bottom surface of the hemisphere is the pan direction, and the circumferential direction 504 of the semicircle orthogonal to the bottom surface of the hemisphere is the tilt direction. It corresponds to. An area 502 and an area 503 indicate the shooting angle of view of the camera device 1. An area 502 in FIG. 8 shows a shooting angle of view of the camera apparatus 1 before the control by the control command of S202, and an area 503 shows an image shot after the shooting direction and zoom magnification of the camera apparatus 1 are controlled. The corresponding area is shown. Here, when the control by the control command in step S202 is executed in the state of FIG. 8, the pan / tilt / zoom control of the camera apparatus 1 is performed so that the area 503 becomes the shooting field angle of the camera apparatus 1 as shown in FIG. Is executed. The relationship between the relative positions and sizes of the areas 502 and 503 in FIGS. 8 and 9 is the same.

  In the second embodiment described below, a panorama acquired in advance from the camera device 1 in a blank area other than an image displayed after image processing for enlarging / reducing or moving an image in response to a user operation. This is an embodiment in which a part of an image is cut out and displayed. The point that the image is moved and enlarged / reduced in the image display area within the movable range of the camera device 1 after the camera control is the same as in the above embodiment.

  FIG. 10 is a system configuration diagram of the second embodiment. Although the system configuration is almost the same as the system configuration of FIG. 1, a partial panorama image generation unit 209 is added. The partial panoramic image generation unit 209 cuts out a part of the panoramic image corresponding to the entire movable range of the camera device 1 that is captured by controlling the movable pan head 104 with the lens 101 set to the wide-angle end in advance. Correct distortion and output. The position and size of the image cutout are determined according to the position and size of the image 411 that has been reduced, enlarged, or moved with respect to the image display area 410.

  FIG. 11 is a flowchart illustrating processing executed by the display control unit 203 of the client terminal device 2 according to the second embodiment. Steps S101 to S104 are the same as those in the flowchart of the first embodiment shown in FIG. 6. In FIG. 11, steps S105 to S106 are added. Note that the processing of FIG. 11 may be applied to the first embodiment.

  In S105, the camera position calculation unit S208 is controlled as shown in FIGS. 8 and 9 based on the ratio (display magnification) of the position and size of the current image 411 subjected to image processing with respect to the image display area 410. The position of the camera device 1 to be calculated is calculated.

  Then, it is determined whether or not the camera position calculated in S105 is within the pan / tilt movable range. If the camera position is out of the range, the camera position calculation unit S208 determines the camera position and the display position of the image 411 that are within the range in S106. calculate. In S107, the display control unit 203 changes the display position of the image 411 based on the calculated position.

  FIG. 12 is a diagram illustrating the positional relationship between the shooting position and the movable range of the camera device 1 and the panoramic image.

  The panoramic image corresponds to an image obtained by projecting the entire scene onto a spherical surface of a hemisphere, and the movable range of the camera device 1 corresponds to 506, which is a part of the movable range. The partial panorama image generated by the partial panorama image generation unit 209 corresponds to a portion of the region 503 that protrudes from the region 502. In addition, the camera position calculated in S105 is a camera position corresponding to the center of the region 503, and when it is outside the movable range 506, S106 and S107 are executed.

  As mentioned above, although preferable embodiment of this invention was described, this invention is not limited to these embodiment, A various deformation | transformation and change are possible within the range of the summary.

  In the second embodiment, the movable pan head 104 of the camera apparatus 1 has a mechanical mechanism limitation, and panning and tilting are performed within a range corresponding to the mechanical mechanism. Further, the movable range of pan and tilt can be further limited within the drive range of the mechanical function for the purpose of privacy protection.

  FIG. 13 is a diagram illustrating an example of a hardware configuration of the client terminal device according to the first and second embodiments. The client terminal device 2 includes a CPU 215, a primary storage device 225, a secondary storage device 235, a touch panel 255, a display 265, and a network I / F 290.

  The touch panel 255 corresponds to the operation input unit 207 and is disposed on the front surface of the display 265, and detects an instruction input from the user.

  The display 265 corresponds to the display included in the display unit 201 and displays a captured image from the camera device 1. A network I / F 290 is an I / F for connecting to the network 10 and corresponds to the communication unit 206. Data exchanged with the network camera 100 is transmitted / received via the network I / F 290. The secondary storage device 235 is configured by a non-volatile storage medium, and stores a program for executing the processing of the client terminal device 2 shown in FIGS. The temporary storage device 225 has a temporary storage memory such as a RAM for expanding a program read from the secondary storage device.

The CPU 215 corresponds to the display control unit 203, the image expansion unit 204, the operation command generation / interpretation unit 205, and the camera position calculation unit 208 shown in FIGS. The compression / decompression unit 204 may be realized as hardware by a circuit different from the CPU 215.
(Other examples)
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

201 Display Unit 203 Display Control Unit 205 Operation Command Generation / Interpretation Unit 208 Camera Position Calculation Unit

Claims (7)

An information processing device that controls a camera device via a network,
Display means for displaying an image acquired from the camera device;
Display control means for controlling processing of moving or changing the size of an image to be displayed by the display means in response to a user operation;
And generating means for generating a control command for controlling the photographing position of the camera device in response to the processing of moving or changing the size of the image processed by the display control means. Information processing device.   The information processing apparatus according to claim 1, wherein the display control unit is capable of moving or changing a size of the image within a range in which a predetermined ratio of the image is displayed on a display unit. .   The information processing apparatus according to claim 1, wherein the display control unit is capable of moving and changing a size of the image within a controllable range of the camera device.   The display control means displays a partial panorama image obtained by cutting out a part of a panoramic image generated by photographing the entire movable range of the camera device in advance on the part of the image not displayed by the display means. The information processing apparatus according to any one of claims 1 to 3. A processing method of an information processing apparatus for executing control of a camera device via a network,
A display control step for controlling a process of moving or changing a size of an image to be displayed in a display step of displaying an image acquired from the camera device in accordance with a user operation;
A generation step of generating a control command for controlling the photographing position of the camera device in response to the movement or size change processing of the image processed in the display control step. Processing method of information processing apparatus.   A program for causing an information processing apparatus to execute the processing method of claim 5.   A non-volatile storage medium storing the program of claim 6.

Images