JP2012054841A - Image display device and method for controlling the same - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technique for enabling photography of the entire area or a desired partial area of an image.SOLUTION: An image taken by an imaging apparatus while a display screen displayed by an image display device is zoomed out to the maximum is obtained. Then, an area most similar to the image within the display screen is identified and the display screen is reduced into the size of the identified area and displayed at the position of the area.

Description

  The present invention relates to a display control technique.

  Conventionally, a technique for controlling zoom and focus based on information from a distance measuring device included in an imaging apparatus has been common (Patent Document 1). In addition, a technique for realizing zoom and focus by synthesizing videos accumulated by an imaging apparatus has been common (Patent Document 2). In addition, a technique for adjusting zoom and focus by calculating a zoom ratio according to a distance input in advance to the imaging apparatus has been common (Patent Document 3).

JP 11-155093 A Japanese Unexamined Patent Publication No. 07-181954 JP 2006-019852 JP

  Here, consider a case where an entire video displayed by a video output device such as a television or a projector is shot using an imaging device such as a digital camera. If the imaging device is too close to the image position, zooming out will not be possible even if the zoom / pan / tilt / brightness adjustment mechanism of the imaging device is operated to fit the entire image in the viewfinder of the imaging device. A state may be reached. This is the same when the imaging device is too far from the video position. That is, it is not possible to make adjustments such that the entire image is shot only with the adjustment function of the imaging device. The same applies to the case where a desired area in the video is shot as well as the entire video.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide a technique for enabling photographing of the entire video or a desired partial region.

  In order to achieve the object of the present invention, for example, an image display apparatus of the present invention comprises the following arrangement. That is, an image display device that is communicable with an imaging device, and means for acquiring a captured image captured by the imaging device in a state where the display screen displayed by the image display device cannot be zoomed out any more, and the display screen Specifying means for specifying an area most similar to the captured image, and display control means for reducing the size of the display screen to the size of the area specified by the specifying means and displaying the display screen at the position of the area. Features.

  With the configuration of the present invention, the entire video or a desired partial area can be captured.

The block diagram which shows the structural example of a system. 1 is a block diagram illustrating a configuration example of an imaging apparatus 100. FIG. FIG. 2 is a block diagram illustrating a configuration example of a video output device 200. FIG. 3 is a block diagram illustrating a configuration example of a router 300. 5 is a flowchart showing operations of the imaging apparatus 100 and the video output apparatus 200. 5 is a flowchart showing operations of the imaging apparatus 100 and the video output apparatus 200. 5 is a flowchart showing operations of the imaging apparatus 100 and the video output apparatus 200. The figure which shows the structural example of a message.

  Preferred embodiments of the present invention will be described below with reference to the accompanying drawings. The embodiment described below shows an example when the present invention is specifically implemented, and is one of the specific examples of the configurations described in the claims.

[First Embodiment]
As shown in FIG. 1, the system according to this embodiment includes a video output device 200 that functions as an image display device such as a projector, a liquid crystal screen, and a CRT, an imaging device 100 that has an imaging function such as a digital camera, and a router 300. , Is composed of. The video output device 200 and the imaging device 100 are connected to a router 300 and are configured to communicate with each other via the router 300. The network between the video output device 200 and the imaging device 100 is not limited to a specific network, and any network may be used as long as it can communicate with each other.

  In the present embodiment, the video output device 200 will be described as a projector that projects a video on a wall surface or a screen. In the present embodiment, the imaging device 100 and the video output device 200 will be described as separate devices. However, the video output device 200 and the imaging device 100 may be integrated into one device.

  Further, each function may be distributed to a plurality of video output devices 200 as necessary. When a plurality of video output devices 200 are used, the video output devices 200 are connected to a network such as a local area network (LAN) so that they can communicate with each other. Communication using messages between the devices is performed by Hypertext Transfer Protocol (http). Or it is performed by Hypertext Transfer Protocol over Secure Socket Layer (https). Or it is performed by Simple Object Access Protocol (SOAP).

  Next, a configuration example of the imaging apparatus 100 will be described using the block diagram of FIG. The CPU 101 performs operation control of each unit constituting the imaging apparatus 100 using a computer program and data stored in the ROM unit 110.

  Light received from the outside world enters an image receiving unit 102 configured by an image sensor such as a CCD or CMOS via an image input unit 108 formed of an optical system such as a lens. The image receiving unit 102 converts the incident light into an electric signal to form an image. This image is a captured image. A lens zoom control unit 107 controls the optical system of the image input unit 108 to perform zoom in and zoom out.

  The display unit 103 includes a touch panel type liquid crystal screen, and an image formed by the image receiving unit 102 is displayed on the liquid crystal screen. Further, the user can specify an arbitrary position on the display screen of the liquid crystal screen with his / her finger or the like. Since the display unit 103 notifies the instructed position to the CPU 101, for example, when the user instructs two points on the display screen, the display unit 103 notifies the CPU 101 of the position of these two points. A rectangle with a point as the opposite vertex can be set.

  The network I / F (interface) unit 104 is for connecting the imaging device 100 to the router 300 described above, and the imaging device 100 outputs video that is connected to the router 300 via the network I / F unit 104. Data communication with the apparatus 200 is performed.

  The message receiving unit 105 receives a message (described later) transmitted from the video output device 200 via the network I / F unit 104. The message transmission unit 106 transmits the message generated by the imaging device 100 to the video output device 200 via the network I / F unit 104.

  The operation unit 112 includes a button group for inputting various operation instructions to the CPU 101. FIG. 1 shows a mode input button 113 and a photographing shutter button 114 as an example. The mode input button 113 includes a button for inputting an instruction for zooming in and zooming out, and a button for instructing various imaging settings. The imaging shutter button 114 can notify the CPU 101 of an imaging instruction by pressing this button. Of course, each button constituting the operation unit 112 is for the user to operate.

  The storage unit 109 includes a work area used for the CPU 101 to execute various processes, an area for temporarily storing data of an image formed by the image receiving unit 102, and the like. The ROM unit 110 stores computer programs and data used by the CPU 101 to execute various processes, setting data of the imaging apparatus 100, and the like. The operations of the frame coordinate calculation unit 111 and the image recognition unit 115 will be described later.

  Next, a configuration example of the video output device 200 will be described with reference to the block diagram of FIG. The CPU 201 performs operation control of each unit constituting the video output device 200 using a computer program and data stored in the ROM unit 207.

  The network I / F unit 202 is for connecting the video output device 200 to the router 300, and the video output device 200 is connected to the router 300 via the network I / F unit 202. Data communication with.

  The message receiving unit 203 receives a message (described later) transmitted from the imaging apparatus 100 via the network I / F unit 202. The message transmission unit 204 transmits the message generated by the video output device 200 to the imaging device 100 via the network I / F unit 202.

  The image output unit 206 includes an optical system such as a lens and a light source. The image output unit 206 projects an image held in a memory (not shown) by the video output device 200 onto a wall surface or a screen. The display screen is displayed (projected) on the surface. The lens zoom control unit 205 controls the lens in the optical system, and changes the position and size of the projected display screen by performing zoom-out, zoom-in, and position change.

  The ROM unit 207 stores computer programs and data used by the CPU 201 to execute various processes. The image comparison unit 208 performs a process of comparing the images and obtaining an area most similar to the other image in one image.

  The image reduction rate calculation unit 209 obtains the reduction rate of the other image with respect to one image. The image enlargement rate calculation unit 211 obtains the enlargement rate of the other image with respect to one image. The image reduction movement distance calculation unit 210 and the image enlargement movement distance calculation unit 212 are both distances from the upper and lower sides of one image to the upper and lower sides of the other image, and from the left and right sides of one image to the left and right sides of the other image. Find the distance. The luminance control unit 215 adjusts the luminance in the image.

  Next, a configuration example of the router 300 will be described with reference to the block diagram of FIG. The network I / F unit 301 is connected to the imaging device 100 and the video output device 200. The CPU 302 controls the operation of the router 300. The RAM 303 has a work area used for the CPU 302 to execute various processes, an area for holding data for temporary storage in communication between apparatuses, and the like.

  Next, operations (control methods) of the imaging apparatus 100 and the video output apparatus 200 will be described with reference to the flowchart of FIG. An image formed by the image receiving unit 102 is displayed on the display unit 103 of the imaging apparatus 100. The user operates the mode input button 113 while viewing this image, and performs operation instructions such as zooming in, zooming out, and brightness adjustment. Enter. Here, when a zoom-out instruction is input using the mode input button 113, the instruction is notified to the CPU 101. Therefore, in step S601, the CPU 101 first controls the lens zoom control unit 107 to zoom into the image input unit 108. Realize out.

  Here, when the zoom-out is continued, a state (zoom-out limit state) in which no further zoom-out is possible is reached. When the lens zoom control unit 107 detects the zoom-out limit state, the lens zoom control unit 107 notifies the CPU 101 to that effect. Of course, the zoom-out can be continued as long as the zoom-out limit state is not detected.

  However, when the CPU 101 detects the zoom-out limit state, the process proceeds to step S603 via step S602, and when the zoom-out limit state is not detected, the process returns to step S601 via step S602. In step S <b> 603, the CPU 101 stores the image formed by the image receiving unit 102 at this time in the storage unit 109 as a captured image to be transmitted to the video output apparatus 200.

  In step S <b> 604, the CPU 101 generates a message 690 to be transmitted to the video output device 200. A configuration example of the message 690 is shown in FIG. In the area indicated as item 1, the address (IP address here) of the transmission source (imaging device 100) is stored. In the area indicated as item 2, the address (IP address here) of the transmission destination (video output device 200) is stored. In the area indicated as item 3, a value indicating the zoom-out limit state is stored. In the area indicated as item 4, the image data (captured image data) stored in the storage unit 109 in step S603 is stored.

  However, when the CPU 101 generates the message 690 having such a configuration, in step S605, the message transmission unit 106 transmits the message 690 to the video output device 200 via the network I / F unit 104.

  In step S611, the message receiving unit 203 of the video output device 200 receives (acquires) the message 690 via the network I / F unit 202. In step S612, the image comparison unit 208 reads the captured image stored in the area indicated as item 4 in the message 690, and reads the captured image on the display screen held by the video output device 200. An area most similar to the image (similar area) is specified. As a technique for specifying a similar region, a known image recognition technique, a matching processing technique, or the like may be used.

  Note that when the region most similar to the read captured image is specified in the display screen held by the video output device 200, if the similarity is less than the threshold value, the process goes to step S613. The process proceeds to S617. On the other hand, if the similarity is greater than or equal to the threshold, the process proceeds to step S614 via step S613.

  In step S614, the image reduction ratio calculation unit 209 obtains, as the reduction ratio, a fractional value with the display screen size as the denominator and the size of the similar area as the numerator. In step S615, the image reduction movement distance calculation unit 210 obtains a reduction movement distance between the display screen and the similar area. The reduction movement distance is the distance x from the upper side of the display screen to the upper side of the similar region, the distance y from the lower side of the display screen to the lower side of the similar region, the distance z from the right side of the display screen to the right side of the similar region, Information on the distance w from the left side to the left side of the similar region is included.

  In step S616, the CPU 201 controls the lens zoom control unit 107 to output the image so that the display screen is reduced in size to the size of the similar region in the similar region in the projected display screen. The optical system of the unit 206 is controlled. That is, a reduced display screen is obtained by reducing the projected display screen at the reduction ratio. At this time, the upper side of the reduced display screen is a distance x below the upper side of the display screen, the lower side of the reduced display screen is a distance y above the lower side of the display screen, and the right side of the reduced display screen is a distance z left from the right side of the display screen. The left side of the display screen is positioned at a distance w to the right from the left side of the display screen.

  If the display screen is not projected onto a surface such as a screen but directly displayed on a screen such as a monitor, such optical system control is not performed and the following display control is performed. Will do. That is, an area most similar to the captured image in the display screen displayed on the monitor is specified as a similar area, and the display screen is reduced to the size of the similar area and displayed at the position of the similar area.

  Then, the process returns to step S611. On the other hand, in step S617, the CPU 201 generates a message 691. A configuration example of the message 691 is shown in FIG. In the area indicated as item 1, the address (IP address here) of the transmission source (video output device 200) is stored. In the area indicated as item 2, the address (IP address here) of the transmission destination (imaging device 100) is stored. The area indicated as item 3 stores an error ID indicating that a similar area was not found as a result of image comparison.

  However, when the CPU 201 generates the message 691 having such a configuration, the message transmission unit 204 transmits the message 691 to the imaging apparatus 100 via the network I / F unit 202. Similarly, when the zoom on the video output device side reaches the limit point, a message is issued to the video input device side. Then, the process returns to step S611.

  On the other hand, in step S <b> 606, the message receiving unit 105 on the imaging device 100 side receives this message 691 transmitted from the video output device 200 via the network I / F unit 104.

[Second Embodiment]
In the present embodiment, the same system as in the first embodiment is used. The operations of the imaging apparatus 100 and the video output apparatus 200 will be described using the flowchart of FIG. An image formed by the image receiving unit 102 is displayed on the display unit 103 of the imaging apparatus 100. The user operates the mode input button 113 while viewing this image, and performs operation instructions such as zooming in, zooming out, and brightness adjustment. Enter. Here, when a zoom-in instruction is input using the mode input button 113, this instruction is notified to the CPU 101. In step S 701, the CPU 101 first controls the lens zoom control unit 107 to zoom in the image input unit 108. make it happen.

  Here, when zooming in continues, it reaches a state (zoom-in limit state) where zoom-in cannot be performed any more. When the lens zoom control unit 107 detects the zoom-in limit state, the lens zoom control unit 107 notifies the CPU 101 to that effect. Of course, zooming in can be continued as long as the zoom-in limit state is not detected.

  However, if the CPU 101 detects the zoom-in limit state, the process proceeds to step S703 via step S702. If the zoom-in limit state is not detected, the process returns to step S701 via step S702. In step S703, the CPU 101 stores the image formed by the image receiving unit 102 at this time in the storage unit 109 as a captured image.

  In step S704, the CPU 101 displays the captured image stored in the storage unit 109 in step S703 on the display unit 103, and accepts an area designation operation on the display screen of the display unit 103. As described above, the user can designate an area by instructing on the display screen of the display unit 103. Accordingly, in step S705, when the CPU 101 detects the designation operation, the CPU 101 generates area information (for example, coordinate positions of the upper left corner and the lower right corner of the designated area) indicating the area designated by the designation operation.

  In step S <b> 706, the CPU 101 generates a message 790 to be transmitted to the video output device 200. A configuration example of the message 790 is shown in FIG. In the area indicated as item 1, the address (IP address here) of the transmission source (imaging device 100) is stored. In the area indicated as item 2, the address (IP address here) of the transmission destination (video output device 200) is stored. In the area indicated as item 3, a value indicating the zoom-in limit state is stored. In the area indicated as item 4, the captured image data stored in the storage unit 109 in step S703 is stored. The area indicated as item 5 stores the area information (frame coordinates) generated in step S705.

  However, when the CPU 101 generates the message 790 having such a configuration, in step S707, the message transmission unit 106 transmits the message 790 to the video output device 200 via the network I / F unit 104.

  In step S <b> 709, the message receiving unit 203 of the video output apparatus 200 receives this message 790 via the network I / F unit 202. In step S <b> 710, the image comparison unit 208 specifies an area indicated by the area information stored in the item 5 in the captured image stored in the area indicated as the item 4 in the message 790. Then, the image comparison unit 208 sets the image in the specified region as a partial image, and specifies a region (similar region) most similar to the partial image in the display screen held by the video output device 200. As a technique for specifying a similar region, a known image recognition technique, matching technique, or the like may be used.

  Note that, when the region most similar to the partial image is specified in the display screen held by the video output device 200, if the similarity is less than the threshold, the process goes to step S715 via step S711. move on. On the other hand, if the similarity is greater than or equal to the threshold, the process proceeds to step S712 via step S711.

  In step S <b> 712, the image enlargement ratio calculation unit 211 obtains, as an enlargement ratio, a fractional value with the display screen size as the numerator and the size of the similar area as the denominator. In step S713, the image enlargement movement distance calculation unit 212 obtains the enlargement movement distance between the display screen and the similar area. The enlargement movement distance is the distance x from the upper side of the display screen to the upper side of the similar area, the distance y from the lower side of the display screen to the lower side of the similar area, the distance z from the right side of the display screen to the right side of the similar area, Information on the distance w from the left side to the left side of the similar region is included.

  In step S714, the CPU 201 controls the lens zoom control unit 107 to enlarge the image in the similar region to the size of the display screen and display the enlarged image as the display screen. To control. That is, the above-described enlargement ratio is a region in which the lower side is the upper side, the lower side is the upper side, the lower side is the upper side, the right side is the distance z, the left side is the right side, and the left side is the distance w. The enlarged image is displayed.

  If the display screen is not projected onto a surface such as a screen but directly displayed on a screen such as a monitor, such optical system control is not performed and the following display control is performed. Will do. That is, an area most similar to the partial image in the display screen displayed by the monitor is specified as a similar area, an image (similar image) in the similar area is enlarged to the size of the display screen, and the enlarged similar image is used as the display screen. indicate.

  Then, the process returns to step S709. On the other hand, in step S715, the CPU 201 generates a message 791. The message 791 is the same as the message 691 described above. When the CPU 201 generates the message 791, the message transmission unit 204 transmits this message 791 to the imaging apparatus 100 via the network I / F unit 202.

  On the other hand, in step S <b> 708, the message receiving unit 105 on the imaging device 100 side receives the message 791 transmitted from the video output device 200 via the network I / F unit 104.

[Third Embodiment]
In the present embodiment, the same system as in the first embodiment is used. The operations of the imaging apparatus 100 and the video output apparatus 200 will be described using the flowchart of FIG. An image formed by the image receiving unit 102 is displayed on the display unit 103 of the imaging apparatus 100. The user operates the mode input button 113 while viewing this image, and performs operation instructions such as zooming in, zooming out, and brightness adjustment. Enter. Here, when a brightness adjustment instruction is input using the mode input button 113, the instruction is notified to the CPU 101. In step S801, the CPU 101 first uses the image formed by the image receiving unit 102 at this time as a captured image. The image is sent to the image recognition unit 115.

  In step S802, the image recognizing unit 115 checks whether or not whiteout / blackout has occurred in the captured image with reference to each pixel value constituting the captured image. Since this check is a well-known technique, a description thereof will be omitted. As a result of this check, if whiteout / blackout has occurred, the process proceeds to step S804, and if not, this process ends.

  In step S <b> 804, the CPU 101 generates a message 890 to be transmitted to the video output device 200. A configuration example of the message 890 is shown in FIG. In the area indicated as item 1, the address (IP address here) of the transmission source (imaging device 100) is stored. In the area indicated as item 2, the address (IP address here) of the transmission destination (video output device 200) is stored. In the area indicated as item 3, overexposure (the pixel value in the captured image is above the upper limit of the dynamic arrangement), blackout (the pixel value in the captured image is below the lower limit of the dynamic arrangement) A flag indicating which one of these has occurred is stored.

  However, when the CPU 101 generates the message 890 having such a configuration, in step S804, the message transmission unit 106 transmits this message 890 to the video output device 200 via the network I / F unit 104.

  In step S <b> 806, the message receiving unit 203 of the video output apparatus 200 receives this message 890 via the network I / F unit 202. In step S807, the brightness control unit 215 refers to the flag stored in the area indicated as the item 3 in the message 890, and specifies whether whiteout or blackout occurs. Then, the brightness control unit 215 reduces the brightness value of the display screen by a specified amount when whiteout occurs. The specified amount is not particularly limited. For example, when the range of luminance values is 0 to 255, the specified amount is decreased by one. On the other hand, the luminance control unit 215 increases the luminance value of the display screen by a specified amount when blackout occurs. Although the specified amount is not particularly limited, for example, when the range that the luminance value can take is 0 to 255, it is increased by 1.

  In step S808, the CPU 201 generates a message 891. The message 891 has the same configuration as the message 691 described above, but in the area indicated as the item 3, when an error occurs in the increase / decrease of the luminance value, information indicating that is stored as an error ID. However, the generation and transmission of this message 891 is not essential. When the CPU 201 generates the message 891, the message transmission unit 204 transmits the message 891 to the imaging device 100 via the network I / F unit 202.

  In step S <b> 805, the message receiving unit 105 of the imaging apparatus 100 receives the message 891 transmitted from the video output apparatus 200 via the network I / F unit 104.

  The above embodiments may be combined as appropriate. That is, when the first embodiment and the second embodiment are combined, an imaging apparatus having the zoom-out technique described in the first embodiment and the zoom-in technique described in the second embodiment is obtained. realizable. Further, when combined with the brightness adjustment technique according to the third embodiment, the zoom-out technique described in the first embodiment, the zoom-in technique described in the second embodiment, and the third technique. An imaging apparatus having the brightness adjustment technique described in the embodiment can be realized.

<Effect>
According to each embodiment mentioned above, there can exist the following various effects. First, it becomes possible to take a picture by assisting the zoom of the image pickup apparatus with the zoom of the video output apparatus when the zoom photographing at the short distance / far distance cannot be performed any more. In addition, it is possible to shoot under appropriate exposure conditions by linking the luminance of the video output device and the shooting conditions of the imaging device 100.

  As described above, a case that can be solved in each of the above-described embodiments includes, for example, a case where the whole cannot be accommodated in a scene where a whiteboard on which conference materials are projected is to be photographed. In addition, a scene where a character image (an anime character, a dinosaur, etc.) projected by a projector and a person is photographed at an amusement park attraction is also applicable. In addition, a scene where an infant photo of the bride and groom projected by the projector and the bride and groom are photographed as one subject in a photo studio or a wedding reception is also applicable. In this case, there are various cases in which subjects to be photographed together have various sizes and height differences, and the size cannot be changed to an appropriate size.

  In addition, there is a case in which a digital camera with a projector function has a restriction that the standing position cannot be adjusted for camera shooting in a scene where the projector is projected and the camera shooting is desired. In addition, there is a case where the projector brightness and the brightness of the subject do not match, and the two cannot be captured clearly.

  As described above, since it is possible to control each peripheral device from one camera, a staff member who is not bright in the operation of the device can shoot by one person. In addition, a mobile photography studio business that can be installed on the road and traveled on the road can be realized. As a result, it is possible to establish a business model of a new entry venture that can provide users with high-quality photos at a photo studio level that does not require an initial cost compared to a large photo studio that requires equipment costs.

[Other Embodiments]
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

Claims (5)

An image display device capable of communicating with an imaging device,
Means for acquiring a captured image captured by the imaging device in a state where the display screen displayed by the image display device cannot be zoomed out any more;
A specifying means for specifying a region most similar to the captured image in the display screen;
An image display device comprising: display control means for reducing the display screen to the size of the area specified by the specifying means and displaying the display screen at the position of the area.   The image display according to claim 1, wherein the specifying unit specifies a region most similar to the captured image in the display screen by performing a matching process with the captured image in the display screen. apparatus. Furthermore,
Means for acquiring an image in a set area as a partial image in a captured image captured by the imaging device in a state where the display screen displayed by the image display device cannot be zoomed in any more;
Means for specifying a region most similar to the partial image in the display screen as a similar region;
3. The image display device according to claim 1, further comprising: a display control unit that enlarges an image in the similar region to a size of the display screen and displays the enlarged similar image as the display screen. .   The image display device according to claim 1, wherein the image display device is integrated with the imaging device. An image display device control method capable of communicating with an imaging device,
The acquisition means of the image display device acquires a captured image captured by the imaging device in a state where the display screen displayed by the image display device cannot be zoomed out any more;
A specifying step in which the specifying unit of the image display device specifies a region most similar to the captured image in the display screen;
The display control means of the image display device comprises display control means for reducing the display screen to the size of the area specified in the specifying step and displaying the display screen at the position of the area. Control method.

Images