JP6120541B2 - Display control apparatus and control method thereof - Google Patents

Description

  The present invention relates to a display control apparatus for enlarging and reducing an image to be superimposed on a video in accordance with a zoom amount of the video and displaying the superimposed image.

  There is an apparatus that can draw hand-drawn characters and illustrations on a screen displaying a camera video or a playback video using a pointing device such as a mouse or a touch panel. In addition, it is known as AR (Augmented Reality / Augmented Reality) technology that information based on position information by GPS and identification information based on a subject detection function are superimposed and displayed on a video (Patent Document 1).

  As described above, when another image is superimposed and displayed on the camera video or the playback video, the image display is enlarged or reduced in accordance with the zoom variation of the video, thereby providing a sense of unity between the video and the image ( Patent Document 2).

Japanese Patent Laid-Open No. 10-42282 JP 2010-088076 A

  However, if the superimposed image is enlarged in conjunction with the zoom operation, depending on the display content, the majority of the screen will be filled with the superimposed image, and the image below will be almost invisible, impairing the visibility of the image. was there.

  Therefore, the present invention is a display that makes it easy to visually recognize the lower video even when the superimposed item occupies a large part of the screen for displaying the video by enlarging the superimposed item in conjunction with the enlargement of the video. It is an object to provide a control device and a control method thereof.

  In order to solve the above-described problem, the display control device of the present invention displays an image generated by the imaging process of the imaging unit at the first zoom magnification on the touch panel, and further, at a coordinate position touched on the touch panel. Display control means for displaying on the touch panel an image generated by sequentially drawing on the video image with a first transmittance, and a change for changing the zoom magnification used in the imaging unit to the second zoom magnification And setting means for setting a second transmittance based on a difference between the first zoom magnification and the second zoom magnification, and the display control means is used in the imaging unit by the changing means. When the zoom magnification to be changed is changed to the second zoom magnification, the image is enlarged based on the second zoom magnification and the second transmission set by the setting means In, wherein the superimposed on the generated image by the second imaging processing of the imaging unit at the zoom magnification displayed on the touch panel.

  According to the present invention, when an image on which an image is superimposed is enlarged and displayed, the transmittance of the image is changed based on the zoom magnification at the time when the image is superimposed, so that the image is superimposed on the image. When the image is visually confirmed and then further magnified, the visibility of the image below the image can be improved.

The block diagram of the video camera which implement | achieves embodiment which concerns on this invention Contents of frame memory in memory 107 and screen configuration displayed on display unit 108 Flow chart showing the operation of this embodiment (A) Display example in which a hand-drawn illustration 402 is superimposed on a camera video 401. (B) A display example in which a hand-drawn illustration 402 enlarged and increased in transmittance is superimposed on an enlarged camera image 401. (C) A display example in which a hand-drawn illustration 402 that is further enlarged and further increased in transmittance is superimposed on a further enlarged camera image 401.

  Preferred embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a block diagram illustrating a configuration example of a digital video camera 100 according to the present embodiment.

In FIG. 1, a lens group 101 is an optical system that guides subject light to the imaging unit 102, and is an imaging lens group including a zoom lens, a focus lens, and a diaphragm.
The imaging unit 102 is an imaging unit including a CCD or CMOS element that converts an optical image that has passed through a lens group into an electrical signal, and an A / D converter that converts an analog signal into a digital signal.

  The lens driving unit 103 is a lens driving unit for controlling zoom, focus, and aperture.

The video processing unit 104 performs resize processing and color conversion processing such as predetermined pixel interpolation and reduction on the data from the imaging unit 102 or the data from the memory control unit 106. The video processing unit 104 performs predetermined arithmetic processing using the captured video data, and the system control unit 109 performs zoom control, exposure control, and measurement via the lens driving unit 103 based on the obtained calculation result. Perform distance control. Thereby, zoom processing, AF (autofocus) processing, and AE (automatic exposure) processing are performed. The video processing unit 104 further performs predetermined arithmetic processing using the captured video data, and also performs AWB (auto white balance) processing based on the obtained arithmetic result.
The CODEC 105 converts the video data from the video processing unit 104 into MPEG2 or H.264. The encoded video data such as H.264 is encoded, or the encoded video data passed from the memory control unit 106 is decoded.

  Output data from the imaging unit 102 is written into the memory 107 via the video processing unit 104, the CODEC 105, and the memory control unit 106.

  The memory 107 stores video data encoded by the CODEC 105 and video data to be displayed on the display unit 108. The memory 107 has a recording capacity sufficient to store moving image and audio data for a predetermined time. The memory 107 also serves as a bitmap memory for OSD display and a video memory for image display.

  The display unit 108 includes a display such as an LCD and performs display according to display video data read from the memory 107. The video data accumulated in the memory 107 by the imaging unit 102 is sequentially transferred to the display unit 108 and displayed, thereby functioning as an electronic viewfinder (through image display). The display unit 108 also includes a function of displaying an OSD (on-screen display) such as an icon or a time code drawn on the bitmap memory in the memory 107 or a hand-drawn illustration image by the touch panel 112 so as to be superimposed on the video data.

  The nonvolatile memory 110 is an electrically erasable / recordable memory, and for example, an EEPROM or the like is used. In the nonvolatile memory 110, constants, programs, and the like for operating the system control unit 109 are recorded. Here, the program is a program for executing various flowcharts described later in the present embodiment.

  A system control unit 109 controls the entire digital video camera 100. By executing the program recorded in the non-volatile memory 110 described above, each process of the present embodiment to be described later is realized.

A RAM is used as the system memory 111, and constants and variables for operation of the system control unit 109, programs read from the nonvolatile memory 110, and the like are expanded. The system control unit also performs display control by controlling the memory 107, the display unit 108, and the like. The system memory 111 may be common with the memory 107.
The power switch 115, zoom lever 114, trigger button 113, and touch panel 112 are operation means for inputting various operation instructions to the system control unit 109.

  The power switch 115 switches between power on and power off of the entire digital video camera 100.

  The zoom lever 114 is a lever for instructing the system control unit 109 to change the focal length of the lens group 101 via the lens driving unit 103. When operated in a certain direction, the zoom lever 114 is moved to the tele side. Operate to zoom to the wide side.

  The trigger button 113 generates a trigger signal for instructing the system control unit 109 to start and end moving image shooting. The system control unit 109 performs a series of operations until the encoded video data stored in the memory 107 is written to the recording medium 119 via the imaging unit 102, the video processing unit 104, the CODEC 105, and the memory control unit 106 according to the trigger signal. Start or end the shooting process.

  The touch panel 112 is a touch panel that can detect contact with the display unit 108. The touch panel 112 and the display unit 108 can be configured integrally. For example, the touch panel 112 is configured such that the light transmittance does not hinder the display of the display unit 108 and is attached to the upper layer of the display surface of the display unit 108. Then, the input coordinates on the touch panel 112 are associated with the display coordinates on the display unit 108. Thereby, it is possible to configure the GUI as if the user can directly operate the screen displayed on the display unit 108. The system control unit 109 can detect the following operations / states on the touch panel 112. Touching the touch panel with a finger or a pen (hereinafter referred to as touchdown). The touch panel is touched with a finger or a pen (hereinafter referred to as touch-on). The touch panel is moved while being touched with a finger or a pen (hereinafter referred to as a move). The finger or pen that was touching the touch panel is released (hereinafter referred to as touch-up). A state where nothing is touched on the touch panel (hereinafter referred to as touch-off). The system control unit 109 is notified of these operations and the position coordinates where the finger or pen touches the touch panel, and the system control unit 109 determines what operation has been performed on the touch panel based on the notified information. judge. Regarding the move, the moving direction of the finger or pen moving on the touch panel can also be determined for each vertical component / horizontal component on the touch panel based on the change of the position coordinates. It is also assumed that a stroke is drawn when touch-up is performed on the touch panel through a certain move from touch-down. The touch panel may be any of various types of touch panels such as a resistive film method, a capacitance method, a surface acoustic wave method, an infrared method, an electromagnetic induction method, an image recognition method, and an optical sensor method. .

  The power control unit 116 includes a battery detection circuit, a DC-DC converter, a switch circuit that switches a block to be energized, and the like, and detects whether or not a battery is attached, the type of battery, and the remaining battery level. Further, the power control unit 116 controls the DC-DC converter based on the detection result and an instruction from the system control unit 109, and supplies a necessary voltage to each unit including the recording medium 119 for a necessary period.

  The power supply unit 117 includes a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, a NiMH battery, or a Li battery, an AC adapter, or the like.

  An interface 118 is an interface with a recording medium 119 such as a memory card or a hard disk. The recording medium 119 is a recording medium such as a memory card, and includes a semiconductor memory, a magnetic disk, or the like.

  In the recording medium 119, the encoded video data stored in the memory 107 is recorded by the I / F 118, or the encoded video data and accompanying data recorded in the recording medium 119 are read by the I / F 118 and transferred to the memory 107. To do. The recording medium 119 may be a memory card, a hard disk drive, or a disk attached to the video camera 100, or a flash memory or a hard disk drive incorporated in the video camera 100.

  Next, the operation of this embodiment will be described with reference to FIGS. In the present embodiment, an operation of displaying a hand-drawn illustration drawn on the touch panel 112 in conjunction with the camera video in the digital video camera 100 of FIG. 1 will be described.

  FIG. 2 is a diagram showing the contents of the frame memory in the memory 107 and the screen configuration displayed by the display unit 108 via the memory control unit 106. The memory 107 requires a graphic frame memory area for at least three screens and a camera through image or reproduced image frame memory.

  FIG. 2A shows a graphic frame memory in which drafts of hand-drawn illustrations drawn in advance by the touch panel 112 are recorded. This is realized by the system control unit 109 reading the coordinates at which the touch panel 112 is touched on at regular intervals, and sequentially rendering the pen tip images at corresponding coordinate positions in the frame memory (A). Also, if it is drawn only once when touched down, it can be treated like a stamp. A plurality of patterns of pen nibs and stamp images are held in the non-volatile memory 110, and the system control unit 109 reads and uses necessary ones each time. By operating a GUI such as a button (not shown) on the touch panel 112, the image shape and color of the pen tip and the stamp can be changed, and various illustrations can be drawn.

  FIG. 2B shows a graphic frame memory for preparing an image to be displayed on the display unit 108, and a partial area of the draft frame memory (A) is enlarged and copied. When this enlarged copy is performed, the transmittance at the time of superimposition with a camera video frame, which will be described later, is set.

  The frame memory (B) has a two-sheet configuration, and while the copy process is being performed in one frame memory, the memory controller 106 reads out the other frame memory. As a result, it is possible to avoid a halfway frame image being copied being read out by the memory control unit 106 and displayed on the display unit 108 as flickering. When the copy process is completed, the reading source frame memory is switched.

  In FIG. 2C, when the digital video camera 100 is in the camera mode (shooting mode), the image is captured by the imaging unit 102 and buffered in the memory 107 via the video processing unit 104 and the memory control unit 106. This is a frame memory for a through image (camera image). Alternatively, if the digital video camera 100 is in the playback mode, the encoded video data read by the recording medium 119 by the I / F 118 is decoded by the CODEC 105 and buffered in the memory 107, thereby being a frame memory for a playback image. .

  FIG. 2D shows screen content (display example) displayed by the display unit 108. The contents of the frame memories (B) and (C) are read by the memory control unit 106 and the display unit 108 performs superposition and synthesis. This image may be displayed on a display device in a device such as an LCD panel (not shown), or may be output as a video signal to a video output terminal (not shown).

  FIG. 3 is a flowchart showing an operation when a zoom operation is performed in the camera mode or the playback mode of the digital video camera 100 realizing the present embodiment. The operation of this flowchart is realized by the system control unit 109 reading a program recorded in the nonvolatile memory 110 and controlling each unit of the digital video camera 100.

  FIG. 4 is a display example showing a screen configuration displayed by the display unit 108 at that time. FIG. 4A shows a display example in which a hand-drawn illustration 402 is superimposed on a camera video 401. FIG. 4B is a display example in which a hand-drawn illustration 402 enlarged and increased in transmittance is superimposed on an enlarged camera image 401. FIG. 4C shows a display example in which a hand-drawn illustration 402 that is further enlarged and further increased in transmittance is superimposed on a further enlarged camera image 401. S301 is a step of setting a specified value for the transmittance in the copy process of S313.

  S302 is a step of setting an initial value 0 to a variable for managing a screen update request.

  S303 is a step of determining whether or not to change the draft frame update request, that is, the contents of FIG. For example, if the content of the draft frame is not a stationary hand-drawn illustration but an animated object, S303 is true at a constant cycle. If this step is true, the process proceeds to S304, and if it is false, the process proceeds to S306.

  S304 is a step of updating the drawing content of the draft frame FIG. 2A.

  S305 is a step of setting 1 to a variable for managing the screen update request.

  S306 is a step of determining whether or not the zoom lever 114 has been operated. If this step is true, the process proceeds to S307, and if it is false, the process proceeds to S312.

  S307 is a step of zooming the contents of the frame memory of FIG. In the camera mode, the video signal obtained by the imaging unit 102 can be optically zoomed by controlling the lens driving unit 103 to move the lens group 101 to tele or wide. Alternatively, the video can be digitally zoomed (electronic zoom, electronic enlargement / reduction) by controlling the video processing unit 104 to trim the video signal. On the other hand, in the playback mode, the playback digital zoom can be realized in the same manner as in the camera mode by trimming the video data decoded by the CODEC 105 by the video processing unit 104. In any method, the video output from the display unit 108 can be freely zoomed as shown in FIGS.

  S308 is a step of acquiring the zoom magnification from the control result of S307.

  S309 is a step of setting 1 to a variable for managing the screen update request.

  S310 is a step of determining whether or not the transmittance of the hand-drawn illustration currently displayed on the display unit 108 is less than a predetermined threshold value. If this step is true, a new transmittance is set in S311. If S310 is false, S311 is skipped and the transmittance is maintained. Based on the determination in step 310, it is possible to control so that a hand-drawn illustration that has been gradually transmitted and disappeared from the screen is not displayed again. This is useful when hand-drawn illustrations are used like title telops.

  S311 is a step of determining a new transmittance according to the conditions. The condition is that the transmittance is higher when the zoom magnification is low than when the zoom magnification is low.

  For example, a unique value proportional to the zoom magnification (enlargement magnification) acquired in S308 may be used as the transmittance. In other words, a transmittance table or calculation formula uniquely associated with the enlargement ratio is stored in advance in the nonvolatile memory 110, and the transmittance associated with the current enlargement magnification is determined from this table or calculation formula. Good.

  Alternatively, the area of the hand-drawn illustration area may be calculated in the graphic frame of FIG. 2B to be copied and drawn in S313, and a unique value proportional to the value may be used as the transmittance. That is, the transmittance may be determined according to the area occupied by the enlarged superimposed item in the display area where the enlarged video is displayed.

  Further, the transmittance may be increased depending on how much the handwritten illustration is drawn relative to the zoom magnification at the time of drawing the handwritten illustration. For example, the control is as follows. When the user draws a hand-drawn illustration at a transmittance of 0% with the optical zoom doubled (2x from the wide-angle end), the optical zoom is set to the reference magnification. Thereafter, when the optical zoom is magnified 6 times with the handwritten illustration superimposed, the background camera image and the reproduced image can be seen through by setting the transmittance of the handwritten illustration to 30%. Further, when the optical zoom is magnified 12 times, the transmittance of the handwritten illustration is set to 60% to make the background image easier to see. On the other hand, when the optical zoom is reduced to 1 ×, the transmittance is not changed from the reference magnification (2 × optical zoom).

  S312 is a step of determining whether or not a screen update request is set in S305 or S309. If S312 is true, the process proceeds to S313, and if it is false, the screen is not updated, and the process returns to S302 to repeat the subsequent processing.

  In step S313, a part or all of the draft frame in FIG. 2A is enlarged and copied to the drawing frame in FIG. 2B at the same magnification as the zoom magnification acquired in step S308 and based on the transmittance determined in step S311. It is a step to do.

  S314 is a step of switching between the drawing side and the display side of the two frames in FIG. As a result, the content newly copied in S 311, that is, the display content in accordance with the zoom magnification of the camera image or the reproduced image is read to the memory control unit 106 and displayed on the display unit 108. When the display is performed with the transmittance determined under the conditions in S311, the hand-drawn illustration is relatively small in size and low in transmittance as shown in FIG. Is displayed. If the zoom is close to the telephoto, the size is relatively large and the transmittance is displayed as shown in FIG. If the zoom position is near the middle, as shown in FIG. 4B, the size and transmittance of the hand-drawn illustration are in a state between (A) and (C).

  In addition, when the above-described processing is performed in a state where a camera video is recorded (that is, during moving image shooting), as described above, a hand-drawn illustration whose transmittance is changed according to zoom (enlargement) is superimposed. The camera video is recorded on the recording medium 119 as moving image data of the moving image file.

  As described above, according to the present invention, even when the hand-drawn illustration is enlarged in accordance with the zoom of the camera image or the reproduced image, the hand-drawn illustration is gradually transmitted and displayed. For this reason, it is possible to avoid that hand-drawn illustrations are displayed on the entire screen and the camera image and the reproduced image cannot be visually recognized.

In the present embodiment, an image (superimposed item) to be superimposed and displayed on the camera video or the reproduced image is a hand-drawn illustration, but the present invention is not limited to this.
For example, the position information of the imaging device is acquired by a GPS unit or the like, and for the captured image displayed on the display of the imaging device, the direction symbol indicating the east, west, south, and north based on the position information, the name of the municipality, the boundary of the municipality, map information It is conceivable to superimpose these as superimposition items. Even in such a system, when the photographed image is zoomed, the visibility of the image can be maintained by enlarging the superimposed item according to the enlargement and gradually transmitting the superimposed item.

  Alternatively, subject detection for detecting a specific subject (for example, a human face or animal face) from the photographed video is performed, and information regarding the detected subject is superimposed and displayed at a position based on the position of the subject in the photographed video. There is a device. For example, there is an apparatus that identifies a person from a face of a person detected by performing face detection, and displays a face frame and the name of the person near the face. Even in such a device, when the captured video is zoomed, the visibility of the video is maintained by gradually transmitting superimposed items (such as a face frame and a name) related to the subject detected from the captured video according to the present invention. be able to.

  Note that the control of the system control unit 109 described above may be performed by a single piece of hardware, or the entire apparatus may be controlled by a plurality of pieces of hardware sharing the processing.

  Although the present invention has been described in detail based on the preferred embodiments thereof, the present invention is not limited to these specific embodiments, and various forms without departing from the gist of the present invention are also included in the present invention. included. Furthermore, each embodiment mentioned above shows only one embodiment of this invention, and it is also possible to combine each embodiment suitably.

  In the above-described embodiment, the case where the present invention is applied to the digital video camera 100 has been described as an example. However, this is not limited to this example. Display control device having a function of enlarging a superimposition item displayed superimposed on some background video (either a still image or a moving image) such as a camera video (through image) or a playback image in conjunction with the expansion of the background video If so, the present invention is applicable. That is, the present invention can be applied to a digital camera, a personal computer, a PDA, a mobile phone terminal, a portable image viewer, a printer device including a display, a digital photo frame, a music player, a game machine, an electronic book reader, and the like.

(Other embodiments)
The present invention is also realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, etc.) of the system or apparatus reads the program code. It is a process to be executed. In this case, the program and the storage medium storing the program constitute the present invention.

Claims (7)

The image generated by the imaging process of the imaging unit at the first zoom magnification is displayed on the touch panel, and the image generated by sequentially drawing at the coordinate position touched on the touch panel is displayed with the first transmittance. Display control means for displaying on the touch panel superimposed on the video;
Changing means for changing the zoom magnification used in the imaging unit to a second zoom magnification;
Setting means for setting a second transmittance based on a difference between the first zoom magnification and the second zoom magnification;
The display control unit enlarges the image based on the second zoom magnification and sets the image by the setting unit when the zoom magnification used in the imaging unit is changed to the second zoom magnification by the changing device. The display control apparatus, wherein the second control unit displays the image on the touch panel by superimposing the image generated by the image capturing process of the image capturing unit at the second zoom magnification.   2. The setting means sets a value larger than the first transmittance as the second transmittance when the second magnification is larger than the first magnification. The display control apparatus according to 1.   The setting means sets a value equal to the first transmittance as the second transmittance when the second magnification is smaller than the first magnification. The display control apparatus according to 2.   An image generated by the imaging process of the imaging unit at the second zoom magnification is superimposed on an image enlarged based on the second zoom magnification at the second transmittance, and is recorded on a recording medium. 4. The display control apparatus according to claim 1, further comprising recording means for controlling. The image generated by the imaging process of the imaging unit at the first zoom magnification is displayed on the touch panel, and the image generated by sequentially drawing at the coordinate position touched on the touch panel is displayed with the first transmittance. Displaying on the touch panel superimposed on the video;
Changing the zoom magnification used in the imaging unit to a second zoom magnification;
Setting a second transmittance based on a difference between the first zoom magnification and the second zoom magnification;
When the zoom magnification used in the imaging unit is changed to the second zoom magnification, the image is enlarged based on the second zoom magnification, the second transmittance, and the second zoom magnification. A control method for a display control apparatus, comprising: a step of superimposing on an image generated by an imaging process of an imaging unit and displaying the image on the touch panel.   The program for functioning a computer as each means of the display control apparatus as described in any one of Claims 1 thru | or 4.   A computer-readable storage medium storing a program for causing a computer to function as each unit of the display control device according to any one of claims 1 to 4.

Images