JP2010278511A - Electronic equipment - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide electronic equipment for improving user-friendliness by suppressing deformation, or the like of a subject by proper image processing for display. <P>SOLUTION: An image processing section 109 performs trapezoidal correction on an image G2 and turns the image 180 degrees (vertical/horizontal reversal), thus displaying an upright image G3 on a monitor 112. Therefore, both the user US and a user on the other end of the line can easily understand content of a document DCM, can intuitively understand it since the image G3 also moves to the right when the user US moves the document DCM to the right, and further can instantly read characters, or the like written on the document DCM by the user US. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an electronic apparatus using a solid-state image sensor such as a CCD (Charge Coupled Devices) type image sensor or a CMOS (Complementary Metal-Oxide Semiconductor) type image sensor.

  In recent years, video cameras and digital cameras equipped with an imaging device using a solid-state imaging device such as a CCD type image sensor or a CMOS type image sensor have become widespread, and a solid having a high number of pixels so that a higher quality image can be obtained. A device equipped with an image pickup device using an image pickup element has been supplied to the market. In addition, with the spread of the Internet, so-called web cameras or the like that are attached to a personal computer to acquire a sender's image and transmit it to the other party are already on the market.

  By the way, there is an increasing need for digitizing documents for information security or remote communication using handwritten documents. There is a scanner as an apparatus conventionally used for digitizing a document. However, the scanner captures an image plane that faces the imaging optical system, and a large-sized scanner device equipped with a line sensor is often used particularly when a document containing characters is digitized on a desktop. However, there is a problem that the device itself is relatively bulky. On the other hand, a digital camera may be used instead of a scanner, but in this case, it is necessary to shoot with the image plane and the digital camera facing each other at a certain distance. There is a problem that it has to be placed on the floor or the like to take an image and is not easy to use.

  On the other hand, in Patent Document 1, an imaging target is imaged by an imaging device that is movably attached to a table on which an imaging target is set, based on an image signal output from the imaging device. A presentation system that displays moving images and still images on a television monitor is disclosed. It is also conceivable to capture a document using such a presentation system and transmit it to the other party.

JP 09-214823 A

  By the way, in the presentation system of Patent Document 1, since the imaging device and the television monitor are separated, the imaging device can easily face the imaging target, but on the personal computer like the scanner. There is a problem that it is difficult to incorporate. In particular, since notebook computers and the like tend to place importance on portability, there is a demand for imaging documents and the like using existing devices as much as possible.

  Here, some personal computers on the market have a built-in web camera for the purpose of performing a video conference or the like. This web camera is generally integrated into a display monitor unit or a personal computer main body, and is often used to transmit an image of a user who operates the personal computer to a destination. When considering using such a web camera to capture a document, it is desirable to capture the image while the document is placed on a desk in consideration of the user's effort. In such a case, the imaging axis of the web camera attached to the display monitor unit or the PC main body is always tilted with respect to the normal line of the document, and there is a problem that the image is distorted trapezoidally. However, there is a problem that it is difficult to see the other party of the transmission destination.

  The present invention has been made in view of such problems, and an object of the present invention is to provide an electronic device that displays images while suppressing distortion and the like of a subject by appropriate image processing and improving user convenience. To do.

The electronic device according to claim 1 is:
The body,
A camera unit having an imaging element and an imaging optical system that forms an optical image on a light receiving surface of the imaging element;
Image processing means for performing image processing on an image signal output from the camera unit;
A monitor unit that is arranged at least at an angle with respect to the main body, has the camera unit attached, and displays an image based on an image signal image-processed by the image processing unit,
The image processing means is configured to output an image signal output from the camera unit according to a setting of a first imaging mode for imaging a planar subject having a normal line inclined with respect to the optical axis of the imaging optical system. The keystone correction is performed, and image processing is performed so that the image displayed on the monitor unit is vertically and horizontally reversed.

  According to the present invention, the image processing means outputs from the camera unit according to the setting of the first imaging mode for imaging a planar subject having a normal inclined with respect to the optical axis of the imaging optical system. The trapezoidal correction is performed on the image signal and the image processing is performed so that the image displayed on the monitor unit is flipped up and down, left and right, so that the camera unit attached to the monitor unit can be made compact in order to reduce the size. Even if an image is taken, an appropriate image can be displayed, and convenience for the user can be achieved. For the up / down / left / right reversal processing, for example, the technique disclosed in Japanese Patent Laid-Open No. 2003-102000 can be used.

  According to a second aspect of the present invention, there is provided the electronic device according to the first aspect, wherein the electronic device has a storage unit, and the image processing unit has a predetermined value while the first imaging mode is set. Trapezoidal correction parameters are acquired from the image of the planar object when the operation is performed, stored in the storage means, and image processing for keystone correction is performed based on the stored trapezoidal correction parameters. Therefore, for example, when the user moves the subject, the keystone correction parameter can be updated by the user's intention, and an appropriate image can be obtained.

  According to a third aspect of the present invention, there is provided the electronic device according to the first aspect, wherein the electronic device includes a storage unit, and the image processing unit performs a predetermined time while the first imaging mode is set. Since trapezoidal correction parameters are acquired from the image of the planar object at intervals, stored in the storage means, and image processing for keystone correction is performed based on the stored trapezoidal correction parameters. The trapezoidal correction parameters can be automatically updated without operating the user himself / herself, and an appropriate image can be obtained.

  According to a fourth aspect of the invention, there is provided the electronic device according to the first aspect of the invention, wherein the electronic device has a storage unit, and the image processing unit includes the main body while the first imaging mode is set. When the tilt angle of the camera unit with respect to the optical axis of the imaging optical system changes, a keystone correction parameter is obtained from the image of the planar object, stored in the storage means, and based on the stored keystone correction parameter Then, image processing for keystone correction is performed. When the tilt angle between the main body and the optical axis of the imaging optical system of the camera unit changes, the image of the subject imaged by the camera unit also changes, so that an appropriate image can be obtained by updating the keystone correction parameter accordingly. Can be obtained.

  According to a fifth aspect of the present invention, in the electronic device according to any one of the first to fourth aspects, the image processing unit is responsive to a setting of a second imaging mode for imaging a user operating the electronic device as a subject. Then, image processing is performed on the image signal output from the camera unit so that the image displayed on the monitor unit is reversed horizontally. When the user himself is imaged with a web camera or the like and the image is displayed on the monitor, the right hand of the user is displayed on the left side of the monitor. However, since humans have been accustomed to seeing mirror images since they were small, if the user's right hand is projected on the right side of the monitor, the user will feel as if he is looking at the mirror and feel comfortable. Therefore, when the second imaging mode is set, the image processing unit performs image processing so that the image displayed on the monitor unit is reversed left and right without upside down, thereby improving user convenience. Can be increased. For the left-right reversal processing, for example, the technique disclosed in Japanese Patent Laid-Open No. 11-4370 can be used.

  According to a sixth aspect of the present invention, in the electronic device according to the fifth aspect, the direction of the optical axis of the imaging optical system of the camera unit is detected, and if the detected direction is lower than the horizontal, the first imaging mode. And a mode setting means for setting the second imaging mode if the direction detected by the sensor is above horizontal. Thereby, even if the user does not select the mode himself, the imaging mode is automatically selected, so that the convenience for the user is enhanced. However, the imaging mode may be selected by a user operation.

  According to the present invention, it is possible to provide an electronic device that displays images while suppressing distortion and the like of a subject by appropriate image processing, thereby improving user convenience.

It is a perspective view of notebook type personal computer PC carrying imaging device CA concerning this embodiment. It is a block diagram of notebook type personal computer PC which mounts imaging device CA concerning this embodiment. It is the figure seen from the side in the state which has arrange | positioned document DCM in the vicinity of notebook type personal computer PC. (A) is a figure which shows the primary acquisition image before correction | amendment, (b) is a figure which shows the image which performed the trapezoid correction | amendment with respect to the primary acquisition image, (c) is up-down left-right inversion FIG. 4 is a diagram illustrating an image obtained by cutting out only a document. It is the figure seen from the side in the state which has arrange | positioned document DCM in the vicinity of the notebook type personal computer PC concerning a modification. It is a figure which compares and shows the coordinate of the image before trapezoid correction, and the image after trapezoid correction.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a perspective view of a notebook personal computer (electronic device) PC equipped with an imaging device CA according to the present embodiment. FIG. 2 is a block diagram of a notebook personal computer PC equipped with an imaging device (camera unit) CA according to the present embodiment.

  As shown in FIG. 1, the notebook PC PC has a main body PCB and a monitor unit PCD connected by a hinge PCH and can be folded, and the imaging device CA is fixed to the upper center of the monitor 112 in the monitor unit PCD. That is, the monitor 112 is on the front side of the imaging device CA. The image pickup apparatus CA is incorporated in the notebook personal computer PC, and the monitor 112 is used as a display means, so that the image captured by the monitor 112 can be confirmed and image processing can be performed at high speed using an image processing circuit on the personal computer side. Can be sent, pasted on other materials, extraction of characters and pictures, and reuse. In addition, since the display unit PCD whose tilt angle with respect to the main body PCB can be adjusted by the hinge PCH can be used as a support unit of the imaging apparatus CA, there is also an advantage that a field angle at the time of imaging the document DCM placed on the desk can be secured.

  In FIG. 2, the imaging apparatus CA includes an imaging optical system 101, a solid-state imaging device 102, and an A / D conversion unit 103. On the other hand, the notebook personal computer PC includes a control unit 104, an optical system driving unit 105, a timing generation unit 106, an image sensor driving unit 107, an image memory 108, an image processing unit 109, an image compression unit 110, The image recording unit 111, the monitor 112, the keyboard 113, and the sensor 114 that detects the relative inclination angle of the monitor unit PCD and the main body PCB are included. This is omitted here. The control unit 104 and the image processing unit 109 constitute an image processing unit.

  The imaging optical system 101 has a function of forming a subject image on the imaging surface (light receiving surface) of the solid-state imaging device 102. The solid-state image sensor 102 is an image sensor such as a CCD or CMOS, and photoelectrically converts incident light for each of R, G, and B and outputs an analog signal thereof. The A / D conversion unit 103 converts an analog signal into digital image data.

  The control unit 104 has a function of controlling the imaging device CA. The control unit 104 includes a CPU (Central Processing Unit), a RAM (Random Access Memory), and a ROM (Read Only Memory), and various programs read out from the ROM and expanded in the RAM, and various in cooperation with the CPU. Execute the process.

  The optical system drive unit 105 controls the drive of the imaging optical system 101 during focusing, exposure, and the like under the control of the control unit 104. The timing generator 106 outputs an analog signal output timing signal under the control of the controller 104. The image sensor driving unit 107 causes the solid-state image sensor 102 to output an analog signal in synchronization with the timing signal output from the timing generator 106.

  The image memory 108 stores image data so as to be readable and writable. The image processing unit 109 performs various image processes on the image data. The image compression unit 110 compresses captured image data by a compression method such as JPEG (Joint Photographic Experts Group). The image recording unit 111 records image data on a recording medium such as a memory card set in a hard disk or a slot (not shown).

  The monitor 112 is a color liquid crystal panel or the like, and can display image data that has undergone image processing after shooting, a through image before shooting, and the like. The keyboard 113 includes an imaging mode setting command and various operation keys for inputting information, and outputs information input by the user to the control unit 104.

  The sensor 114 is built in the hinge PCH (FIG. 1), detects the tilt angle of the monitor unit PCD with respect to the main body PCB, and transmits it to the control device 104.

  Here, operations in the imaging apparatus CA and the notebook personal computer PC will be described. The user can use the keyboard 113 to capture the first imaging mode in which the document DCM (a planar subject extending approximately in the horizontal direction) can be taken and the self (subject extending in the vertical direction). One of two imaging modes can be selected. It is preferable to set the focal length shorter in the first imaging mode than in the second imaging mode.

  However, the selection of either the first imaging mode or the second imaging mode is not limited to a user operation, and may be performed automatically. Here, when imaging the user US itself in a TV conference or the like, as shown in FIG. 3, the position of the face of the user US as the subject is the imaging apparatus CA in a state where the notebook personal computer PC is placed on the desk. Located above the horizontal line HL (that is, in the vertical direction). On the other hand, as shown in FIG. 3, the document DCM is positioned below the horizontal line HL of the imaging apparatus CA (that is, in the vertical direction) while being placed on a desk in the vicinity of the main body PCB of the notebook personal computer PC. .

  Therefore, in the present embodiment, the sensor 114 calculates the tilt angle θ of the monitor unit PCD with respect to the main body PCB, and automatically sets the imaging mode from the tilt angle θ. More specifically, the control unit 104 assumes that the main body PCB is placed on a horizontal plane together with the document DCM, and when the tilt angle θ is positive, the control unit 104 sets the imaging axis of the imaging apparatus CA (of the imaging optical system 101). The second imaging mode is automatically set assuming that the optical axis) is directed upward from the horizontal line HL. On the other hand, when the tilt angle θ is negative, the imaging axis line of the imaging device CA is below the horizontal line HL. As a result, the first imaging mode is automatically set. Obviously, the normal of the document DCM is tilted with respect to the imaging axis.

  First, a case where the second imaging mode is set and the user US himself / herself is imaged as a subject will be described. In the first imaging mode, subject monitoring (through image display) and image shooting execution are performed as necessary. In monitoring, an image of a subject obtained through the imaging optical system 101 is formed on the light receiving surface of the solid-state imaging device 102. As a photoelectric conversion output corresponding to an optical image formed at fixed intervals, the solid-state imaging device 102 disposed behind the imaging optical axis of the imaging optical system 101 is scanned and driven by the timing generation unit 106 and the imaging device driving unit 107. The analog signal is output for one screen.

  The analog signal is appropriately gain-adjusted for each primary color component of RGB, and then converted into digital data by the A / D conversion unit 103. The digital data is reversed left and right by image processing by the image processing unit 109 and subjected to color process processing including pixel interpolation processing and γ correction processing to obtain a luminance signal Y and color difference signals Cb, Cr (images of digital values). Data) is generated and stored in the image memory 108, the signal is periodically read out, the video signal is generated, output to the monitor 112, and transmitted to the other party connected via the Internet. The In the example of FIG. 1, the monitor 112 displays the image MG <b> 1 transmitted from the other party in a full screen, and the image MG <b> 2 of the user US itself is only flipped horizontally without being flipped up and down. It is displayed small. At this time, if the user US raises his left hand, the image MG2 in a mirror image relationship is raised on the same side, so that the user US looks at the mirror and feels uncomfortable. The image MG2 may be displayed on the full screen.

  The monitor 112 functions as an electronic viewfinder in monitoring, and displays captured images in real time. In this state, focusing, exposure, and the like of the imaging optical system 101 are set by the drive of the optical system driving unit 105 as needed by the image plane AF.

  In such a monitoring state, the user can acquire still image data by operating a predetermined key on the keyboard 113 at a timing when still image shooting is desired. In response to such a key operation, one frame of image data stored in the image memory 108 is read out and compressed by the image compression unit 110. The compressed image data is recorded on a recording medium by the image recording unit 111.

  Next, a case where the first imaging mode is set and the document DCM is imaged as a subject will be described. Even in the first imaging mode, subject monitoring (through image display) and image capturing execution are performed as described above. Here, in order to easily capture the document DCM, the user tilts the monitor unit PCD and the imaging apparatus CA together while visually recognizing the image displayed on the monitor 112. In this state, the center of the optical axis of the imaging optical system 101 is located on the document DCM placed on the table. Here, in the document DCM, the optical system driving unit 105 is set such that the upper position on the screen (the position farther away in the document DCM) than the optical axis center position (that is, the center of the screen) of the imaging optical system 101 in the document DCM. Drive. Thereby, the depth of field can be used effectively. Focusing is preferably performed automatically by the image plane AF as described above, but a distance measuring device may be provided separately.

  In this state, the user can acquire still image data by operating a predetermined key on the keyboard 113. Here, based on the signal from the solid-state image sensor 102, the control unit 104 includes an image including both the outline of the main body PCB that is a part of the notebook personal computer PC and the document DCM placed on the table. G1 (referred to as a primary acquired image) is acquired (see FIG. 4A). The side of the image of the document DCM near the solid-state image sensor 102 is longer than the side of the image of the document DCM far from the solid-state image sensor 102. Therefore, both side pieces of the image of the document DCM in the primary acquisition image G1 are tapered. It is deformed. Therefore, the control unit 104 obtains the contours of both sides of the main body PCB using the edge recognition technique or the like in the primary acquired image G1, and the reference lines L1 and L2 extending both sides and the side edges (vertical length) of the screen. Are obtained together with the image data of the primary acquired image G1 in a common file and stored in the image memory 108 as storage means. The image data may be subjected to predetermined compression. Such a file is recorded on a recording medium by the image recording unit 111.

  When image correction is performed by the image processing unit 109, the horizontal width of the primary acquired image G1 is increased toward the upper side of the screen so that the inclination angles γ1 and γ2 become zero, and the inclination angles γ1 and γ2 The image G2 is formed by performing the keystone correction (image processing) by enlarging the vertical direction at a setting magnification corresponding to (see FIG. 4B). With this correction, the document DCM has a rectangular shape with an original aspect ratio. Thereafter, the image processing unit 109 corrects the keystone of the image G2 and performs image processing so that the image G2 is inverted 180 degrees (inverted vertically and horizontally), so that the monitor 112 has an upright image as shown in FIG. G3 is displayed. Therefore, both the user US and the transmission destination can easily understand the contents of the document DCM, and when the user US moves the document DCM to the right, the image G3 also moves to the right. The characters written by the US in the document DCM can be read immediately. It should be noted that the keystone correction may be performed after the image is inverted vertically and horizontally. Alternatively, image processing may be performed so as to cut out along the outline of the document DCM to form the image G3 of only the document DCM.

  Alternatively, a file storing the image data of the primary acquired image G1 and the inclination angles γ1, γ2 is transmitted from the notebook personal computer PC via the LAN or the Internet, and the same trapezoid correction is performed by the image forming apparatus of the transmission destination. Also good. Note that the trapezoidal correction parameters are not limited to the inclination angles γ1 and γ2. For example, the sides of the image of the document DCM near the solid-state image sensor 102 and the sides of the image of the document DCM far from the solid-state image sensor 102 in the primary acquired image G1. It may be a ratio or the like.

  Note that the update timing of the trapezoidal correction parameter is, for example, the tilt angles γ1 and γ2 may be detected according to the operation of the keyboard 113 by the user US, and may be stored and updated as the trapezoidal correction parameter. The inclination angles γ1 and γ2 may be stored and updated at a predetermined cycle such as a second interval. Further, it is determined that the monitor unit PCD is displaced with respect to the main body PCB in response to a change in the output from the sensor 114 built in the hinge PCH, and the inclination angles γ1 and γ2 are detected and stored as trapezoid correction parameters. It may be updated.

  FIG. 5 is a diagram illustrating a modification in which the imaging device CA is provided so as to be tiltable with respect to the monitor unit PCD. FIG. 6 is a diagram illustrating an image before correction (dotted line) and an image after correction (solid line). Here, the inclination angle α of the monitor unit PCD with respect to the main body PCB can be detected by the signal of the sensor 114 as described above. Further, by providing another angle sensor (not shown) in the imaging apparatus CA, the inclination angle β of the imaging axis of the imaging apparatus CA with respect to the monitor unit PCD can be obtained. Here, the trapezoidal correction parameters are the inclination angles α and β. The means for obtaining the inclination angles α and β is not particularly limited.

The location point of the corrected image a _i, and c _ij, position a _i point image before correction ', when c _ij, are in a relationship expressed by the following equation. If the inclination angles α and β are known, the image processing unit 109 can perform keystone correction of the document DCM by collectively converting the coordinates of the entire screen based on the following formula.

  The present invention has been described above with reference to the embodiments. However, the present invention should not be construed as being limited to the above-described embodiments, and can be modified or improved as appropriate. For example, the electronic device is not limited to a personal computer but can be applied to a mobile phone or the like.

DESCRIPTION OF SYMBOLS 101 Image pick-up optical system 102 Solid-state image pick-up element 103 Conversion part 104 Control part 105 Optical system drive part 106 Timing generation part 107 Image pick-up element drive part 108 Image memory 109 Image processing part 110 Image compression part 111 Image recording part 112 Monitor 113 Keyboard 114 Sensor CA Imaging device DCM Document PC Notebook PC PCB Main body PCD Monitor section PCH Hinge section

Claims (6)

The body,
A camera unit having an imaging element and an imaging optical system that forms an optical image on a light receiving surface of the imaging element;
Image processing means for performing image processing on an image signal output from the camera unit;
A monitor unit that is arranged at least at an angle with respect to the main body, has the camera unit attached, and displays an image based on an image signal image-processed by the image processing unit,
The image processing means is configured to output an image signal output from the camera unit according to a setting of a first imaging mode for imaging a planar subject having a normal line inclined with respect to the optical axis of the imaging optical system. An electronic apparatus that performs keystone correction and performs image processing so that an image displayed on the monitor unit is vertically and horizontally reversed.   The electronic apparatus includes a storage unit, and the image processing unit sets a trapezoidal correction parameter from the image of the planar subject when a user performs a predetermined operation while the first imaging mode is set. The electronic apparatus according to claim 1, wherein image processing for keystone correction is performed based on the trapezoid correction parameter acquired and stored in the storage unit.   The electronic apparatus includes a storage unit, and the image processing unit acquires a keystone correction parameter from the image of the planar object at a predetermined time interval while the first imaging mode is set, and stores the storage unit. The electronic apparatus according to claim 1, wherein image processing for keystone correction is performed based on the trapezoid correction parameter stored in the means.   The electronic apparatus includes a storage unit, and the image processing unit is configured to change an inclination angle between the main body and the optical axis of the imaging optical system of the camera unit while the first imaging mode is set. The keystone correction parameter is acquired from the image of the planar object, stored in the storage means, and image processing for keystone correction is performed based on the stored keystone correction parameter. The electronic device as described in.   The image processing unit is configured to display an image displayed on the monitor unit with respect to an image signal output from the camera unit according to a setting of a second imaging mode in which a user operating the electronic device is captured as a subject. The electronic apparatus according to claim 1, wherein the image processing is performed so as to be reversed horizontally without performing upside down.   The direction of the optical axis of the imaging optical system of the camera unit is detected. If the detected direction is below horizontal, the first imaging mode is set. If the detected direction of the sensor is above horizontal, the first is set. 6. The electronic apparatus according to claim 5, further comprising mode setting means for setting two imaging modes.

Images