JP2007156757A - Display device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a display deice reduced in heating value noise and size capable of capturing an image with high resolution. <P>SOLUTION: This display device is provided with a display 2 having light permeability and an image pickup unit. Each image pickup unit is provided with image pickup parts 41a and 42a(or 42a and 42b) having an imaging optical system and an image selection part. The image pickup parts 41a and 41b(or 42a and 42b) receive the rays of light made incident from an observer side to a display panel 2, and passing through the display panel 2 and the image pickup optical system, and pick up the status of the observer side in the same region within the display region of the display panel 2. The image selection part compares the images picked up by the image pickup parts 41a and 41b(42a or 42b) with each other, and selects the image having the least number of pixels whose luminance exceed a threshold, and defines it as a pickup image. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a display device, and more particularly to a display device provided with an image input function.

  In recent years, in the field of display devices, display devices having an input function in addition to a display function have become widespread. An example of such a display device is a display device with a touch panel (see, for example, Patent Document 1). The display device disclosed in Patent Document 1 guides light emitted from a projector to a display area, and receives light reflected by a user's finger placed on the display area with the CCD camera. The touch position is detected by.

In addition to a display device with a touch panel, a liquid crystal display device configured to be able to capture an image itself is also disclosed (see, for example, Patent Document 2). The liquid crystal display device disclosed in Patent Document 2 includes a plurality of photodiodes arranged in a matrix on an active matrix substrate, thereby capturing an image of an object on a display screen.
JP 2001-350586 A (FIG. 1) JP 2004-159273 A (FIGS. 2 to 3)

  However, in the display device disclosed in Patent Document 1, since it is necessary to use a projector, there are a problem that the noise caused by the power supply fan is large, a problem that the heat generation amount is large, and a problem that the entire apparatus cannot be reduced in size.

  Further, the liquid crystal display device disclosed in Patent Document 2 has a configuration that enables capturing of an image, but does not include an imaging optical system. It is impossible to obtain a captured image. The liquid crystal display device of Patent Document 2 has a problem that it is impossible to capture a high resolution.

  An object of the present invention is to provide a display device that solves the above-described problems, generates a small amount of heat, can be reduced in size and size, and can capture a high-resolution image.

  In order to achieve the above object, a display device according to the present invention includes a light-transmissive display panel and an imaging unit, and the imaging unit includes a plurality of imaging units having an imaging optical system, and an image selection unit. Each of the plurality of imaging units receives light that is incident on the display panel from the observer side and passes through the display panel and the imaging optical system, and is the same in the display area of the display panel. The image selection unit compares the images captured by each of the plurality of imaging units with each other, and selects and selects the image with the smallest number of pixels whose luminance exceeds the threshold. The image data is output to the outside.

  As described above, according to the display device of the present invention, since an image can be displayed using a light-transmissive display panel such as a liquid crystal display panel or an EL display panel, a reduction in heat generation, a reduction in noise, and a reduction in size can be achieved. . Furthermore, in the display device according to the present invention, since the imaging unit includes the imaging optical system, it is possible to capture a high-resolution image as compared with a display device having a conventional input function.

  The display device according to the present invention includes a light-transmissive display panel and an imaging unit, and the imaging unit includes a plurality of imaging units having an imaging optical system and an image selection unit, and the plurality of imaging units. Each receives light that is incident on the display panel from the viewer side and passes through the display panel and the imaging optical system, and is on the viewer side in the same region within the display region of the display panel. The image selection unit compares the images captured by the plurality of imaging units with each other, selects an image with the smallest number of pixels whose luminance exceeds a threshold value, and outputs data of the selected image to the outside It is characterized by doing.

  With the above features, the display device according to the present invention can reduce the amount of heat generation, reduce the noise, and reduce the size, and can capture a high-resolution image. Further, in the display device according to the present invention, one area is captured by a plurality of imaging units, and an image with little overexposure among the captured images is output as a captured image of the area. For this reason, it is possible to suppress overexposure in the captured image when light having high directivity is incident on the display region from the outside.

  Further, the display device according to the present invention includes a detection light source unit that emits detection light to the viewer side of the display panel, each of the plurality of imaging units is incident on the display panel from the viewer side, and It is preferable that the detection light passing through the display panel and the imaging optical system is received to image a state on the observer side in the same area in the display area. According to the above aspect, the resolution of the captured image can be improved.

  Moreover, in the display device according to the present invention, the plurality of imaging units are located on the back side of the display panel, in the periphery of the region overlapping the display region in the thickness direction of the display panel. It is also preferable that the imaging directions of some imaging units and the imaging directions of some other imaging units are arranged so as to intersect when viewed from the observer side. According to the above aspect, since light with high directivity (environmental light) incident from the observer side can be prevented from being incident on all of the plurality of image capturing units at the same time, overexposure is performed on all images captured by the image capturing unit. The situation that occurs can be avoided. That is, according to the said aspect, it can suppress further that overexposure arises in a captured image.

  Furthermore, the display device according to the present invention includes a plurality of the imaging units, and each of the plurality of imaging units targets a different area within the display area as an imaging target, and the plurality of imaging units included in each imaging unit. It is also preferable that the unit images the state on the observer side of the region that is an imaging target by the imaging unit including them. According to the said aspect, since the area of the area | region which one imaging unit makes into an imaging object can be made small, the imaging area of each imaging part can be narrowed. Therefore, compared with the case where there is a single imaging unit, the optical imaging distance (focal length) required by each imaging unit can be shortened, and the display device can be thinned.

  In this case, it is preferable that adjacent regions partially overlap each other among regions to be imaged of the plurality of imaging units. Thereby, the position of the subject can be accurately recognized.

  In the display device according to the present invention, it is preferable that the detection light source unit emits light having a wavelength of 700 nm or more, and each of the plurality of imaging units receives only light having a wavelength of 700 nm or more. Preferably, the detection light source unit emits light having a wavelength of 800 nm or more and 1000 nm or less, and each of the plurality of imaging units receives only light having a wavelength of 800 nm or more. In such a case, noise due to visible light can be removed, and the resolution of the captured image can be further increased.

  Furthermore, in the display device according to the present invention, the display panel may be a liquid crystal display panel or an EL display panel.

(Embodiment 1)
Hereinafter, the display device according to Embodiment 1 of the present invention will be described with reference to FIGS. First, the overall configuration of the display device according to the first embodiment will be described with reference to FIGS.

  FIG. 1 is a cross-sectional view showing a schematic configuration of a display device according to Embodiment 1 of the present invention. FIG. 2 is a plan view showing the positional relationship between the imaging unit and the display panel that constitute the display device shown in FIG. FIG. 3 is a cross-sectional view illustrating a schematic configuration of the imaging unit illustrated in FIGS. 1 and 2. 4 is a diagram showing a schematic configuration of the detection light source unit shown in FIGS. 1 and 2, FIG. 4A is a cross-sectional view cut along the emission direction of detection light, and FIG. 4B is a front view. FIG.

  As shown in FIGS. 1 and 2, the display device according to the first embodiment includes a display panel 2 having optical transparency and an imaging unit. The imaging unit includes a plurality of imaging units having an imaging optical system and an image selection unit. In the first embodiment, a first imaging unit having imaging units 41a and 41b, a second imaging unit having imaging units 42a and 42b, a third imaging unit having imaging units 43a and 43b, And a fourth imaging unit having imaging units 44a and 44b. In FIG. 1 and FIG. 2, the illustration of the image selection unit is omitted. The image selection unit will be described later with reference to FIGS. In FIG. 1, the illustration of the imaging units 43 a to 44 b is omitted because of the illustration.

  As shown in FIG. 2, in the first embodiment, the first to fourth imaging units have different areas in the display area 3 of the display panel 2 as imaging targets. Specifically, the first imaging unit has an area A as an imaging target, and the imaging area (2) has an area B as an imaging target. Furthermore, the third imaging unit has an area C as an imaging target, and the fourth imaging unit has an area D as an imaging target.

  Further, the plurality of imaging units provided in the same imaging unit images the state on the observer side of the region that is the imaging target of the imaging unit including them. That is, the plurality of image pickup units provided in the same image pickup unit pick up an image of the state on the observer side of the same region. Therefore, in the first embodiment, the state of the region A on the observer side is imaged by the imaging units 41a and 41b, and the state of the region B on the observer side is imaged by the imaging units 42a and 42b. . Further, the state on the observer side of the region C is imaged by the imaging units 43a and 43b, and the state on the observer side of the region D is imaged by the imaging units 44a and 44b. However, the areas to be imaged by each imaging unit are set so that adjacent areas partially overlap as shown in FIG. 2 so that the position of the subject 1 can be accurately recognized. Is preferred.

  In the first embodiment, the display panel 2 is a liquid crystal display panel, and the display device is a liquid crystal display device. On the back side of the display panel 2, a backlight device 5 for illuminating the display panel 2 is disposed. The display panel 2 includes an active matrix substrate 2c, a liquid crystal layer 2b, and a filter substrate (counter substrate) 2a. The liquid crystal layer 2b is sandwiched between the active matrix substrate 2c and the filter substrate 2a. Illustration of a seal for sealing the liquid crystal layer 2b is omitted. In addition, a polarizing plate (not shown) is provided on the surface opposite to the liquid crystal layer 2b side in each of the filter substrate 2a and the active matrix substrate 2c.

  A plurality of active elements (not shown) arranged in a matrix are formed on the active matrix substrate 2c. The active element constitutes a pixel, and a region overlapping with the region where the pixel is provided in the thickness direction (indicated by a thick line arrow in FIG. 1) is a display region 3. Further, although not shown, the active matrix substrate 2c is provided with a drive circuit such as a gate drive circuit and a source drive circuit. A plurality of color filters (not shown) corresponding to each pixel and counter electrodes are formed on the filter substrate 2a.

  In addition, the display panel 2 should just be provided with the light transmittance. Other examples of the display panel 2 include an EL display panel. When the display panel 2 is an EL display panel, it is not necessary to arrange a backlight device because the EL display panel emits light. Furthermore, in this case, the display panel 2 is, for example, an ITO (Indium Tin Oxide) film that becomes a transparent electrode (anode), a hole transport layer, an electron transport layer, a back electrode (cathode), etc. in this order on a transparent substrate. It is constructed by stacking.

  The backlight device 5 is a direct-type backlight device and includes a plurality of fluorescent lamps 6 and an optical layer 13. The plurality of fluorescent lamps 6 are disposed in a bathtub-type housing 8 in a state parallel to each other (see FIG. 2). A reflective sheet is attached to the inner surface of the housing 8. The optical layer 13 is formed by sequentially laminating a diffusion plate 9, a diffusion sheet 10, a prism sheet 11, and a reflection / polarization sheet 12.

  Thus, in the first embodiment, the display device is a liquid crystal display device. Therefore, the imaging units 41a to 44b are arranged on the back surface side of the display panel and around the area overlapping the display area 3 in the thickness direction of the display panel 2 in consideration of adverse effects on the illumination light from the backlight device 5. Has been placed. However, in this case, since the imaging units 41a to 44b must perform imaging from an oblique direction, trapezoidal distortion may occur in the captured image. For this reason, in this Embodiment 1, the imaging parts 41a-44b are provided with the shift optical system.

  Specifically, as illustrated in FIG. 3, the imaging units 41 a to 44 b include a lens element 30 that forms an imaging optical system, a solid-state imaging element 32 that receives an image formed by the lens element 30, and an optical unit. And a filter 31. The solid-state imaging device 32 is a solid-state imaging device such as a CCD solid-state imaging device or a MOS solid-state imaging device. The optical filter 31 is provided to guide only detection light, which will be described later, to the solid-state imaging device 32. In the first embodiment, the optical filter 31 is a high-pass filter that transmits only light having a wavelength equal to or greater than the infrared region.

  The lens element 30 and the solid-state imaging element 32 constitute a so-called shift optical system. That is, in the solid-state imaging element 32 and the lens element 30, the normal line 32a passing through the center of the light receiving surface of the solid-state imaging element 32 and the optical axis 30a of the lens element 30 are parallel, and the optical axis 30a is shifted from the normal line 32a. In this state, the frame 33 is held. Further, as illustrated in FIG. 1, in each of the imaging units 41 a to 44 b, the light receiving surface of the solid-state imaging device 32 is parallel to the display region 3, and the normal line 32 a of the solid-state imaging device 32 is the lens element 30. It arrange | positions so that it may be located in the outer side of the display area 3 rather than the optical axis 30a.

  Thus, in this Embodiment 1, since the imaging parts 41a-44b employ | adopted the shift optical system, even if it is imaging from the diagonal direction, the light-receiving surface of the solid-state image sensor 32 has trapezoid distortion. An image with less is formed. According to the first embodiment, the occurrence of trapezoidal distortion is suppressed in the images captured by the imaging units 41a to 44b.

  2 and 3 indicate the imaging direction of each imaging unit. The imaging direction is a direction along a line connecting the center of the light receiving surface of the solid-state imaging device 32 and the center of the region to be imaged. Further, in the first embodiment, the imaging units 41 a to 44 b capture an image of the subject 1 on the display area 3 and in the vicinity thereof. Therefore, the focusing range of the lens element 30 (imaging optical system) of the imaging units 41a to 44b is preferably set within the range of 1 cm near the surface of the display panel 2, for example, from the display area 3 to the viewer side. .

  Further, in the first embodiment, the display panel 2 and the backlight device 5 are compared with a conventional liquid crystal display device in order to increase the optical imaging distance (focal length of the imaging optical system) of the imaging units 41a to 44b. Placed at a distance. Specifically, the display panel 2 and the backlight device 5 are held at a certain distance L by the frame 20, and a cavity exists between them. In addition, the imaging units 41 a to 44 b are also held in the frame 20.

  For example, if the size of the display panel 2 is about 30 inches, the distance L between the display panel 2 and the backlight device 5 is set to about 15 cm. A transparent resin material may be filled between the display panel 2 and the backlight device 5 in order to increase the strength of the display device.

  Further, since the distance between the display panel 2 and the backlight device 5 is large, as shown in FIGS. 1 and 2, the backlight device 5 has a light emitting area larger than the display area 3. It is preferable to configure. This is because the irradiation area of the display panel 2 tends to decrease when the distance between the display panel 2 and the backlight device 5 is large.

  Further, in the first embodiment, a different area is set as an imaging target for each imaging unit. For this reason, compared with the case where the entire display area 3 is imaged by a single imaging unit, the imaging area of one imaging unit can be narrowed, and the optical imaging distance required by the imaging units 41a to 44b is shortened. be able to. Therefore, the distance L between the display panel 2 and the backlight device 5 can be shortened compared to the case where only a single imaging unit is provided, and the display device can be thinned.

  As shown in FIGS. 1 and 2, in the first embodiment, the display device includes a detection light source unit 7 that emits detection light to the viewer side of the display panel 2. As shown in FIG. 4, in the first embodiment, the detection light source unit 7 includes a plurality of light sources 21 that emit detection light. The light sources 21 are arranged in parallel so that their emission directions coincide. Each light source 21 is a light emitting diode that emits light having a wavelength in the infrared region. Reference numeral 22 denotes a frame of the detection light source unit 7, and reference numeral 23 denotes a resin fixing member that fixes each light source 21 to the frame 22. Reference numeral 24 denotes a light guide formed of a transparent resin material.

  Furthermore, as shown in FIG. 2, in the first embodiment, a plurality of detection light source units 7 are arranged around the display area 3 on the viewer side of the display panel 2 in order to reduce the light amount loss of the detection light. Has been. Specifically, four detection light source units 7 are arranged, and each detection light source unit 7 is arranged so as to surround the display region 3 along any side of the display region 3. However, the number of the detection light source parts 7 is not specifically limited. For example, the number of the detection light source units 7 may be two, and the detection light source units 7 may be arranged only on two opposite sides. In addition, the detection light source unit 7 can be arranged on the back surface side of the display panel 2 with the detection light emission direction directed toward the observer side. From the viewpoint of irradiation efficiency, the detection light source unit 7 is shown in FIGS. Thus, it is preferable to arrange on the viewer side of the display panel 2.

  As described above, in the first embodiment, since the display device includes the detection light source unit 7, the detection light emitted from the detection light source unit 7 is from the subject 1 existing on or near the display region 3. Reflected. Further, the reflected detection light enters the display panel 2 from the observer side of the display panel 2 and passes through the display panel 2 and the imaging optical system (see FIG. 3). In the first embodiment, the imaging units 41a to 44b receive the detection light that has passed through the display panel 2 and the imaging optical system, thereby imaging the state on the observer side of the region to be imaged.

  Therefore, according to the display device in the present embodiment, the amount of heat generation can be reduced, the noise can be reduced, and the size can be reduced. Moreover, since the imaging units 41a to 44b include the imaging optical system, the conventional input function is provided. Compared to a display device, it is possible to capture a high-resolution image.

  By the way, since the display device is used in various environments, ambient light having high directivity may be incident on the display region 3 from the outside. Such light includes, for example, sunlight, and there is also light having a wavelength in the infrared region in the same manner as detection light. Therefore, when such light is incident on the display area, overexcitation occurs in the captured image, and the quality of the image is degraded. Note that overexposure means that the luminance of part of the pixels constituting the image exceeds a threshold value, and part or all of the image becomes whitish.

  For this reason, in this Embodiment 1, as above-mentioned, the area | region used as one imaging object is imaged by changing the angle by the several imaging part. For example, when light with high directivity is incident on the area A shown in FIG. 2, a large overexposure occurs in the image captured by the imaging unit 41a, but the image captured by the imaging unit 41b is conspicuous. There may be no whiteout. At this time, if the image picked up by the image pickup unit 41b is taken as the picked-up image of the region A, a picked-up image with little quality degradation can be obtained.

  Therefore, as described above, each imaging unit includes an image selection unit in addition to the imaging unit. The image selection unit compares images captured by each of the plurality of imaging units, selects an image having the smallest number of pixels whose luminance exceeds a threshold value, and uses the selected image as a captured image.

  Furthermore, as shown in FIG. 2, in each imaging unit, the two imaging units intersect when the imaging direction of one imaging unit and the imaging direction of the other imaging unit are viewed from the observer side. Are arranged as follows. For this reason, environmental light incident from the observer side and having high directivity can be prevented from being incident on one imaging unit and the other imaging unit at the same time, so that the images captured by both imaging units are overexposed. It is possible to avoid such a situation and easily obtain a captured image with less overexposure.

  Further, the angle formed by the imaging direction of one imaging unit and the imaging direction of the other imaging unit when viewed from the observer side may be determined in consideration of the incident angle of the ambient light incident on the display device. . However, in order to more effectively suppress the simultaneous incidence of ambient light to both imaging units, the imaging direction of one imaging unit and the imaging direction of the other imaging unit when viewed from the observer side As shown in FIG. 2, the angle formed is preferably 90 degrees. In addition, when the angle formed by both is 90 degrees, it is preferable to dispose the imaging unit along two orthogonal sides of the display region 3 as shown in FIG.

  In the first embodiment, the display device further includes a control device, and performs image processing and the like on images obtained by the respective imaging units. Here, image selection processing by the image selection unit and image processing by the control device will be described. First, the configuration of the control device will be described with reference to FIG. FIG. 5 is a block diagram illustrating a configuration of a control device provided in the display device illustrated in FIGS. 1 and 2.

  As shown in FIG. 5, in the first embodiment, image selection units 45 to 48 are provided for each of the first to fourth imaging units. For example, the image selection unit 45 compares the images captured by the imaging units 41a and 41b with each other. Specifically, the image selection unit 45 measures the number of pixels whose luminance is equal to or greater than a threshold for each image, and compares the measured values. Next, the image selection unit 45 selects an image having the smallest number of pixels whose luminance is equal to or higher than the threshold value. Furthermore, the image selection unit 45 sets the selected image as a captured image, and outputs captured image data of the selected image to the image input control device 105.

  It should be noted that the threshold value of luminance serving as a reference for selection may be set in advance at the factory shipment stage, or may be in an aspect that can be arbitrarily set by the user of the display device. Further, the image selection unit may be single and shared by each imaging unit.

  As shown in FIG. 5, the display device 110 according to the first embodiment includes a control device 109 that performs image processing and the like based on imaging data output from the imaging unit 4. The control device 109 mainly includes an image processing device 100, an image input control unit 105, and a display control unit 107. The control device 109 is connected to the external device 108. Examples of the external device 108 include a personal computer, a game device, a TV tuner, a DVD player, and various devices that output video signals to a display device such as a home appliance.

  The image input control unit 105 obtains output of imaging data from the first imaging unit to the fourth imaging unit in response to an instruction by a control signal from the external device 108. Each of the first to fourth imaging units outputs imaging data of the selected image to the image input control unit 105 when an output request is notified by a control signal from the image input control unit 105. The image input control unit 105 temporarily stores imaging data in the memory 106 for each of the first to fourth imaging units, and then outputs each imaging data to the image processing apparatus 100. Further, the image input control unit 105 can also perform an imaging instruction, sensitivity setting, resolution setting, and the like for the imaging unit of each imaging unit in response to an instruction by a control signal of the external device 108.

  Each piece of imaging data input to the image processing apparatus 100 is first synthesized by the image synthesis unit 101 to be converted into one image data and sent to the noise removal unit 102. First, the noise removal unit 102 subtracts the offset component from the image data output by the image synthesis unit 101. Furthermore, the noise removing unit 102 removes display components from the image data after removing the offset components. Thereafter, the noise removal unit 102 outputs the obtained image data to the image recognition unit 103.

  Here, the offset component is irradiated from the backlight device 5 (see FIG. 1), and then reflected by the surface of the constituent member of the display device 110, the interface between the constituent members, and the like, and solid-state imaging of the imaging units 41a to 44b. The light quantity component incident on the element 32 (see FIG. 3). The display component refers to a light amount component incident on the imaging units 41 a to 44 b from the outside via the display panel 2. The display component varies depending on the image displayed in the display area 3 (see FIGS. 1 and 2).

  The display component is calculated from the set reference data, and the above processing is performed using the calculated value. The offset component is set in advance and stored in a memory (not shown) provided in the image processing apparatus 100. The reference data for calculating the display component is similarly set in advance and stored in a memory (not shown) included in the image processing apparatus 100.

  The offset component can be set by the following procedure, for example. First, it is assumed that the incidence of light from the outside to the display area 3 is blocked. For example, the display device is placed in a dark room, or the display region 3 is covered with a sheet or dark curtain that transmits detection light but does not transmit visible light. Furthermore, it is assumed that there is no object as a subject on the display area 3.

  Next, the light source 6 of the backlight device 5 is turned on and the detection light source unit 7 irradiates the detection light, and the imaging data output by the first to fourth imaging units at this time is acquired. The imaging data acquired at this time, that is, the light amount component incident on the solid-state imaging device 32 of the imaging units 41a to 44b in the above state becomes an offset component. The setting of the offset component can be performed at the factory shipment stage of the display device, or can be performed by the user as needed after the start of use.

  The reference data for calculating the display component can be set, for example, by the following procedure. First, under the installation environment of the display device 110, it is assumed that there is no object as a subject on the display area 3. Next, when an external light component enters the imaging units 41a to 44b via the display panel 2, the maximum value and the minimum value of the imaging data output by these components, that is, when the display panel 2 is in a white display state and black display is displayed. Capture image data when in a state. Next, the output width of the display component due to the incidence of the external light component in the installation environment of the display device 110 is calculated from the acquired imaging data. Data of the output width calculated at this time becomes data (reference data) for calculating the display component. If the use environment of the display device 110 is the same, the setting operation described above may be performed once at the time of installation.

  The display component is calculated from the gradation level of the display image obtained from the video signal and the output width (reference data) calculated in advance under the situation where the display device 110 is used. Specifically, the above-described removal of the display component is performed by the following procedure. First, the noise removing unit 102 extracts the gradation level of the display image from the video signal output from the external device 108, and calculates a display component from the extracted gradation level and reference data. Next, the noise removing unit 102 subtracts the display component from the imaging data output by each imaging unit at this time. Further, as described above, since the display component varies depending on the image displayed in the display area, it is necessary to calculate the display component as needed according to the change in the display state.

  Further, the image recognition unit 103 identifies the image and position of the subject based on the image data from which noise has been removed, and outputs these to the external device 108 as image data. The external device 108 performs various types of processing using the input image data, and outputs a video signal reflecting the processing to the display control unit 107. Examples of processing performed by the external device 108 include cursor movement processing and click operation processing if the external device 108 is a personal computer. Further, the display control unit 107 generates a control signal based on the video signal and outputs it to a drive circuit (not shown) of the display panel 2.

  Next, processing performed by the imaging unit and the control device will be described with reference to FIG. FIG. 6 is a flowchart showing processing performed in the display device shown in FIGS. 1 and 2. The process illustrated in FIG. 6 is performed every time the imaging units 41a to 41b perform imaging, that is, for each frame.

As shown in FIG. 6, first, when the image input control unit 105 instructs the first imaging unit to the fourth imaging unit to output imaging data, all the imaging units 41 a to 44 b perform imaging and the captured images. Are output to the corresponding image selection units 45 to 48 (step S1).
Next, each image selection unit measures the number of pixels whose luminance is equal to or greater than a threshold value for the image captured by the corresponding imaging unit (step S2). Subsequently, each image selection unit compares the value measured in step S2 with respect to the image captured by the corresponding imaging unit (step S3). For example, the image selection unit 46 compares the image captured by the imaging unit 42a with the image captured by the imaging unit 42b.

  Next, each image selection unit selects an image having a smaller number of pixels whose luminance is equal to or higher than a threshold as an image of the region to be imaged (step S4), and selects the selected image (selected image). It outputs to the image input control apparatus 105 (step S5). Thereafter, as described above, the image input control unit 105 temporarily stores the imaging data of the selected image in the memory 106 for each of the first imaging unit to the fourth imaging unit, and then performs image processing on each imaging data. Output to the device 100.

  Note that if there is no pixel whose luminance is greater than or equal to the threshold value in both images, the image selection unit, for example, selects an image with a higher ratio of the highest luminance and the lowest luminance, that is, a higher contrast ratio in step S4. It is good also as an aspect to do. According to this aspect, it is possible to easily perform processing in image processing described later.

  Furthermore, when an imaging unit that is preferentially selected is set in advance, such as an imaging unit arranged along a vertical side of the display area 3, and there is no pixel whose luminance is equal to or greater than a threshold value The image picked up by the image pickup unit having a high priority may be selected. Alternatively, the above-described contrast ratio comparison may be performed, and when the contrast ratio is the same, an image captured by an imaging unit with a high priority may be selected. The priority order of the imaging units can be appropriately set in consideration of the size of the display panel, the use of the display device, and the like.

  Next, the image composition unit 101 performs image composition, sets the selected image as one image, and outputs the obtained image data of one image to the noise removal unit 102 (step S6). Further, the noise removal unit 102 performs the above-described noise removal, and outputs the image data of the image from which the noise has been removed to the image recognition unit 103 (step S7). Thereafter, the image recognition unit 103 identifies the subject image and its position from the image from which the noise has been removed, and outputs the image data of the subject image together with the position information to the external device 108 (step S8). .

  With the above steps S1 to S8, the imaging of the subject 1 is completed. Note that steps S1 to S8 may be performed continuously for a set number of times, or may be performed only when the image input unit 105 gives an instruction.

  As described above, in the display device according to the first embodiment, a region to be imaged in the display region 3 is imaged by a plurality of imaging units, and among the captured images, an image with less overexposure is included in the region. The captured image. For this reason, according to the display apparatus in this Embodiment 1, when the light with high directivity injects into the display area from the outside, it can suppress that an overshoot occurs in a captured image.

  In the first embodiment, the control device 109 can be provided by, for example, an IC chip with a built-in CPU. The IC chip can be mounted on a substrate connected to the display panel 2 via an FPC, for example. In this case, the CPU built in the IC chip can function as the display control unit 107, the image input control unit 105, and the image processing apparatus 100. The image selection unit 45 can be provided by an IC chip with a built-in CPU, or can be provided by an IC chip with a built-in counter circuit and comparison circuit.

(Embodiment 2)
Next, a display device according to Embodiment 2 of the present invention will be described with reference to FIGS. First, the overall configuration of the display device according to the second embodiment will be described with reference to FIGS.

  FIG. 7 is a cross-sectional view showing a schematic configuration of the display device according to Embodiment 2 of the present invention. FIG. 8 is a plan view showing the positional relationship between the imaging unit and the display panel that constitute the display device shown in FIG. FIG. 9 is a cross-sectional view illustrating a schematic configuration of the imaging unit illustrated in FIGS. 7 and 8.

  As shown in FIGS. 7 and 8, the display device according to the second embodiment includes imaging units 51a to 54a, and is different from the display device according to the first embodiment in this respect. Except for this point, the display device in the second embodiment is configured in the same manner as the display device in the first embodiment. Hereinafter, differences will be described.

  As shown in FIG. 9, in the imaging units 51 a to 54 b, unlike the imaging units 41 a to 44 b shown in FIG. 3 in the first embodiment, the shift optical system is configured by the lens element 34 and the solid-state imaging element 32. Not. In the second embodiment, the lens element 34 and the solid-state image sensor 32 are arranged on the frame 35 so that the optical axis 34a of the lens element 34 and the normal line 32a passing through the center of the light-receiving surface of the solid-state image sensor 32 coincide. Is retained.

  For this reason, as shown in FIGS. 1 and 2, the imaging units 51 a to 54 b are arranged in a state where the imaging direction is inclined toward the area to be imaged. In the second embodiment, the imaging direction coincides with the optical axis 34 a of the lens element 34 and the normal line 32 a passing through the center of the light receiving surface of the solid-state imaging element 32.

  As described above, in the second embodiment, the shift optical system is not employed in the imaging units 51a to 55b. Therefore, compared with the first embodiment, the diameter of the lens system constituting the imaging optical system is reduced, and the connection is reduced. The image optical system can be simplified, and the design cost can be reduced. However, since the imaging units 51a to 55b do not employ a shift optical system, images captured by the imaging units are distorted in a trapezoidal shape. Therefore, in the second embodiment, as described later, the captured image distorted in the trapezoidal shape is corrected to a rectangular shape by the control device.

  Here, the image processing by the control device in the second embodiment will be described. First, the configuration of the control device will be described with reference to FIG. FIG. 10 is a block diagram illustrating a configuration of a control device provided in the display device illustrated in FIGS. 7 and 8. FIG. 11 is a flowchart illustrating processing performed in the display device illustrated in FIGS. 7 and 8.

  As shown in FIG. 10, the control device 109 according to the second embodiment is different from the control device according to the first embodiment in that the image processing apparatus 100 includes an image correction unit 104. In the second embodiment, the image data of each imaging unit output by the image input control unit 105 is first corrected for trapezoidal distortion by the image correction unit 104, and then the image composition, offset component, and display component are corrected. Removal (noise removal) and image recognition are performed.

  That is, in the second embodiment, as shown in FIG. 11, when the image selection units 45 to 48 output the imaging data of the selected image (step S5), the image correction unit 104 corrects the distortion of each selected image. Is performed (step S9), and then steps S6 to S8 are executed in the same manner as in the first embodiment.

  Specifically, the image correction unit 104 calculates the upper base, the lower base, and the height of the trapezoidal captured image. Calculation of the upper base, the lower base, and the height can be calculated from the positions of the imaging units 51a to 54b and the size and position of the imaging area. The calculation of the upper base, the lower base, and the height can be performed based on the marks included in the captured image by arranging a plurality of marks in the display area 3 in advance.

  Next, the image correction unit 104 enlarges the captured image while changing the magnification according to the position of the scanning line so that the shorter one of the upper base and the lower base of the captured image is the same as the larger one. In the direction perpendicular to the scanning line direction (height direction), the image correcting unit 104 enlarges or reduces the captured image so that the height becomes the original height. The original height is set in advance from the size of the imaging area.

  As described above, in the second embodiment, the problem caused by not employing the shift optical system in the imaging units 51a to 51b is solved by the image processing after imaging. Also in the second embodiment, the optical image of the subject 1 on the display area 3 is formed on the light receiving surface by the imaging optical system. Therefore, even when the display device according to the second embodiment is used, a clear optical image can be obtained as compared with the conventional one, and high-resolution image capture can be achieved as in the first embodiment. Further, also in the second embodiment, since each imaging unit includes a plurality of imaging units, it is possible to suppress overexposure in a captured image when light having high directivity is incident on the display area from the outside. it can.

  In the present invention, the “imaging optical system” refers to a lens system having a focal point in the vicinity of the surface of the display panel and the light receiving surface of the imaging unit, and forming an image in the vicinity of the surface of the display panel on the light receiving surface. Therefore, in the example of FIGS. 3 and 9, the imaging optical system is configured by only the lens elements, but may be configured by a lens group including a plurality of lens elements. However, when the shift optical system shown in FIG. 3 is configured, the imaging optical system needs to be designed so that oblique light is transmitted through the lens system without being kicked. In this case, the imaging optical system needs to be configured by a lens system having a large aperture as compared with the case where the shift optical system is not configured.

  In the present invention, the number of image capturing units included in one image capturing unit is not particularly limited. One imaging unit may include three or more imaging units. The number of image capturing units can be set as appropriate in consideration of the size of the area to be imaged and the imageable range. Further, the number of imaging units arranged in one display area is not limited. A mode in which one imaging unit is arranged for one display area may be employed. In addition, when one display area is divided into a number other than the four divisions shown in the first and second embodiments, a plurality of imaging units may be arranged in accordance with the division number.

  As described above, the display device of the present invention has an input function, is useful as a display device for personal computers, televisions, game machines, and the like, and has industrial applicability.

FIG. 1 is a cross-sectional view showing a schematic configuration of a display device according to Embodiment 1 of the present invention. FIG. 2 is a plan view showing the positional relationship between the imaging unit and the display panel that constitute the display device shown in FIG. FIG. 3 is a cross-sectional view illustrating a schematic configuration of the imaging unit illustrated in FIGS. 1 and 2. 4 is a diagram showing a schematic configuration of the detection light source unit shown in FIGS. 1 and 2, FIG. 4A is a cross-sectional view cut along the emission direction of detection light, and FIG. 4B is a front view. FIG. FIG. 5 is a block diagram illustrating a configuration of a control device provided in the display device illustrated in FIGS. 1 and 2. FIG. 6 is a flowchart showing processing performed in the display device shown in FIGS. 1 and 2. FIG. 7 is a cross-sectional view showing a schematic configuration of the display device according to Embodiment 2 of the present invention. FIG. 8 is a plan view showing the positional relationship between the imaging unit and the display panel constituting the display device shown in FIG. FIG. 9 is a cross-sectional view illustrating a schematic configuration of the imaging unit illustrated in FIGS. 7 and 8. FIG. 10 is a block diagram illustrating a configuration of a control device provided in the display device illustrated in FIGS. 7 and 8. FIG. 11 is a flowchart showing processing performed in the display device shown in FIGS. 7 and 8.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Subject 2 Display panel 2a Filter substrate 2b Liquid crystal layer 2c Active matrix substrate 3 Display area 5 Backlight device 6 Fluorescent lamp 7 Detection light source part 8 Case 9 Diffusion plate 10 Diffusion sheet 11 Prism sheet 12 Reflection / polarization sheet 13 Optical layer 20 Display device frame 21 Light source of detection light source unit 22 Detection light source unit frame 23 Fixing member 24 Light guide path 30, 34 Lens element (imaging optical system)
30a, 34a Optical axis 31 Optical filter 32 Solid imaging device 32a Normal line passing through the center of the light receiving surface of the solid imaging device 33, 35 Imaging unit frame 41a, 41b Imaging unit 42a, 42b imaging imaging region A Imaging region B Imaging units 43a and 43b for imaging Imaging units 44a and 44b for imaging imaging region C Imaging units 45 to 48 and 55 to 58 imaging selection unit 51a and 51b Imaging units 52a and 52b for imaging imaging region A Imaging units 53a and 53b that capture the imaging region B Imaging units 54a and 54b that capture the imaging region C Imaging unit that captures the imaging region D 100 Image processing device 101 Image composition unit 102 Noise removal unit 103 Image recognition unit 104 Image correction unit 105 Image Input Control Unit 106 Memory 107 Display Control Unit 108 External Device 109 Control Device 11 Display device

Claims (8)

A display panel having optical transparency and an imaging unit;
The imaging unit includes a plurality of imaging units having an imaging optical system, and an image selection unit,
Each of the plurality of imaging units receives light that is incident on the display panel from the observer side and passes through the display panel and the imaging optical system, and is the same area in the display area of the display panel The state of the observer side in
The image selection unit compares the images captured by each of the plurality of imaging units, selects an image having the smallest number of pixels whose luminance exceeds a threshold value, and outputs data of the selected image to the outside. Display device. A detection light source unit that emits detection light to an observer side of the display panel;
Each of the plurality of imaging units receives the detection light that is incident on the display panel from the observer side and passes through the display panel and the imaging optical system, and is in the same region within the display region. The display device according to claim 1, wherein the state on the observer side is imaged.   The plurality of imaging units are arranged on the back side of the display panel, around a region overlapping the display region in the thickness direction of the display panel, and the imaging direction of some of the plurality of imaging units 3. The display device according to claim 1, wherein the imaging directions of some of the imaging units are arranged so as to intersect when viewed from the observer side. A plurality of the imaging units are provided,
Each of the plurality of imaging units has a different area within the display area as an imaging target,
The display device according to claim 1, wherein the plurality of imaging units included in each imaging unit capture an image of a state on an observer side of an area that is an imaging target of the imaging unit including the imaging units.   The display device according to claim 4, wherein adjacent regions of the plurality of imaging units are partially overlapped with each other. The detection light source unit emits light having a wavelength of 700 nm or more;
The display device according to claim 2, wherein each of the plurality of imaging units receives only light having a wavelength of 700 nm or more. The detection light source unit irradiates light having a wavelength of 800 nm to 1000 nm;
The display device according to claim 6, wherein each of the plurality of imaging units receives only light having a wavelength of 800 nm or more.   The display device according to claim 1, wherein the display panel is a liquid crystal display panel or an EL display panel.

Images