WO2023047719A1

WO2023047719A1 - Electronic apparatus

Info

Publication number: WO2023047719A1
Application number: PCT/JP2022/023227
Authority: WO
Inventors: 孝志大場
Original assignee: ソニーセミコンダクタソリューションズ株式会社
Priority date: 2021-09-21
Filing date: 2022-06-09
Publication date: 2023-03-30
Also published as: JP2023045100A

Abstract

[Problem] In one embodiment of the present disclosure, to provide an electronic apparatus for which bezel width can be reduced, and which can suppress a reduction in the image quality of a distance image. [Solution] This electronic apparatus comprises: a display unit; and an infrared imaging unit that is disposed on the side of the display unit opposite a display surface. The infrared imaging unit includes a plurality of photoelectric conversion units that photoelectrically convert infrared light which is incident via the display unit.

Description

Electronics

This disclosure relates to electronic equipment.

Recent electronic devices such as smartphones, mobile phones, and PCs (Personal Computers) are equipped with an infrared imaging unit that irradiates the user with infrared light and generates a distance image with distance information. is being considered.

In addition, smartphones and mobile phones are often carried in pockets or bags, so it is necessary to make the external size as compact as possible. On the other hand, if the size of the display screen is small, the higher the display resolution, the smaller the size of the displayed characters, making them difficult to see. For this reason, it is being studied to increase the size of the display screen as much as possible without increasing the external size of the electronic device by reducing the width of the frame (bezel) of the display section around the display screen.

JP 2021-97312 A

However, when a laser irradiation unit that irradiates infrared light and an infrared imaging unit that captures laser light are mounted on the frame (bezel) of the display unit, the frame (bezel) width of the display unit becomes large.

One aspect of the present disclosure provides an electronic device that can reduce the width of the bezel while suppressing deterioration in the image quality of the distance image.

In order to solve the above problems, the present disclosure provides a display unit,
An infrared imaging unit arranged on the opposite side of the display surface of the display unit,
The infrared imaging unit is
a plurality of photoelectric conversion units that photoelectrically convert infrared light incident through the display unit;
An electronic device is provided.

An irradiation unit that irradiates infrared light on the side opposite to the display surface of the display unit may be further provided.

The infrared imaging section and the irradiating section may be arranged at a distance such that internal reflected light within the display section due to the infrared light emitted by the irradiating section is reduced to a predetermined level.

The infrared imaging unit and the irradiating unit may be arranged at a distance such that the light amount of the internally reflected light is one-eighth or less when the infrared imaging unit and the irradiating unit are adjacent to each other. .

When the light propagation property of the internal structure in the display unit is anisotropic, the infrared imaging unit and the irradiation unit are placed at a distance equal to or less than the amount of internally reflected light at a position a predetermined distance away from the irradiation unit. The part may be arranged.

The infrared imaging unit and the irradiation unit may be arranged at the same height and in contact with the rear surface of the display unit opposite to the display surface.

Aligning the height of the infrared imaging unit and the irradiation unit,
A light blocking plate may be further provided between the infrared imaging section and the irradiation section.

The directions of the optical axes of the infrared imaging unit and the irradiation unit may be set so that the angles of view of the infrared imaging unit and the irradiation unit overlap.

A visible imaging unit may be arranged on the opposite side of the display surface of the display unit.

The infrared imaging section, the irradiation section, and the visible imaging section may be arranged on the opposite side of the display surface of the display section so that their centers of gravity form equilateral triangles.

The infrared imaging section, the irradiation section, and the visible imaging section may be linearly arranged on the opposite side of the display surface of the display section.

The visible imaging section may be arranged in an intermediate portion between the infrared imaging section, the irradiation section, and the visible imaging section.

The infrared imaging section and the irradiation section may be arranged so as to be parallel to the short side of the display surface of the display section.

The infrared imaging section and the irradiation section may be arranged so as to be parallel to the long side of the display surface of the display section.

When the light propagation property of the internal structure in the display section is anisotropic, the infrared imaging section and the irradiation section may be arranged in a direction in which light propagation is small.

A first signal processing unit that generates a two-dimensional distance image based on the distance signal generated by the infrared imaging unit may be further provided.

The visible imaging section may further include a second signal processing section that generates a two-dimensional visible image.

The control unit may cause the display unit to display the two-dimensional range image and the two-dimensional visible image side by side.

The second signal processing unit may generate, as the two-dimensional visible image, an image corresponding to the coordinates of the two-dimensional range image as the two-dimensional visible image.

The display unit may include a display panel using organic light emitting diodes.

2A is a schematic external view of the electronic device of FIG. 1, and FIG. 2B is a sectional view taken along line AA in FIG. The figure which shows typically a part of cross section of the BB line direction of Fig.1 (a). FIG. 2 is a block diagram showing a configuration example of a sensor unit; FIG. 2 is a block diagram showing a configuration example of a distance measurement unit; The figure explaining unnecessary light. FIG. 4 is a diagram showing the relationship between the distance between the laser irradiation unit and the infrared imaging unit and the amount of unnecessary light; FIG. 7 is a diagram showing an example in which the distance between the laser irradiation unit and the infrared imaging unit is set to be equal to or less than the threshold th shown in FIG. 6; The figure which shows a part of structure of the electronic device which concerns on 2nd Embodiment. FIG. 8 is a diagram showing the relationship between the distance between the laser irradiation unit and the infrared imaging unit in FIG. 7 and the amount of unnecessary light; FIG. 2 is a plan view schematically showing a configuration example of a display panel using organic light emitting diodes; The figure explaining the setting of distance when there is anisotropy in propagating property of light. FIG. 4 is a diagram showing an example of optical axes between a laser irradiation unit and an infrared imaging unit; The figure which shows the example of the optical axis between the laser irradiation part and infrared imaging part which concern on 4th Embodiment. The figure which shows the typical structural example of the electronic device by a 1st modification. The figure which shows the typical structural example of the electronic device by a 2nd modification. The figure which shows the typical structural example of the electronic device by a 3rd modification. The figure which shows the camera module of the electronic device which concerns on 4th Embodiment, and the whole system configuration example. 4 is a flowchart showing an example of processing according to the embodiment; FIG. 4 is a diagram showing an example of a distance image and a visible image displayed on the display panel;

Embodiments of the electronic device will be described below with reference to the drawings. Although the main components of the electronic device will be mainly described below, the electronic device may have components and functions that are not illustrated or described. The following description does not exclude components or features not shown or described.

(First embodiment)
FIG. 1 is a diagram showing a schematic configuration example of an electronic device 1 according to the first embodiment. The electronic device 1 in FIG. 1 is any electronic device having both a display function and a photographing function, such as a smart phone, a mobile phone, a tablet, or a PC. FIG. 1(a) is a schematic external view of the electronic device 1, and FIG. 1(b) is a sectional view taken along line AA in FIG. 1(a).

The electronic device 1 in FIG. 1 includes a camera module (visible imaging section) 3, a laser irradiation section 8, and an infrared imaging section 9 arranged on the opposite side of the display surface of the display section 2. As described above, the electronic device 1 of FIG. 1 has the camera module 3 , the laser irradiation section 8 , and the infrared imaging section 9 behind the display surface of the display section 2 . Therefore, the camera module 3 and the infrared imaging section 9 perform imaging through the display section 2 .

In the example of FIG. 1(a), the display screen 1a extends close to the external size of the electronic device 1, and the width of the bezel 1b around the display screen 1a is several millimeters or less. Normally, a front camera is often mounted on the bezel 1b. In FIG. 1(a), as indicated by the dashed line, a camera module 3 and a laser irradiation unit 8 are provided on the rear side of the substantially central portion of the display screen 1a. , and an infrared imaging unit 9 are arranged.

In this embodiment, the distance between the camera module 3 and the laser irradiation unit 8, the distance between the camera module 3 and the infrared imaging unit 9d, and the distance between the laser irradiation unit 8 and the infrared imaging unit 9d are approximately arranged to be equal. That is, the center of gravity of the camera module 3, the laser irradiation unit 8, and the infrared imaging unit 9 forms an equilateral triangle. By providing the camera module 3, the laser irradiation unit 8, and the infrared imaging unit 9 on the back side of the display screen 1a in this way, the camera module 3, the laser irradiation unit 8, and the infrared imaging unit 9 can be mounted on the bezel 1b. There is no need to dispose the bezel 1b, and the width of the bezel 1b can be narrowed.
As shown in FIG. 1, the display unit 2 is a structure in which a display panel 4, a circularly polarizing plate 5, a touch panel 6, and a cover glass 7 are laminated in order. The display panel 4 may be, for example, an organic EL display using OLED (Organic Light Emitting Dode), a liquid crystal display section, a MicroLED, or a display based on other display principles. A member having a low transmittance in the circularly polarizing plate 5 may have a through-hole formed in accordance with the locations where the laser irradiation unit 8 and the infrared imaging unit 9 are arranged. The image quality of the image picked up by the infrared imaging unit 9 can be improved by allowing the laser light passing through the through hole to enter the infrared imaging unit 9 .

The circularly polarizing plate 5 is provided to reduce glare and improve the visibility of the display screen 1a even in a bright environment. The touch panel 6 incorporates a touch sensor. There are various types of touch sensors, such as a capacitance type and a resistive film type, and any type may be used. Also, the touch panel 6 and the display panel 4 may be integrated. A cover glass 7 is provided to protect the display panel 4 and the like.

FIG. 2A is a diagram schematically showing a part of the cross section taken along line BB in FIG. 1(a). As shown in FIG. 2A, the laser irradiation unit 8 irradiates the imaging object with, for example, weak infrared laser light through the display unit 2 . This laser irradiation unit 8 comprises a beam diffuser 80 and a laser 81 on a substrate 82 .

The beam diffuser 80 functions as a holder that safely terminates the beam of the laser 81. The laser 81 is, for example, an infrared laser diode, and can emit weak infrared light in a pulsed manner. Note that the laser 81 according to this embodiment is, for example, an infrared laser diode, but is not limited to this. For example, a laser capable of irradiating infrared light may be used.

The infrared imaging unit 9 converts the light reflected from the imaging object via the display unit 2 into a distance signal. The infrared imaging section 9 includes an objective lens 91 , a lens barrel 92 , an infrared transmission filter (IRBPF) 93 and a sensor section 94 . The objective lens 91 collects the light reflected from the object to be imaged onto the light receiving surface of the sensor section 94 . A lens barrel 92 holds an objective lens 91 .

FIG. 2B is a block diagram showing a configuration example of the sensor section 94. As shown in FIG. As shown in FIG. 2B, the sensor section 94 has a sensor 94a and a readout circuit 94b. The sensor 94a is composed of iTOF pixels arranged two-dimensionally. The iTOF pixel has a photodiode as a photoelectric conversion section that generates a photocurrent corresponding to the amount of incident light. The readout circuit 94b generates a distance signal indicating distance information to an object based on the photocurrent output from each iTOF pixel. More specifically, the distance signal is a signal that indicates the amount of light received according to the elapsed time from the timing at which the laser 81 irradiates the pulsed laser. That is, when there is no noise light such as unnecessary light or ambient light, which will be described later, the signal value exhibits the maximum value (peak) at the timing when the reflected light returns from the imaging object.

FIG. 3 is a block diagram showing a configuration example of the distance measurement unit 20 according to this embodiment. As shown in FIG. 3 , the electronic device 1 has a distance measuring section 20 . The distance measurement unit 20 is a so-called lidar (LIDAR: Light Detection and Ranging, Laser Imaging Detection and Ranging). That is, the distance measurement unit 20 is a device that generates a distance image by, for example, the IToF (Time of Flight) method, and includes a laser irradiation unit 8 (see FIGS. 1 and 2A) and an infrared imaging unit 9 (FIGS. 2A and 2B), a control unit 10, and a processing unit 11.

The control unit 10 includes, for example, a CPU, and controls the laser irradiation unit 8, the infrared imaging unit 9, and the processing unit 11. The control unit 10 synchronizes the irradiation timing of the pulsed laser light of the laser irradiation unit 8 and the signal generation timing of the infrared imaging unit 9 .

The processing unit 11 processes the distance signal output by the infrared imaging unit 9 . The processing unit 11 has an AD conversion unit 11a, a memory unit 11b, a signal processing unit 11c, and an output unit 11d. The AD conversion section 11a converts the distance signal output from the infrared imaging section 9 into a digital signal and stores it in the memory section 11b. That is, the memory section 11b stores a digital distance signal.

The signal processing unit 11c obtains the maximum value (peak) of the digital distance signal and generates a distance value according to the elapsed time from the timing of laser irradiation to the occurrence of the maximum value (peak). That is, the signal processing unit 11c multiplies the timing at which the reflected light returns from the imaging object by the speed of light C, and divides the result by 2 to generate the distance value to the imaging object. Then, the signal processing unit 11c generates two-dimensional distance image data of the imaged object based on the distance information based on the sensor in which the photoelectric conversion units are two-dimensionally arranged, and stores the data in the memory unit 11b. Further, the signal processing unit 11c can perform recognition processing of two-dimensional distance image data. For example, the signal processing unit 11c can perform face authentication using two-dimensional distance image data. The output unit 11d outputs two-dimensional distance image data and the like to the display panel 4 and the like.

FIG. 4 is a diagram for explaining unnecessary light according to this embodiment. As shown in FIG. 4, when the laser irradiation unit 8 irradiates the laser light from the back side of the display unit 2, for example, the surface reflected light of the display panel 4 directly above the laser irradiation unit 8 and the structure in the display panel 4 ( (See FIG. 4) resulting in internally reflected light that propagates. Among the light other than the incident light from the front side of the display section 2, the light incident on the infrared imaging section 9 is referred to as unnecessary light in the present embodiment.

FIG. 5 is a diagram showing the relationship between the distance between the laser irradiation unit 8 and the infrared imaging unit 9 and the amount of unnecessary light. The horizontal axis indicates the distance between the laser irradiation unit 8 and the infrared imaging unit 9, and the vertical axis indicates the amount of unnecessary light. As shown in FIG. 5, the amount of unnecessary light decreases as the distance between the laser irradiation unit 8 and the infrared imaging unit 9 increases. Therefore, in the present embodiment, the distance between the laser irradiation unit 8 and the infrared imaging unit 9 is increased to a level at which the unnecessary light is, for example, equal to or less than the threshold value th. The threshold th is experimentally set within a range in which the signal level indicating the distance of the distance signal is not buried in the level of unnecessary light. For example, the threshold th is set to the maximum value of unnecessary light, that is, 1/8 of the value when the laser irradiation unit 8 and the infrared imaging unit 9 are adjacent to each other.

FIG. 6 is a diagram showing an example in which the distance between the laser irradiation unit 8 and the infrared imaging unit 9 is made equal to or less than the threshold th shown in FIG. As shown in FIG. 6, the distance between the laser irradiation unit 8 and the infrared imaging unit 9 is set within a range in which the amount of unnecessary light is equal to or less than the threshold th shown in FIG. In addition, the heights of the laser irradiation unit 8 and the infrared imaging unit 9 to the display panel 4 are made uniform to prevent surface reflected light from entering the infrared imaging unit 9 . Note that referring to FIG. 1 again, the distance between the laser irradiation unit 8 and the infrared imaging unit 9 is set within a range in which the amount of unnecessary light is equal to or less than the threshold value th shown in FIG.

As described above, according to the present embodiment, the laser irradiation unit 8 and the infrared imaging unit 9 are arranged on the side opposite to the display surface of the display unit 2, and the incident light through the display unit 2 is converted into infrared light. The image was taken by the imaging unit 9 . This makes it possible to further reduce the width of the frame (bezel) of the display unit 2 . In addition, the distance between the laser irradiation unit 8 and the infrared imaging unit 9 is arranged at such a distance that the influence of unnecessary light is suppressed. This suppresses the internal reflected light and the surface reflected light of the display section 2 from entering the infrared imaging section 9 .

(Second embodiment)
The electronic device 1 according to the second embodiment differs from the electronic device 1 according to the first embodiment in that a light blocking plate 15 is provided between the laser irradiation section 8 and the infrared imaging section 9 . Differences from the electronic device 1 according to the first embodiment will be described below.

FIG. 7 is a diagram showing part of the configuration of the electronic device 1 according to the second embodiment. As shown in FIG. 7 , the electronic device 1 according to the second embodiment includes a light shielding plate 15 between the laser irradiation section 8 and the infrared imaging section 9 . In addition, the heights of the laser irradiation unit 8, the infrared imaging unit 9, and the light shielding plate 15 to the display panel 4 are aligned, and the surface reflected light enters the infrared imaging unit 9 by the light shielding plate 15. is suppressed.

FIG. 8 is a diagram showing the relationship between the distance between the laser irradiation unit 8 and the infrared imaging unit 9 and the amount of unnecessary light in the configuration of FIG. The horizontal axis indicates the distance between the laser irradiation unit 8 and the infrared imaging unit 9, and the vertical axis indicates the amount of unnecessary light. As shown in FIG. 8, the amount of unnecessary light decreases as the distance between the laser irradiation unit 8 and the infrared imaging unit 9 increases. Also, compared with the example of the unnecessary light in FIG. 5, it shows that the distance at which the threshold value th or less is shorter.

As described above, according to this embodiment, the light shielding plate 15 is provided between the laser irradiation unit 8 and the infrared imaging unit 9 . As a result, the surface reflected light from the back side of the display unit 2 is suppressed from entering the infrared imaging unit 9, and the distance between the laser irradiation unit 8 and the infrared imaging unit 9 can be further shortened. be.

(Third Embodiment)
In the electronic device 1 according to the third embodiment, when the light propagation property of the display panel 4 is anisotropic, the light propagation property of the display panel 4 is also taken into consideration, and the laser irradiation unit 8 and the infrared imaging unit 9 is different from the electronic device 1 according to the first embodiment in that the distance between is set. Differences from the electronic device 1 according to the first embodiment will be described below.

FIG. 9 is a plan view schematically showing a configuration example of the display panel 4 using organic light emitting diodes. As shown in FIG. 9, the display panel 4 has a pixel section 38, a vertical scanning circuit 40, and a horizontal scanning circuit 42 formed on, for example, a silicon substrate. A plurality of scanning lines from the vertical scanning circuit 40 extend horizontally to the pixel section 38, and a plurality of data lines from the horizontal scanning circuit 42 extend vertically. The pixel portions 38 are connected in a matrix to the data lines extending in the vertical direction and the scanning lines extending in the horizontal direction. FIG. 9 also shows a control section 30 that controls the display panel 4 .

Three scanning lines are wired for each pixel row along the row direction (pixel arrangement direction of the pixel row) with respect to the arrangement of the matrix-shaped pixel portions 38 . In addition, data lines are wired for each pixel column along the column direction (the direction in which the pixels in the pixel column are arranged) with respect to the arrangement of the pixel portions 38 arranged in a matrix. The display panel 4 is provided with pixel circuits corresponding to pixels of three primary colors, as indicated by R (red), B (blue), and G (green). These three pixels represent one dot of the color image. Also, the combination of pixels expressing one unit (one dot) is not limited to this, and may be configured by adding W (white) pixels for improving brightness, adding complementary color pixels for expanding the color reproduction range, can be configured as As shown in FIG. 9, as indicated by R (red), B (blue), and G (green), the X-direction and Y-direction structures of the three primary color pixels and the structure of the circuit layer are different. As a result, it is known that there is anisotropy in light propagation between the X direction and the Y direction of the display panel 4 .

FIG. 10 is a diagram for explaining distance setting when the display panel 4 shown in FIG. 9 has anisotropy in light propagation. FIG. 10 shows an example in which the light transmittance is anisotropic in the x direction and the y direction, and the light propagation area A10 in the x direction is higher than that in the y direction, as described above. That is, the internally reflected light in the x direction with respect to the laser irradiation unit 8 is higher than the internally reflected light in the y direction. In such a case, by comparing the intensity of the internally reflected light in the region A12 separated from the laser irradiation unit 8 by a predetermined distance in the y direction with the intensity of the internally reflected light in the region A14 whose distance is changed in the x direction, It is possible to set the distance more accurately. For example, when the intensity of the internally reflected light in the area A12 and the intensity of the internally reflected light in the area A14 are almost equal, the internal reflection in the x-direction with high light transmittance converges to the same level as the internally reflected light in the area A14. indicate that

Therefore, in the electronic device 1 according to the third embodiment, the distance in the x direction, which is highly transient, between the laser irradiation unit 8 and the infrared imaging unit 9 is determined by the internally reflected light of the comparison area A14 in the y direction, which has low transmittance. shall be set based on As a result, even when the laser irradiation unit 8 and the infrared imaging unit 9 are arranged in the direction of high light propagation, they can be arranged at a distance at which the influence of the internally reflected light is suppressed.

(Fourth embodiment)
The electronic device 1 according to the fourth embodiment differs from the electronic device 1 according to the first embodiment in that the angle between the optical axes O8 and O9 between the laser irradiation unit 8 and the infrared imaging unit 9 is narrowed. Differences from the electronic device 1 according to the first embodiment will be described below.

FIG. 11 is a diagram showing an example of optical axes O8 and O9 between the laser irradiation unit 8 and the infrared imaging unit 9 of the electronic device 1 according to the first embodiment. When the display unit 2 with strong unnecessary light is used, if the distance between the laser irradiation unit 8 and the infrared imaging unit 9 is increased, the angle of view between the laser irradiation unit 8 and the infrared imaging unit 9 does not overlap. , it becomes difficult to image the imaging object 100 .

FIG. 12 is a diagram showing an example of optical axes O8 and O9 between the laser irradiation unit 8 and the infrared imaging unit 9 of the electronic device 1 according to the fourth embodiment. When using the display unit 2 with strong unnecessary light, the angles of the optical axes O8 and O9 between the laser irradiation unit 8 and the infrared imaging unit 9 are narrowed. As a result, the angles of view of the laser irradiation unit 8 and the infrared imaging unit 9 are overlapped, and the imaging accuracy of the imaging object 100 is further improved.

(First modification of the first to third embodiments)
In the electronic device 1 according to the first to third embodiments, the camera module 3, the laser irradiation unit 8, and the infrared imaging unit 9 are arranged separately in the long side direction of the display unit 2. The electronic device 1 according to the first modification is different in that it is arranged side by side with a camera module (visible imaging section) 3 in the long side direction of the display section 2 . Differences from the electronic device 1 according to the first to third embodiments will be described below.

FIG. 13 is a diagram showing a schematic configuration example of the electronic device 1 according to the first modified example. The electronic device 1 in FIG. 13 is an arbitrary electronic device having both a display function and a photographing function, such as a smart phone, a mobile phone, a tablet, or a PC. FIG. 13(a) is a schematic external view of the electronic device 1, and FIG. 13(b) is a sectional view taken along line AA of FIG. 13(a). As shown in FIG. 13 , a camera module (visible imaging section) 3 , a laser irradiation section 8 , and an infrared imaging section 9 are linearly arranged in the long side direction of the display section 2 . A camera module (visible imaging unit) 3 is arranged in the center. That is, the laser irradiation unit 8 and the infrared imaging unit 9 are arranged above and below the camera module (visible imaging unit) 3 .

As a result, when the unnecessary light in the long side direction of the display unit 2 is smaller than the unnecessary light in the short side direction, the unnecessary light can be further reduced, so the distance between the laser irradiation unit 8 and the infrared imaging unit 9 can be further shortened. . In addition, since the camera module 3, the laser irradiation unit 8, and the infrared imaging unit 9 are arranged linearly in the long side direction, a distance image symmetrical to the image of the camera module 3 in the short side direction can be obtained. Imaging becomes possible.

(Second Modification of First to Third Embodiments)
In the electronic device 1 according to the first to third embodiments, the camera module 3, the laser irradiation unit 8, and the infrared imaging unit 9 are arranged separately in the long side direction of the display unit 2. The electronic device 1 according to the second modification is different in that it is arranged side by side with a camera module (visible imaging section) 3 in the short side direction of the display section 2 . Differences from the electronic device 1 according to the first to third embodiments will be described below.

FIG. 14 is a diagram showing a schematic configuration example of the electronic device 1 according to the second modified example. The electronic device 1 in FIG. 14 is an arbitrary electronic device having both a display function and a photographing function, such as a smart phone, a mobile phone, a tablet, or a PC. 14(a) is a schematic external view of the electronic device 1, and FIG. 14(b) is a cross-sectional view taken along line AA of FIG. 14(a). As shown in FIG. 14 , a camera module (visible imaging section) 3 , a laser irradiation section 8 , and an infrared imaging section 9 are arranged linearly in the short side direction of the display section 2 . A camera module (visible imaging unit) 3 is arranged in the center. That is, the laser irradiation unit 8 and the infrared imaging unit 9 are arranged on the left and right sides of the camera module (visible imaging unit) 3 .

As a result, when unnecessary light in the short side direction of the display section 2 is smaller than unnecessary light in the long side direction, the unnecessary light can be further reduced, so that the distance between the laser irradiation section 8 and the infrared imaging section 9 can be further shortened. . In addition, since the camera module 3, the laser irradiation unit 8, and the infrared imaging unit 9 are arranged in a straight line in the short side direction, a distance image symmetrical to the image of the camera module 3 in the long side direction can be obtained. Imaging becomes possible.

(Third Modification of First to Third Embodiments)
In the electronic device 1 according to the first to third embodiments, the camera module 3, the laser irradiation unit 8 and the infrared imaging unit 9 are arranged apart in the long side direction of the display 3 unit 2, In the electronic device 1 according to the third modification, a camera module (visible imaging section) 3, a laser irradiation section 8, and an infrared imaging section 9 are arranged side by side in a direction oblique to the longitudinal direction of the display section 2. differ in Differences from the electronic device 1 according to the first to third embodiments will be described below.

FIG. 15 is a diagram showing a schematic configuration example of the electronic device 1 according to the third modified example. The electronic device 1 in FIG. 15 is any electronic device having both a display function and a photographing function, such as a smart phone, a mobile phone, a tablet, or a PC. FIG. 15(a) is a schematic external view of the electronic device 1, and FIG. 15(b) is a sectional view taken along line AA of FIG. 15(a). As shown in FIG. 15 , a camera module (visible imaging section) 3 , a laser irradiation section 8 , and an infrared imaging section 9 are arranged in a straight line in a direction oblique to the longitudinal direction of the display section 2 . A camera module (visible imaging unit) 3 is arranged in the center. That is, the laser irradiation unit 8 and the infrared imaging unit 9 are arranged at obliquely separated positions of the camera module (visible imaging unit) 3 .

As a result, when unnecessary light in the oblique direction of the display unit 2 is smaller than unnecessary light in the direction of the short side or the direction of the long side, the unnecessary light can be further reduced. can be shortened.

(Fifth embodiment)
The electronic device 1 according to the fifth embodiment is further capable of displaying the captured image of the camera module 3 and the distance image of the distance measurement unit 20 in association with each other, which is the same as the electronic device 1 according to the first to third embodiments. It is different from the electronic device 1 . Differences from the electronic device 1 according to the first to third embodiments will be described below.

FIG. 16 is a diagram showing an example of the camera module 3 and the overall system configuration of the electronic device 1 according to the fifth embodiment. The camera module 3 of the electronic device 1 according to the fifth embodiment includes a pixel section 80, a vertical drive section 130, analog-to-digital conversion (hereinafter referred to as "AD conversion")

sections

140 and 150, a

column processing section

160, 170 , a memory unit 180 , a system control unit 190 , a signal processing unit 51 and an interface unit 52 . FIG. 16 further shows the camera module 3, the control unit 10 (see FIG. 3), and the central control unit 300 that controls the control unit 30 (see FIG. 9). The central control unit 300 controls the electronic device 1 as a whole. Note that the control unit 10 , the control unit 30 , and the central control unit 300 may be configured integrally as a control processing unit 1000 .

As shown in FIG. 16, pixels are arranged in a matrix in the pixel unit 80 . In this pixel array, a pixel drive line is wired along the row direction for each pixel row, and two

vertical signal lines

310 and 320, for example, are wired along the column direction for each pixel column. The pixel drive lines transmit drive signals for driving when reading out signals from the pixels of the pixel section 80 . One end of the pixel drive line is connected to an output terminal corresponding to each row of the vertical drive section 130 .

The vertical driving section 130 is composed of a shift register, an address decoder, etc., and drives each pixel of the pixel section 80 simultaneously or in units of rows. The vertical drive section 130 generally has a configuration having two scanning systems, a readout scanning system and a discharge scanning system. The readout scanning system sequentially selectively scans each pixel row by row. A signal read from each pixel is an analog signal. The sweep-scanning system performs sweep-scanning ahead of the read-out scanning by the shutter speed for the read-out rows to be read-scanned by the read-out scanning system.

Due to sweeping scanning by this sweeping scanning system, unnecessary charge is swept out from the photoelectric conversion section of each pixel in the readout row, thereby resetting the photoelectric conversion section. A so-called electronic shutter operation is performed by sweeping out (resetting) unnecessary charges by this sweeping scanning system. Here, the electronic shutter operation refers to an operation of discarding photocharges in the photoelectric conversion unit and newly starting exposure (starting accumulation of photocharges).

The signal read out by the readout operation by the readout scanning system corresponds to the amount of light received after the immediately preceding readout operation or the electronic shutter operation. The period from the readout timing of the previous readout operation or the sweep timing of the electronic shutter operation to the readout timing of the current readout operation is the exposure period of the photocharges in the unit pixel.

A pixel signal output from each pixel in the pixel row selected by the vertical drive section 130 is input to the

AD conversion sections

140 and 150 through two systems of

vertical signal lines

310 and 320 . Here, the vertical signal line 310 of one system transmits the pixel signals output from the pixels 80, 82, 80a, and 82a of the selected row in the first direction (one side in the pixel column direction/ (upward direction in the figure). The vertical signal line 320 of the other system transmits the pixel signal output from each pixel of the selected row in the second direction opposite to the first direction (the other side in the pixel column direction/downward direction in the figure). It consists of a signal line group (second signal line group).

The

AD converters

140 and 150 each consist of a set of AD converters 141 that AD-convert input pixel signals. The pixel data (digital data) AD-converted by the

AD converters

140 and 150 are supplied to the memory section 180 via the

column processors

160 and 170 . The signal processing unit 81 performs signal processing such as noise reduction processing on the pixel data, and supplies visible image data to the display panel 4 via the interface unit 52 .

FIG. 17 is a flowchart showing a processing example according to this embodiment. As shown in FIG. 17, a visible image of an object is captured by the camera module 3 under the control of the central control unit 300 (step S100). Subsequently, the distance image of the captured object is captured by the distance measurement unit 20 under the control of the central control unit 300 (step S102). Then, under the control of the central control unit 300, the display panel 4 associates and displays the distance image and the visible image. For example, the central control unit 300 controls the signal processing unit 11c and the signal processing unit 51 so that the coordinate positions of the distance image and the visible image correspond. For example, the distance image g1(x, y) and the visible image g2(x, y) are associated so that the same position of the imaging unit is arranged. That is, the coordinates (x, y) of the range image and the coordinates (x, y) of the visible image are configured to point to the same location on the captured object. For example, the signal processing unit 51 uses the distance value of the distance image and the information of the optical system of the visible image to perform processing to associate the distance image g1(x, y) with the visible image g2(x, y). will be Since the camera module 3 and the infrared imaging unit 9 are arranged close to the back surface of the display unit 2, such processing can be performed more easily.

FIG. 18 is a diagram showing an example of a distance image and a visible image displayed on the display panel 4. FIG. In FIG. 18, the distance image and the visible image are generated so that their coordinate positions correspond to each other and are arranged side by side. This facilitates comparison between the distance image and the visible image.

As described above, according to the present embodiment, the display panel 4 included in the display unit 2 displays the distance image captured by the distance measurement unit 20 and the visible image captured by the camera module 3 in association with each other. bottom. This facilitates comparison between the distance image and the visible image.

This technology can be configured as follows.

(1)
a display unit;
An infrared imaging unit arranged on the opposite side of the display surface of the display unit,
The infrared imaging unit is
a plurality of photoelectric conversion units that photoelectrically convert infrared light incident through the display unit;
electronic equipment.

(2)
The electronic device according to (1), further comprising an irradiating section that irradiates infrared light on the side opposite to the display surface of the display section.

(3)
The electronic device according to (2), wherein the infrared imaging unit and the irradiation unit are arranged at a distance such that internal reflected light in the display unit due to infrared light emitted by the irradiation unit is reduced to a predetermined level. device.

(4)
The infrared imaging unit and the irradiating unit are arranged at a distance such that the light amount of the internally reflected light is 1/8 or less when the infrared imaging unit and the irradiating unit are adjacent to each other, ( 3) The electronic device described in 3).

(5)
When the light propagation property of the internal structure in the display unit is anisotropic, the infrared imaging unit and the irradiation unit are placed at a distance equal to or less than the amount of internally reflected light at a position a predetermined distance away from the irradiation unit. The electronic device according to (2), wherein the unit is arranged.

(6)
The electronic device according to (2), wherein the infrared imaging unit and the irradiating unit are arranged at the same height and in contact with the rear surface of the display unit opposite to the display surface.

(7)
Aligning the height of the infrared imaging unit and the irradiation unit,
The electronic device according to (2), further comprising a light blocking plate arranged between the infrared imaging section and the irradiation section.

(8)
The electronic device according to (2), wherein directions of optical axes of the infrared imaging unit and the irradiation unit are set so that angles of view of the infrared imaging unit and the irradiation unit overlap.

(9)
The electronic device according to (2), further comprising a visible imaging section arranged on the side opposite to the display surface of the display section.

(10)
The infrared imaging unit, the irradiating unit, and the visible imaging unit are arranged on the opposite side of the display surface of the display unit so that their centers of gravity form equilateral triangles. Electronics.

(11)
The electronic device according to (9), wherein the infrared imaging section, the irradiation section, and the visible imaging section are linearly arranged on the opposite side of the display surface of the display section.

(12)
The electronic device according to (9), wherein the visible imaging section is arranged in an intermediate portion between the infrared imaging section, the irradiation section, and the visible imaging section.

(13)
The electronic device according to (9), wherein the infrared imaging unit and the irradiation unit are arranged so as to be parallel to a short side of the display surface of the display unit.

(14)
The electronic device according to (9), wherein the infrared imaging unit and the irradiation unit are arranged so as to be parallel to the long side of the display surface of the display unit.

(15)
The electronic device according to (2), wherein the infrared imaging section and the irradiating section are arranged in a direction of less light propagation when the light propagation property of the internal structure of the display section is anisotropic.

(16)
The electronic device according to (9), further comprising a first signal processing unit that generates a two-dimensional distance image based on the distance signal generated by the infrared imaging unit.

(17)
The electronic device according to (16), wherein the visible imaging section further includes a second signal processing section that generates a two-dimensional visible image.

(18)
Further comprising a control unit for controlling the display unit,
The electronic device according to (17), wherein the control unit causes the display unit to display the two-dimensional distance image and the two-dimensional visible image side by side.

(19)
The electronic device according to (18), wherein the second signal processing unit generates, as the two-dimensional visible image, an image corresponding to the coordinates of the two-dimensional range image as the two-dimensional visible image.

(20)
The electronic device according to (5), wherein the display unit includes a display panel using an organic light emitting diode.

Aspects of the present disclosure are not limited to the individual embodiments described above, but include various modifications that can be conceived by those skilled in the art, and the effects of the present disclosure are not limited to the above-described contents. That is, various additions, changes, and partial deletions are possible without departing from the conceptual idea and spirit of the present disclosure derived from the content defined in the claims and equivalents thereof.

1: Electronic device 1a: Display screen 1b: Bezel 2: Display unit 3: Camera module (visible imaging unit) 4: Display panel 8: Laser irradiation unit 9: Infrared imaging unit 10: Control unit 11c: Signal processing unit 30: Control unit 38: Pixel unit 51: Signal processing unit 94a: Sensor 200: Central control unit.

Claims

a display unit;
An infrared imaging unit arranged on the opposite side of the display surface of the display unit,
The infrared imaging unit is
a plurality of photoelectric conversion units that photoelectrically convert infrared light incident through the display unit;
electronic equipment.
The electronic device according to claim 1, further comprising an irradiation unit that irradiates infrared light on the side opposite to the display surface of the display unit.
3. The electronic device according to claim 2, wherein said infrared imaging unit and said irradiating unit are arranged at a distance such that internal reflected light within said display unit due to infrared light emitted by said irradiating unit is reduced to a predetermined level. device.
The infrared imaging unit and the irradiating unit are arranged at a distance such that the light amount of the internally reflected light is one-eighth or less when the infrared imaging unit and the irradiating unit are adjacent to each other. Item 4. The electronic device according to item 3.
When the light propagation property of the internal structure of the display unit is anisotropic, the infrared imaging unit and the irradiation unit are placed at a distance equal to or less than the amount of internally reflected light at a position at a predetermined distance from the irradiation unit. 3. The electronic device according to claim 2, wherein the unit is arranged.
3. The electronic device according to claim 2, wherein the infrared imaging unit and the irradiation unit are arranged at the same height and in contact with the rear surface of the display unit opposite to the display surface.
Aligning the height of the infrared imaging unit and the irradiation unit,
3. The electronic device according to claim 2, further comprising a light blocking plate arranged between said infrared imaging section and said irradiation section.
The electronic device according to claim 2, wherein the directions of the optical axes of the infrared imaging unit and the irradiation unit are set so that the angles of view of the infrared imaging unit and the irradiation unit overlap.
The electronic device according to claim 2, further comprising a visible imaging section arranged on the side opposite to the display surface of the display section.
10. The apparatus according to claim 9, wherein the infrared imaging unit, the irradiation unit, and the visible imaging unit are arranged on the opposite side of the display surface of the display unit so that their centers of gravity form equilateral triangles. Electronics.
The electronic device according to claim 9, wherein the infrared imaging section, the irradiation section, and the visible imaging section are linearly arranged on the opposite side of the display surface of the display section.
The electronic device according to claim 9, wherein the visible imaging section is arranged in an intermediate portion between the infrared imaging section, the irradiation section, and the visible imaging section.
The electronic device according to claim 9, wherein the infrared imaging unit and the irradiation unit are arranged so as to be parallel to the short side of the display surface of the display unit.
The electronic device according to claim 9, wherein the infrared imaging unit and the irradiation unit are arranged so as to be parallel to the long side of the display surface of the display unit.
3. The electronic device according to claim 2, wherein the infrared imaging section and the irradiating section are arranged in a direction of less light propagation when the internal structure of the display section has an anisotropic light propagation property.
The electronic device according to claim 9, further comprising a first signal processing section that generates a two-dimensional distance image based on the distance signal generated by the infrared imaging section.
The electronic device according to claim 16, wherein the visible imaging section further includes a second signal processing section that generates a two-dimensional visible image.
Further comprising a control unit for controlling the display unit,
18. The electronic device according to claim 17, wherein the control unit causes the display unit to display the two-dimensional distance image and the two-dimensional visible image side by side.
The electronic device according to claim 18, wherein the second signal processing unit generates, as the two-dimensional visible image, an image corresponding to the coordinates of the two-dimensional range image as the two-dimensional visible image.
The electronic device according to claim 5, wherein the display unit includes a display panel using organic light emitting diodes.