WO2019193939A1

WO2019193939A1 - Display device, display system, and code-attached printed matter

Info

Publication number: WO2019193939A1
Application number: PCT/JP2019/010123
Authority: WO
Inventors: 直之高崎
Original assignee: 株式会社ジャパンディスプレイ
Priority date: 2018-04-05
Filing date: 2019-03-12
Publication date: 2019-10-10
Also published as: JP2021131400A

Abstract

This display device is provided with: a display panel having a plurality of pixel electrodes located in a display area, a common electrode located in the display area, and a display function layer located in the display area; a sensor having a plurality of detection electrodes disposed at least in the display area; and a control unit which controls driving of the plurality of pixel electrodes, the common electrode, and the plurality of detection electrodes. The control unit can make at least a portion of the display area of the display panel have a transparent state, and displays, in the display area, a first image associated with information detected by the sensor.

Description

Display device, display system, and printed matter with code

Embodiments of the present invention relate to a display device, a display system, and a printed matter with code.

As a display device, a display device including a camera is known. A technique is known in which information about a printed material is displayed on a screen of the display device by displaying a printed material on a screen of the display device by capturing the printed material with a camera of the display device. For example, the display device can display the photographed printed material so as to start moving.

JP 2006-72428 A

In the present embodiment, a display device that can see a printed material through a screen and display an image associated with the printed material on the screen by electrically detecting the printed material, and a display including the display device A system and a printed matter with a code that can be detected by the display device are provided.

A display device according to an embodiment includes:
A display panel having a plurality of pixel electrodes located in the display area, a common electrode located in the display area, and a display functional layer located in the display area, and a plurality of detection electrodes arranged in at least the display area A control unit that controls driving of each of the plurality of pixel electrodes, the common electrode, and the plurality of detection electrodes, and the control unit includes at least one of the display regions of the display panel. The first image associated with the information detected by the sensor is displayed on the display area.

In addition, a display system according to an embodiment includes:
A display panel having a plurality of pixel electrodes located in the display area, a common electrode located in the display area, and a display functional layer located in the display area, and a sensor having at least a plurality of detection electrodes arranged in the display area A display device comprising: a control unit that controls driving of each of the plurality of pixel electrodes, the common electrode, and the plurality of detection electrodes; a substrate having a main surface; and a printed material that is displayed on the main surface. A printed image with a code having a second image and a transparent first code which is located on the main surface and has information on the second image and which can be detected by the sensor, and the control unit includes: At least a part of the display area of the display panel can be in a transparent state, and is attached to the second image based on the information of the first code of the printed matter detected by the sensor. Displaying the first image in the display area.

Moreover, the printed matter with code according to one embodiment is:
A printed material having a main surface, a second image displayed on the main surface, and a transparent first code that is located on the main surface and has information on the second image and has conductivity.

Moreover, the printed matter with code according to one embodiment is:
A printed material having a main surface, a second image displayed on the main surface, and a transparent first code that is located on the main surface and has information about the second image, is formed of a dielectric, and has a three-dimensional pattern And comprising.

FIG. 1 is a block diagram illustrating a display system according to an embodiment. FIG. 2 is a perspective view showing the display device shown in FIG. FIG. 3 is a plan view showing a configuration example of a part of the display device. FIG. 4 is a cross-sectional view showing a part of the display device. FIG. 5 is a diagram showing some components of the display device. FIG. 6A is a cross-sectional view showing the display panel when the liquid crystal layer is in a transparent state. FIG. 6B is a cross-sectional view showing the display panel when the liquid crystal layer is in a scattering state. FIG. 7 is a plan view showing the configuration of the sensor of FIGS. 1 and 4. FIG. 8 is a plan view showing the printed matter with code shown in FIG. FIG. 9 is a cross-sectional view showing the printed matter along line IX-IX in FIG. FIG. 10 is a cross-sectional view showing a part of the printed matter and the display device, and shows a state where the display device is placed on the printed matter. FIG. 11 is a plan view showing the printed material and the display device, and shows a state where the printed material is displayed through the screen and the first image associated with the printed material is displayed on the screen. . FIG. 12 is a flowchart for explaining a method of taking a photograph and a moving image and storing information in which the photograph and the moving image are associated with each other in the storage unit of the display device. FIG. 13 is a flowchart for explaining a method of displaying the printed material in a watermarked manner over the screen and displaying a moving image accompanying the printed material on the screen. FIG. 14 is a plan view showing the printed matter and the display device, and shows a state in which a first image adjusted to be identifiable is displayed in a display area and the first image is superimposed on the printed matter. is there. FIG. 15 illustrates a method for adjusting the first image so that the first image can be discriminated when the printed material is displayed through the screen in a watermark, the first image is displayed in a display area, and the first image is superimposed on the printed material. FIG. FIG. 16 is a plan view showing the printed matter and the display device, and shows a state in which the first image whose position and orientation are adjusted is displayed in the display area. FIG. 17 is a flowchart for explaining a technique for adjusting the position and orientation of the first image when the printed material is displayed through the screen in a watermark and the first image is displayed in the display area. FIG. 18 is a plan view showing another printed matter and the display device, wherein the display device is placed on the printed matter so that one code of a plurality of codes of the printed matter can be detected, and the printed matter passes over the screen. It is a figure which shows the state currently displayed on the said screen, and displaying the 1st image accompanying the said printed matter on the said screen.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings. It should be noted that the disclosure is merely an example, and those skilled in the art can easily conceive of appropriate modifications while maintaining the gist of the invention are naturally included in the scope of the present invention. In addition, the drawings may be schematically represented with respect to the width, thickness, shape, and the like of each part in comparison with actual aspects for the sake of clarity of explanation, but are merely examples, and the interpretation of the present invention is not limited. It is not limited. In addition, in the present specification and each drawing, elements similar to those described above with reference to the previous drawings are denoted by the same reference numerals, and detailed description may be omitted as appropriate.

In this embodiment, a configuration of a display system including a display device, a printed matter with a code, and the like, and a method for using the display system will be described. Various display devices such as a liquid crystal display device and an organic electroluminescence (EL) display device can be applied to the display device. In the present embodiment, a display device using a polymer-dispersed liquid crystal (PDLC) as a display device will be described as an example.

FIG. 1 is a plan view showing a configuration example of the display system SY in the present embodiment.
As shown in FIG. 1, the display system SY includes a display device DS, a printed matter PR with a code, and a server SV. Various contents (data) are stored in the server SV. Examples of data stored in the server SV include content associated with the printed matter PR (for example, image data associated with the printed matter) and data in which the printed matter and the content are associated with each other.

The printed material PR is printed with an image such as a photograph and is provided with a code CD or the like. The printed matter PR can be detected by the display device DS, and specifically includes a code that can be detected by the display device DS. The display device DS includes a display panel PNL, a sensor SE, a light source unit LU, a storage unit ME, a speaker SP, a communication unit COM, and a control unit CON. The control unit CON controls the driving of the display panel PNL, the sensor SE, the light source unit LU, the storage unit ME, the speaker SP, and the communication unit COM.

The sensor SE electrically detects at least the code CD of the printed matter PR. The display panel PNL displays images (still images) and moving images. Further, the display panel PNL can be switched to a transparent state that further transmits the printed matter PR that has passed through the sensor SE. By switching the display panel PNL to the transparent state, the user can see the printed matter PR through the screen SC of the display device DS in a watermarked manner. The display panel PNL can switch at least a part of the display area DA or the entire display area DA to a transparent state.

Speaker SP outputs audible sound such as voice. The storage unit ME stores various types of content such as image data and moving image data to be displayed on the display panel PNL, audible sound data output from the speaker SP, and data in which printed matter and content are associated with each other. The communication unit COM can communicate with the server SV. When capturing data in the storage unit ME, the communication unit COM captures data from the server SV, and stores the captured data in the storage unit ME.

Based on the information of the code CD of the printed matter PR detected by the sensor SE, the control unit CON can select and execute the content attached to the printed matter.
For example, the control unit CON determines whether or not first image data for displaying a first image CH described later exists in the storage unit ME based on information detected by the sensor SE. When the first image data is present in the storage unit ME, the control unit CON displays the first image CH on the screen SC (display area) using the first image data stored in the storage unit ME. At that time, the control unit CON may display not only the first image but also a moving image associated with information including the first image and detected by the sensor SE on the screen SC (display area).

FIG. 2 is a perspective view showing the display device DS shown in FIG. As shown in FIGS. 1 and 2, the control unit CON can display the first image CH1 of the character string “ABC” attached to the printed matter on the screen SC (display area DA) of the display device DS. Or the control part CON can output the audible sound accompanying the printed matter from the speaker SP. And the control part CON can output the audible sound matched with the information detected by sensor SE from the speaker SP.

Alternatively, the control unit CON can output an audible sound from the speaker SP while displaying a moving image including a still image or a plurality of still images on the screen SC.
At that time, the control unit CON determines whether moving image data for displaying a moving image and audible sound data for outputting an audible sound are present in the storage unit ME based on information detected by the sensor SE. To do. When the moving image data and the audible sound data are present in the storage unit ME, the control unit CON displays the moving image in the display area DA using the moving image data stored in the storage unit ME, and stores it in the storage unit ME. The audible sound is output to the speaker SP using the audible sound data.

The display device DS further includes a bezel BE and a housing HS formed integrally with the bezel BE. The bezel BE accommodates the display panel PNL and the sensor SE, and exposes the screen SC and the detection surface SS opposite to the screen SC. The housing HS accommodates the light source unit LU, the storage unit ME, the speaker SP, the communication unit COM, and the control unit CON, and exposes the output unit of the speaker SP. For example, a battery (not shown) is also accommodated in the housing HS. The display device DS only needs to include the speaker SP as necessary.

FIG. 3 is a plan view showing a configuration example of a part of the display device DS. Here, a part of the configuration of the display device DS is exemplarily shown, and the configuration of the display device DS is not limited to the example shown in FIG.
As shown in FIG. 3, the first direction X and the second direction Y are directions that intersect each other, and the third direction Z is a direction that intersects the first direction X and the second direction Y. The first direction X corresponds to the row direction, and the second direction Y corresponds to the column direction. In one example, the first direction X, the second direction Y, and the third direction Z are orthogonal to each other, but may intersect each other at an angle other than 90 degrees.

The display device DS includes a display panel PNL and the like, and further includes wiring boards F1, F2, F4, F5 and the like. The display panel PNL includes a display area DA that displays an image and a frame-shaped non-display area NDA that surrounds the display area DA. The display area DA includes n gate lines G (G1 to Gn), m source lines S (S1 to Sm), and the like. Note that n and m are both positive integers, and n may be equal to m, or n may be different from m. The plurality of gate lines G each extend in the first direction X and are arranged at intervals in the second direction Y. In other words, the plurality of gate lines G extend in the row direction. The plurality of source lines S each extend in the second direction Y and are arranged at intervals in the first direction X.

The wiring board F1 includes a gate driver GD. A plurality of gate lines G are connected to the gate driver GD. The wiring board F2 includes a source driver SD. A plurality of source lines S are connected to the source driver SD. The wiring boards F1 and F2 are connected to the display panel PNL and the wiring board F4, respectively. The wiring board F5 includes a timing controller TC and the like.

The wiring board F4 is connected to the connector CT of the wiring board F5. The wiring boards F1 and F2 may be replaced with a single wiring board. Moreover, the wiring boards F1, F2, and F4 may be replaced with a single wiring board. The gate driver GD, the source driver SD, and the timing controller TC described above constitute a part of the control unit CON described above. The control unit CON is configured to control driving of the plurality of gate lines G, the plurality of source lines S, a plurality of pixel electrodes described later, a common electrode described later, and the like.

FIG. 4 is a cross-sectional view showing a part of the display device DS. Here, only the main part in the cross section of the display device DS in the YZ plane defined by the second direction Y and the third direction Z will be described. As shown in FIG. 4, the display panel PNL includes a transparent first substrate SUB1, a transparent second substrate SUB2, a liquid crystal layer 30 as a display function layer, and the like.
The first substrate SUB1 includes a transparent substrate 10, a pixel electrode 11, an alignment film 12, and the like. The second substrate SUB2 includes a transparent substrate 20, a common electrode 21, an alignment film 22, and the like. The pixel electrode 11 and the common electrode 21 are formed of a transparent conductive material such as indium tin oxide (ITO) or indium zinc oxide (IZO).

The liquid crystal layer 30 is located at least in the display area DA. The liquid crystal layer 30 includes polymer-dispersed liquid crystal and is located between the alignment film 12 and the alignment film 22. The liquid crystal layer 30 of the present embodiment uses reverse type polymer dispersed liquid crystal (R-PDLC: reverse-mode-polymer-dispersed-liquid-crystal). Unlike the present embodiment, the liquid crystal layer 30 may use normal type polymer dispersed liquid crystal. The liquid crystal layer 30 maintains the parallelism of incident light or scatters incident light according to the applied voltage. The first substrate SUB1 and the second substrate SUB2 are bonded by a sealing material 40.

The wiring boards F1 and F2 are connected to the extending portion EX of the first board SUB1.
The light source unit LU is located in the non-display area NDA outside the display area DA. The light source unit LU is located on the side surface of the display panel PNL. The light source unit LU includes a light emitting element LS, a wiring board F6, and the like. The light emitting element LS is connected to the wiring board F6 and is located on the extension part EX. The light emitting element LS has a light emitting part (light emitting surface) EM facing the end E5 of the transparent substrate 20. As will be described later, the illumination light emitted from the light emitting unit EM enters the end E5 and propagates through the display panel PNL. However, the display device DS may be configured without the wiring substrate F6. In this case, the driving of the light emitting element LS is controlled by the control unit CON via the wiring board F1 or the wiring board F2.
The sensor SE faces at least the display area DA of the display panel PNL. The sensor SE is connected to the wiring board F7, and is connected to the control unit CON described above via the wiring board F7.

FIG. 5 is a diagram illustrating some components of the display device DS.
As shown in FIG. 5, the display device DS (control unit CON) includes a controller CNT indicated by a broken line in the drawing. The controller CNT includes a timing controller TC, a gate driver GD, a source driver SD, a Vcom circuit VC, a light source driver LSD, and the like.

The timing controller TC generates various signals based on the image data read from the storage unit ME and the synchronization signal. In one example, the timing controller TC includes a correction unit COR that performs predetermined signal processing based on image data. Thus, the video signal corrected by the correction unit COR can be output to the source driver SD, and the control signal corrected by the correction unit COR can be output to the light source driver LSD. In one example, the correction unit COR corrects the gradation of the image displayed on the display panel PNL by performing gamma correction. The timing controller TC outputs a control signal generated based on the synchronization signal to the gate driver GD, the source driver SD, the Vcom circuit VC, and the light source driver LSD.

The display area DA indicated by a two-dot chain line in the figure includes a plurality of pixels PX. Each pixel PX includes a switching element SW and a pixel electrode 11. The switching element SW is formed of, for example, a thin film transistor. The switching element SW is electrically connected to the gate line G and the source line S. The plurality of pixel electrodes 11 are located in the display area DA and are provided in a matrix. For this reason, for example, the plurality of pixel electrodes 11 are provided in a plurality of rows. The pixel electrode 11 is connected to the source line S via the switching element SW. The common electrode 21 is located in the display area DA. The common electrode 21 faces the plurality of pixel electrodes 11. Unlike the present embodiment, the common electrode 21 may be divided for at least one pixel PX, connected to a common line, and applied with a common common voltage.

A gate signal is supplied to each gate line G from the gate driver GD. Each of the source lines S is supplied with a video signal (image signal) from the source driver SD. A common voltage Vcom is supplied to the common electrode 21 from the Vcom circuit VC. The video signal supplied to the source line S is applied to the pixel electrode 11 connected to the switching element SW during a period in which the switching element SW is turned on based on the gate signal supplied to the gate line G. . Thereby, the pixel electrode 11 and the common electrode 21 can be set to the same potential, or a potential difference can be generated between the pixel electrode 11 and the common electrode 21.

The light source unit LU is configured to irradiate the liquid crystal layer 30 with light. In the present embodiment, the light source unit LU includes a plurality of light emitting elements LS. For example, the light source unit LU includes a light emitting element (first light emitting element) LSR that irradiates the liquid crystal layer 30 with the first color light, and a light emitting element (second light emitting element) that irradiates the liquid crystal layer 30 with the second color light. LSG and the light emitting element (3rd light emitting element) LSB which irradiates the liquid crystal layer 30 with the light of the 3rd color are provided. For example, the first color is red, the second color is green, and the third color is blue. However, the light emitting element included in the light source unit LU may be only a light emitting element that emits white light. The light source driver LSD controls the lighting period of these light emitting elements LSR, LSG, and LSB.

Next, a method for switching the liquid crystal layer 30 to at least one of a transparent state and a scattering state by the control unit CON will be described.
FIG. 6A is a cross-sectional view showing the display panel PNL when the liquid crystal layer 30 is in a transparent state. In the transparent state, the liquid crystal layer 30 hardly scatters incident light, so that high transparency can be obtained. For example, when the liquid crystal layer 30 is in a transparent state, the printed matter PR can be seen through the screen SC of the display device DS.

As shown in FIG. 6A, the illumination light L11 emitted from the light emitting element LS enters the display panel PNL from the end E5 and propagates through the transparent substrate 20, the liquid crystal layer 30, the transparent substrate 10, and the like. When the liquid crystal layer 30 is in a transparent state, the illumination light L11 is hardly scattered by the liquid crystal layer 30, and therefore hardly leaks from the lower surface 10B of the transparent substrate 10 and the upper surface 20T of the transparent substrate 20.

The external light L12 incident on the display panel PNL is transmitted through the liquid crystal layer 30 with almost no scattering. That is, the external light incident on the display panel PNL from the lower surface 10B is transmitted to the upper surface 20T, and the external light incident from the upper surface 20T is transmitted to the lower surface 10B. For this reason, when the display panel PNL is viewed from the upper surface 20T side, the user can visually recognize the lower surface 10B side (back side) through the display panel PNL.

FIG. 6B is a cross-sectional view showing the display panel PNL when the liquid crystal layer 30 is in a scattering state. In the scattering state, the liquid crystal layer 30 scatters incident light. For example, an image can be displayed on the screen SC of the display device DS by setting the liquid crystal layer 30 in a scattering state.

As shown in FIG. 6B, the illumination light L21 emitted from the light emitting element LS enters the display panel PNL from the end E5 and propagates through the transparent substrate 20, the liquid crystal layer 30, the transparent substrate 10, and the like. In the illustrated example, the liquid crystal layer 30 between the pixel electrode 11A and the common electrode 21 (a liquid crystal layer to which a voltage applied between the pixel electrode 11A and the common electrode 21 is applied) is in a transparent state. The light L21 is hardly scattered in a region of the liquid crystal layer 30 that faces the pixel electrode 11A. On the other hand, since the liquid crystal layer 30 between the pixel electrode 11B and the common electrode 21 (the liquid crystal layer to which a voltage applied between the pixel electrode 11B and the common electrode 21 is applied) is in a scattering state, the illumination light L21 is The liquid crystal layer 30 is scattered in a region facing the pixel electrode 11B. Among the illumination light L21, a part of the scattered light L211 is emitted to the outside from the upper surface 20T, and a part of the scattered light L212 is emitted to the outside from the lower surface 10B.

In the position overlapping with the pixel electrode 11A, the external light L22 incident on the display panel PNL is transmitted almost without being scattered by the liquid crystal layer 30 like the external light L12 shown in FIG. 6A. At a position overlapping the pixel electrode 11B, the external light L23 incident from the lower surface 10B is transmitted from the upper surface 20T after a part of the light L231 is scattered by the liquid crystal layer 30. Further, the external light L24 incident from the upper surface 20T is transmitted from the lower surface 10B after a part of the light L241 is scattered by the liquid crystal layer 30.

Therefore, when the display panel PNL is viewed from the upper surface 20T side, the color of the illumination light L21 can be visually recognized at a position overlapping the pixel electrode 11B. In addition, since some of the external light L231 passes through the display panel PNL, the lower surface 10B side (for example, the printed matter PR) can also be visually recognized through the display panel PNL. Note that when the display panel PNL is viewed from the lower surface 10B side, the color of the illumination light L21 can be visually recognized at a position overlapping the pixel electrode 11B. In addition, some external light L241 passes through the display panel PNL.

FIG. 7 is a plan view showing the configuration of the sensor SE of FIGS. 1 and 4. The sensor SE may be any sensor that electrically detects the printed matter PR, and is, for example, a capacitive sensor. As shown in FIG. 7, the sensor SE includes a transparent third substrate SUB3, a plurality of detection electrodes Sx, and a plurality of wirings LN. The plurality of detection electrodes Sx and the plurality of wirings LN are provided on the third substrate SUB3. In the present embodiment, a group of the detection electrode Sx and the wiring LN is located between the third substrate SUB3 and the display panel PNL. The plurality of pixel electrodes 11 and the common electrode 21 are located between the group of the plurality of detection electrodes Sx and the plurality of wirings LN and the screen SC.
However, unlike the present embodiment, the third substrate SUB3 may be located between the group of the detection electrode Sx and the wiring LN and the display panel PNL. Alternatively, the sensor SE may be configured without the third substrate SUB3, and the group of the detection electrode Sx and the wiring LN may be provided on the display panel PNL such as the lower surface 10B.

The plurality of detection electrodes Sx are provided in a matrix in the first direction X and the second direction Y independently of each other. In this embodiment, each detection electrode Sx is formed of a transparent electrode RE. The transparent electrode RE is formed of a transparent conductive material such as ITO or IZO. In this embodiment, the sizes of the plurality of transparent electrodes RE are uniform. The detection electrode Sx detects a change in capacitance. However, the detection electrode Sx may be formed of a plurality of detection lines that are formed of metal and are thin lines. Alternatively, the detection electrode Sx may be formed of an assembly of the transparent electrode RE and a plurality of detection lines.
The wiring LN is connected to the detection electrode Sx on a one-to-one basis. The wiring LN is made of metal. The plurality of wirings LN are spaced from each other and are electrically insulated from each other. Although not shown, an insulating layer is provided on the detection electrode Sx and the wiring LN.

In a region not facing the display region DA of the display panel PNL, a pad group (OLB pad group) PG of outer lead bonding (Outer 表示 Lead Bonding) is formed on the third substrate SUB3. Wires LN are connected one-to-one to the pads of the OLB pad group PG. The wiring board F7 is connected to the pads of the OLB pad group PG. The driving of the plurality of detection electrodes Sx is controlled by the control unit CON.

Next, the printed matter PR with code will be described. FIG. 8 is a plan view showing the printed matter PR with the code in FIG. FIG. 9 is a cross-sectional view showing the printed matter PR along the line IX-IX in FIG.
As shown in FIGS. 8 and 9, the printed material PR includes a substrate 1, a second image 2, a first layer 3, and a second layer 4. The substrate 1 is a printable medium, and is formed of a material capable of printing the second image 2 and providing the first layer 3 and the second layer 4. The substrate 1 has a main surface 1A. The second image 2 is printed on the main surface 1A. Therefore, the second image 2 is displayed on the main surface 1A.

The first layer 3 is provided on the main surface 1A on which the second image 2 is printed. A second layer 4 is provided on the main surface 1 </ b> A and the first layer 3. The second layer 4 is formed of a transparent insulating resin, for example, as a transparent insulating material. By providing the printed material PR with the second layer 4, for example, the first layer 3 can be protected.

In the present embodiment, the first layer 3 is provided on the entire main surface 1A. By patterning the first layer 3, the first layer 3 includes one or more code CDs including the first code. In the present embodiment, the first layer 3 includes four codes CD including the first to fourth codes. The four code CDs are located at a distance from each other. The first layer 3 only needs to be able to form the code CD, and may not be provided on the entire main surface 1A.

The code CD has information on the second image 2. The information regarding the second image 2 includes information indicating the position and orientation of the second image 2. Further, the information regarding the second image 2 may include information indicating the size of the second image 2. Even if there are a plurality of types of printed matter having different sizes of the second image 2, information indicating the size of the second image 2 can be detected by the sensor SE. In that case, the size of the printed matter PR itself may be different.

The code CD is transparent. The code CD can be detected by the sensor SE. In the present embodiment, the code CD has conductivity. The code CD is formed of, for example, a transparent conductive resin as a transparent conductive material. In the present embodiment, the code CD has a three-dimensional pattern. As shown in FIG. 9, since the code CD has an uneven surface, more information is stored in the code CD than when the code CD has a two-dimensional pattern that does not have an uneven surface. You can have it. In addition, although different from the code CD of the present embodiment, the code CD may have a two-dimensional pattern.
Alternatively, the code CD may be formed of a dielectric and have a three-dimensional pattern. In this case, the dielectric constant of the code CD (first layer 3) is different from the dielectric constant of the second layer 4.

Next, a method for detecting the printed matter PR (code CD) by the sensor SE will be described. FIG. 10 is a cross-sectional view illustrating a part of the printed material PR and the display device DS, and is a diagram illustrating a state in which the display device DS is placed on the printed material PR. In the figure, the sensor SE is bonded to the display panel PNL by the adhesive layer AD, but the present invention is not limited to this, and an air layer may be interposed between the sensor SE and the display panel PNL.

As shown in FIG. 10, first, the display device DS is placed on the printed matter PR. Thereafter, the control unit CON writes a write signal (sensor drive signal) to the detection electrode Sx, and generates a sensor signal between the detection electrode Sx and the printed material PR. Next, the control unit CON reads a read signal indicating a change in a sensor signal (for example, capacitance generated in the detection electrode Sx) from the detection electrode Sx. The sensor signal changes according to the distance between the detection electrode Sx and the first layer 3. The control unit CON can detect the pattern information of the code CD (first layer 3) by reading the read signal. For example, when the code SE of the printed matter PR is detected by the sensor SE, the display device DS is in contact with the main surface 1A side of the printed matter PR.

Here, a method has been described in which the sensor SE includes the detection electrode Sx and detects the code CD in the self-capacitance mode. The presence or absence of an external proximity object can be detected by driving the detection electrode Sx and reading the change in capacitance generated in the detection electrode Sx by the control unit CON. However, the configuration of the sensor SE and the detection mode of the sensor SE are not limited to the present embodiment, and various modifications can be made. For example, a method may be adopted in which the sensor SE includes a plurality of drive electrodes Tx and a plurality of detection electrodes Rx instead of the detection electrode Sx, and detects the code CD in a mutual-capacitive (Sensing) mode. For example, the drive electrode Tx and the detection electrode Rx intersect each other. The controller CON drives the drive electrode Tx, and can read a change in electrostatic capacitance generated between the drive electrode Tx and the detection electrode Rx from the detection electrode Rx.

Next, a method for displaying the first images CH1 and CH2 associated with the printed matter PR in the display area DA of the display device DS while seeing the printed matter PR with code through the screen SC will be described. FIG. 11 is a plan view showing the printed material PR and the display device DS. The printed material PR is displayed through the screen SC, and the first images CH1 and CH2 associated with the printed material PR are displayed on the screen SC. FIG.

As shown in FIGS. 1 and 11, the display device DS is placed on the printed matter PR. Under the control of the control unit CON, the display panel PNL can make the display area DA transparent. For example, the display panel PNL makes the entire display area DA transparent. In this example, the entire printed matter PR is located within the display area DA, all the codes CD of the printed matter PR are opposed to the plurality of detection electrodes Sx, and the sensor SE electrically electrically connects all the codes CD of the printed matter PR. Can be detected. Information regarding the second image 2 can be acquired by detecting a plurality of code CDs by the sensor SE. The control unit CON searches the storage unit ME based on the information of the code CD detected by the sensor SE, and selects the data of the first images CH1 and CH2 as the contents attached to the second image 2.

Under the control of the control unit CON, the display panel PNL displays the first images CH1 and CH2 in the display area DA using the data of the first images CH1 and CH2 stored in the storage unit ME. The position where the first images CH1 and CH2 are displayed is determined in advance in the display area DA according to the type of the first image. The display panel PNL can display the first images CH1 and CH2 in the display area DA by partially switching the display area DA to the scattering state. Here, the first image CH1 is displayed at a position shifted from the second image 2 (outside the second image 2) in the area superimposed on the printed matter PR in the display area DA. The first image CH <b> 2 is a character string “123” and is displayed at a position to be superimposed on the second image 2. From the above, the control unit CON can display the first images CH1 and CH2 associated with the information detected by the sensor SE in the display area DA.

Next, a method for associating the printed material PR with the content attached to the printed material PR will be described. Here, a method for associating a photograph with a moving image will be described as an example. However, the method described here can be applied to a method of associating a photograph with a still image by replacing the moving image with a still image. FIG. 12 is a flowchart for explaining a method of taking a photograph and a moving image and storing information in which the photograph and the moving image are associated with each other in the storage unit ME of the display device DS.

As shown in FIG. 12, FIG. 1, and FIG. 8, when a method for associating a photograph with a moving image is started, first, in step S1a, the user takes a photograph. Next, the process proceeds to step S2a, and the user obtains a printed matter PR on which a photograph is printed and a code CD is provided. A photograph is displayed as the second image 2 on the printed matter PR.

Subsequently, in step S3a, the user shoots a moving image. At that time, recording may be performed. Thereafter, the process proceeds to step S4a, and the moving image data is transferred to the storage unit ME of the display device DS. When recording is performed, the recorded audible sound data is also transferred to the storage unit ME. Note that step S3a may be performed in parallel with step S1a, or may be performed prior to step S1a. Thereafter, in step S5a, the information on the photograph and the moving image is associated with each other, and the information on the association is stored in the storage unit ME. Thus, the method for associating the photograph with the moving image ends.
Note that the transfer destination of the moving image data and the audible sound data may be stored in the server SV instead of being stored in the storage unit ME.
Further, the process of associating a photograph with a moving image may be performed by a processing device (such as a personal computer) other than the display device DS. In this case, the photo data and the moving image data may be temporarily transferred to the processing device, and the information relating the photo and the moving image processed by the processing device may be transferred to the display device DS or the server SV.

Next, a method for displaying a moving image on the screen SC while displaying the printed matter PR through the screen SC in a watermarked manner will be described. FIG. 13 is a flowchart for explaining a technique for displaying the printed material PR through the screen SC in a watermarked manner and displaying a moving image accompanying the printed material PR on the screen SC.

As shown in FIGS. 13, 1, and 8, when a method of displaying a moving image on the screen SC while displaying the printed matter PR through the screen SC in a watermarked manner, first, in step S <b> 1 b, the user The display device DS is placed on the printed matter PR. Since the display panel PNL makes the entire display area DA transparent, the display device DS can display the printed matter PR through the screen SC to the user.

Thereafter, the process proceeds to step S2b. In step S2b, the code SE of the printed matter PR is detected by the sensor SE of the display device DS, and information about the printed matter PR is acquired from the detected code information. Next, in step S3b, the control unit CON determines whether moving image data (content) associated with the photograph (second image 2) of the printed matter PR exists in the storage unit ME. When moving image data (content) exists, the process proceeds to step S5b. If there is no moving image data (content), the process proceeds to step S4b, the display device DS is connected to the server SV, and the moving image data (content) is loaded from the server SV into the storage unit ME.

Next, in step S5b, the control unit CON selects moving image data associated with the photograph (second image 2) from the storage unit ME. In step S6b, the control unit CON determines whether the selected moving image is set to be automatically reproduced. If it is set to automatically play a moving image, the process proceeds to step S8b. When it is not set to automatically reproduce the moving image, the process proceeds to step S7b, and in step S7b, the control unit CON determines whether or not the display device DS is operated by the user to reproduce the moving image. When the above operation is performed, the process proceeds to step S8b. When the above operation is not performed, the control unit CON repeats the determination in step S7b.

And if it transfers to step S8b, the display apparatus DS will display a moving image on the display area DA. At that time, the moving image is displayed so as to be superimposed on the photograph (second image 2) or displayed in the margin of the photograph. The position where the moving image is displayed is set in advance to a position where the moving image is easy to see according to the photograph (second image 2), or a position where the information of the photograph and the moving image is easily associated. Thus, the method of displaying the moving image on the screen SC while displaying the printed matter PR through the screen SC is completed. When the data (content) attached to the photo includes not only moving image data but also audible sound data, the control unit CON outputs the audible sound from the speaker SP while displaying the moving image on the display area DA. Can be made.

From the above, for example, the control unit CON does not display only a still image such as the first image CH in the display area DA, but is a moving image including a still image and based on the information of the code CD. A moving image accompanying the image 2 can be displayed in the display area DA. Moreover, the control part CON can output the audible sound accompanying the 2nd image 2 from the speaker SP based on the information of code | cord | chord CD.

Next, a method related to the method described with reference to FIG. 13 will be described. For example, the method of displaying a moving image shown in FIG. 13 by replacing a moving image with a still image can be applied to a method of displaying a still image.
As shown in FIGS. 1 and 11, the control unit CON has the first image data for displaying the first images CH1 and CH2 (still images) based on the information of the code CD in the storage unit ME. You may decide whether or not. When the first image data does not exist in the storage unit ME, the control unit CON captures the first image data from the server SV via the communication unit COM and stores the first image data in the storage unit ME. The first images CH1 and CH2 can be displayed on the display area DA using the data.

Next, a method that can display the first image CH2 adjusted so as to be identifiable in the display area DA even when the first images CH1 and CH2 are superimposed on the printed matter PR will be described. FIG. 14 is a plan view showing the printed matter PR and the display device DS, in which the first image CH2 adjusted to be identifiable is displayed in the display area DA, and the first images CH1 and CH2 are superimposed on the printed matter PR. FIG.

As shown in FIG. 14, when displaying the first images CH1 and CH2 in the display area DA, the control unit CON controls the first image CH2 that is identifiable and adjusted based on the information of the first image CH1 and the code CD. Are displayed in the display area DA, and the first images CH1 and CH2 are superimposed on the printed matter PR. Note that the control unit CON (correction unit COR) adjusts (for example, gamma correction) so as to increase the luminance level with respect to the first image CH2, for example, but does not adjust the first image CH1.
As described above, the display device DS can make it easier for the user to identify the first image CH by appropriately adjusting the first image CH displayed in the display area DA of the display device DS so that the first image CH can be identified. . For example, if there is no inconvenience even if the color of the first image CH is changed, the color of the first image CH displayed in the display area DA is adjusted in consideration of the color of the second image 2, and the first image CH May be easily identified.

Next, a method for making the user easily identify the first image CH will be described. FIG. 15 shows a method of adjusting the first image CH so as to be identifiable when the printed material PR is displayed through the screen in a watermarked manner, the first image CH is displayed in the display area DA, and the first image CH is superimposed on the printed material PR. It is a flowchart for demonstrating.

As shown in FIGS. 15, 1, and 14, when the method for adjusting the first image CH in an identifiable manner is started, first, in step S <b> 1 c, the user places the display device DS on the printed material PR. . Since the display panel PNL makes the entire display area DA transparent, the display device DS can display the printed matter PR through the screen SC to the user.

Thereafter, in step S2c, the code SE of the printed matter PR is detected by the sensor SE of the display device DS, and information related to the printed matter PR is acquired from the detected code information. Here, the code CD includes information related to the brightness of the second image 2. Thereby, the display device DS can acquire the brightness information of the second image 2. Note that the display device DS may acquire information on the brightness of the margin of the printed matter PR based on the detected code information. Accordingly, the first image CH1 displayed so as to be superimposed on the margin of the printed matter PR can be appropriately adjusted so as to be identifiable.
Further, the code CD may include information on the brightness and color of each place of the printed matter PR. In this case, the brightness level and the color of the first image CH1 can be changed as appropriate according to the display position.

Next, in step S3c, the control unit CON determines whether the brightness of the second image 2 of the printed matter PR is within the standard range based on the information of the code CD. When the brightness of the second image 2 is within the standard range, in step S4c, the control unit CON keeps the standard value without adjusting the contrast value of the first image CH2, and the first image CH2 The luminance level is not adjusted and the standard level is maintained, and the process proceeds to step S8c.

When the brightness of the second image 2 is within the standard range (S3c, NO), in step S5c, the control unit CON determines whether the brightness of the second image 2 of the printed matter PR is brighter than the standard range. . When the brightness of the second image 2 is brighter than the standard range, in step S6c, the contrast value of the first image CH2 is set higher than the standard value, and the process proceeds to step S8c. Since the luminance level of the first image CH2 to be superimposed on the second image 2 is improved as a whole, the contrast value of the first image CH2 can be increased by performing a correction that lowers the gamma value.

When the brightness of the second image 2 is darker than the standard range (S5c, NO), in step S7c, the control unit CON makes the luminance level of the first image C2 higher than the standard level, and proceeds to step S8c. Since the contrast value of the first image CH2 superimposed on the second image 2 is improved as a whole, the contrast value of the first image CH2 can be increased by performing correction to increase the gamma value.

Thereby, in step S8c, the control unit CON can display the identifiable first image CH on the display area DA of the display device DS, and can superimpose the first image CH on the printed matter PR. The method of adjusting to be discernable ends. For example, the control unit CON can display the first image CH2 that can be identified by the adjustment in the display area DA, and can superimpose the first image CH2 on the second image 2. Further, the control unit CON can display the identifiable first image CH1 in the display area DA without adjustment, and can superimpose the first image CH2 on the margin of the printed matter PR.

Next, the display device DS that displays the first images CH1 and CH2 whose positions and orientations are adjusted in the display area DA will be described. FIG. 16 is a plan view showing the printed matter PR and the display device DS, and shows a state in which the first images CH1 and CH2 whose positions and orientations are adjusted are displayed on the display area DA.

As shown in FIGS. 16 and 1, there may be a case where the display device DS overlaps the printed matter PR. The display device DS is placed on the printed material PR. The display panel PNL, for example, makes the entire display area DA transparent. In this example, a part of the printed matter PR is located in the display area DA, the entire two codes CD of the printed matter PR face the plurality of detection electrodes Sx, and the sensor SE electrically connects the two codes CD of the printed matter PR. Can be detected. The code CD includes information regarding the position and orientation of the second image 2. The controller CON determines the position and orientation of the second image 2 based on the information on the code CD detected by the sensor SE. For example, the control unit CON can determine the position and orientation of the second image 2 based on the position of the code CD detected by the sensor SE and the degree of inclination of the code CD. Then, the control unit CON causes the display area DA to display the first images CH1 and CH2 whose positions and orientations are adjusted so as to correspond to the positions and orientations of the second images 2.

As described above, since the first images CH1 and CH2 can be displayed so as to follow the second image 2 of the printed matter PR, for example, the first images CH1 and CH2 are further associated with the second image 2. It can be displayed. Or it can make it easy for a user to see both the 2nd image 2 and 1st image CH1, CH2 simultaneously.

Next, a method for adjusting the position and orientation of the first image CH will be described. FIG. 17 is a flowchart for explaining a technique for adjusting the position and orientation of the first image CH when displaying the printed matter PR through the screen SC in a watermarked manner and displaying the first image CH in the display area DA. .
As shown in FIGS. 17, 1 and 16, when starting the adjustment of the position and orientation of the first image CH, first, in step S1d, the user places the display device DS on the printed matter PR. Since the display panel PNL makes the entire display area DA transparent, the display device DS can display the printed matter PR through the screen SC to the user.

Thereafter, in step S2d, the code SE of the printed matter PR is detected by the sensor SE of the display device DS, and information about the printed matter PR is acquired from the detected code information. Here, the code CD includes information on the position and orientation of the second image 2 (photograph). Thereby, based on the information on the direction of the code CD and the location where the code CD is detected, the control unit CON can acquire the information on the position and orientation of the second image 2.

Next, when the process proceeds to step S3d, the control unit CON determines the position and orientation of the second image 2 (photograph) of the printed matter PR based on the information of the code CD detected by the sensor SE. In step S4d, the control unit CON causes the display area DA to display the first images CH1 and CH2 whose positions and orientations have been adjusted so as to correspond to the positions and orientations of the second images 2 (photographs) of the printed matter PR. Thereby, the adjustment of the position and orientation of the first image CH is completed.

Next, a case where the display system SY includes a plurality of printed materials PR having different sizes of the second image 2 will be described. FIG. 18 is a plan view showing the printed matter PR with code and the display device DS. The display device DS is placed on the printed matter PR so that one code CD of the plurality of code CDs of the printed matter PR can be detected. Is a view showing a state in which the first image CH1 and CH2 attached to the printed matter PR are displayed on the screen SC through the screen SC. In the printed matter PR shown in FIG. 18, the size of the second image 2 is larger than the size of the second image 2 shown in FIG. 11, and the size of the printed matter PR itself is larger than the size of the printed matter PR shown in FIG. .

18 and 1, for example, the size of the second image 2 is larger than the size of the display area DA. When the display device DS processes a plurality of types of printed matter PR having different sizes of the second image 2, the information regarding the size of the second image 2 is weight. Therefore, the code CD shown in FIG. 18 further includes information indicating the size of the second image 2 in addition to the information indicating the position and orientation of the second image 2.

The display device DS is placed on the printed material PR. Under the control of the control unit CON, the display panel PNL, for example, makes the entire display area DA transparent. In this example, a part of the printed matter PR is located in the display area DA, one whole code CD of the printed matter PR faces the plurality of detection electrodes Sx, and the sensor SE electrically connects one code CD of the printed matter PR. Can be detected. The control unit CON can display the first images CH1 and CH2 associated with the second image 2 on the display area DA based on the information of at least one code CD of the printed matter PR detected by the sensor SE. .

By detecting one code CD by the sensor SE, the control unit CON can acquire information on the position, orientation, and size of the second image 2. Further, by detecting one code CD, the control unit CON can also determine which part of the printed matter PR is superimposed on the display area DA. Then, the control unit CON causes the display area DA to display the first images CH1 and CH2 whose positions and orientations are adjusted so as to correspond to the position, orientation, and size of the second image 2 of the printed matter PR. As described above, the display device DS may adjust the position and orientation of the first images CH1 and CH2 in consideration of the position, orientation, and size of the second image 2.

According to the display system SY configured as described above and the method of using the display system SY, the display area DA of the display device DS can be switched to a transparent state. Since the second image 2 itself of the printed code PR with code can be seen through the screen SC, the value of the printed material PR is impaired as compared with the case where the image obtained by photographing the second image 2 is displayed on the screen SC. Can be difficult.

Since the display device DS can display the first image CH on the screen SC while transmitting the second image 2 on the screen SC side, it can provide various information to the user.
Further, the code CD of the printed matter PR is transparent. Therefore, when the color of the code CD is black, the quality of the printed material PR itself can be made less likely to be impaired as compared with the printed material PR having the code CD that can be visually recognized.

From the above, the display device DS that can see the printed matter PR through the screen SC through the watermark and can display the second image 2 attached to the printed matter PR on the screen SC by electrically detecting the printed matter PR, A display system SY including the display device DS and a printed matter PR with a code that can be detected by the display device DS can be obtained.

Although the embodiment of the present invention has been described, the above embodiment is presented as an example, and is not intended to limit the scope of the invention. The above-described novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the spirit of the invention. The above embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

For example, the printed matter PR with code need only include at least one code CD. When the printed matter PR with code includes a plurality of code CDs, at least one code CD may have information on the position and orientation of the second image 2. Alternatively, when the printed matter PR with code includes a plurality of code CDs, information regarding the position and orientation of the second image 2 may be distributed among the plurality of code CDs.

Claims

A display panel having a plurality of pixel electrodes located in the display area, a common electrode located in the display area, and a display functional layer located in the display area;
A sensor having at least a plurality of detection electrodes arranged in the display area;
A control unit that controls driving of each of the plurality of pixel electrodes, the common electrode, and the plurality of detection electrodes;
The controller is
At least a part of the display area of the display panel can be in a transparent state;
Displaying a first image associated with information detected by the sensor in the display area;
Display device.
A storage unit;
The controller is
Determining whether first image data for displaying the first image is present in the storage unit based on information detected by the sensor;
When the first image data is present in the storage unit, the first image is displayed in the display area using the first image data stored in the storage unit;
The display device according to claim 1.
A light source unit that is positioned on a side surface of the display panel, the driving of which is controlled by the control unit, and that irradiates light to the display functional layer;
When displaying the first image in the display area, the control unit, based on information detected by the sensor,
Displaying the first image adjusted to be identifiable in the display area;
The display device according to claim 1.
When displaying the first image in the display area, the control unit includes:
Based on information detected by the sensor, the first image whose position and orientation are adjusted is displayed in the display area.
The display device according to claim 1.
The display panel has a screen for displaying the first image;
The plurality of pixel electrodes and the common electrode are located between the plurality of detection electrode groups and the screen.
The display device according to claim 1.
When displaying the first image in the display area, the control unit includes:
Displaying the moving image associated with the information detected by the sensor including the first image in the display area;
The display device according to claim 1.
A speaker whose driving is controlled by the control unit;
The controller is
Outputting an audible sound associated with information detected by the sensor from the speaker;
The display device according to claim 1 or 6.
A speaker whose driving is controlled by the control unit;
A storage unit;
The controller is
Determining whether moving image data for displaying the moving image and audible sound data for outputting audible sound are present in the storage unit based on information detected by the sensor;
When the moving image data and the audible sound data are present in the storage unit, the moving image is displayed in the display area using the moving image data stored in the storage unit and stored in the storage unit. The audible sound is output to the speaker using the audible sound data.
The display device according to claim 6.
A display panel having a plurality of pixel electrodes located in the display area, a common electrode located in the display area, and a display functional layer located in the display area, and a sensor having at least a plurality of detection electrodes arranged in the display area And a control unit that controls driving of each of the plurality of pixel electrodes, the common electrode, and the plurality of detection electrodes, and a display device comprising:
A printed material having a main surface; a second image displayed on the main surface; and a transparent first code located on the main surface and having information on the second image and detectable by the sensor. A corded printed matter having,
The controller is
At least a part of the display area of the display panel can be in a transparent state;
Based on the information of the first code of the printed matter detected by the sensor, the first image accompanying the second image is displayed in the display area.
Display system.
The display device further includes a storage unit,
The controller is
Determining whether first image data for displaying the first image is present in the storage unit based on the information of the first code;
When the first image data is present in the storage unit, the first image is displayed in the display area using the first image data stored in the storage unit;
The display system according to claim 9.
A server storing data;
The display device further includes a storage unit, and a communication unit whose driving is controlled by the control unit and capable of communicating with the server,
The controller is
Determining whether first image data for displaying the first image is present in the storage unit based on the information of the first code;
When the first image data does not exist in the storage unit, the first image data is taken into the storage unit from the server via the communication unit, and the first image data stored in the storage unit is stored in the storage unit. Use to display the first image in the display area;
The display system according to claim 9.
When displaying the first image in the display area, the control unit includes:
Displaying the first image superimposed on the second image;
The display system according to claim 9.
The display device further includes a light source unit that is located on a side surface of the display panel and whose driving is controlled by the control unit and that irradiates light to the display functional layer
When displaying the first image in the display area, the control unit, based on the information of the first code,
Displaying the first image adjusted to be identifiable in the display area, and superimposing the first image on the printed matter;
The display system according to claim 12.
The first code includes information on the position and orientation of the second image,
When displaying the first image in the display area, the control unit includes:
The first image in which the position and orientation of the second image are determined based on the information of the first code, and the position and orientation are adjusted to correspond to the position and orientation of the second image. Is displayed in the display area,
The display system according to claim 9.
When displaying the first image in the display area, the control unit includes:
Displaying not only the first image but also the moving image accompanying the second image based on the information of the first code including the first image in the display area;
The display system according to claim 9.
The display device further includes a speaker whose drive is controlled by the control unit,
The controller is
Based on the information of the first code, an audible sound accompanying the second image is output from the speaker.
The display system according to claim 9 or 15.
The display device
A speaker whose driving is controlled by the control unit;
A storage unit for storing data;
The controller is
Determining whether moving image data for displaying the moving image and audible sound data for outputting audible sound are present in the storage unit based on the information of the first code;
When the moving image data and the audible sound data are present in the storage unit, the moving image is displayed in the display area using the moving image data stored in the storage unit and stored in the storage unit. The audible sound is output to the speaker using the audible sound data.
The display system according to claim 15.
The printed matter further includes a transparent second code that is located on the main surface, is spaced apart from the first code, has information about the second image, and is detectable by the sensor;
The controller is
Based on information on at least one of the first code and the second code of the printed matter detected by the sensor, a first image accompanying the second image is displayed in the display area.
The display system according to claim 9.
When the first code of the printed matter is detected by the sensor, the display device is in contact with the main surface side of the printed matter.
The display system according to claim 9.
A substrate having a main surface;
A second image displayed on the main surface;
A transparent first code which is located on the main surface and has information on the second image and has conductivity.
Coded printed material.
The first code has a three-dimensional pattern;
The printed matter with code according to claim 20.
The first cord is made of a transparent conductive resin,
The printed matter with code according to claim 20.
A substrate having a main surface;
A second image displayed on the main surface;
A transparent first code that is located on the main surface and has information about the second image, is formed of a dielectric, and has a three-dimensional pattern;
Coded printed material.