WO2010103888A1

WO2010103888A1 - Content display device, content display method, and computer-readable recording medium on which content display program is recorded

Info

Publication number: WO2010103888A1
Application number: PCT/JP2010/052059
Authority: WO
Inventors: 一浩三木; 雅也中村; 真一北林; 浩福富
Original assignee: シャープ株式会社
Priority date: 2009-03-10
Filing date: 2010-02-12
Publication date: 2010-09-16
Also published as: JP2010211505A

Abstract

A content display device (100) is provided with first and second display areas (140, 240), a memory (171) for storing a content and an image group, a reception unit (1101) for receiving first and second commands, a first display control unit (1103) for displaying an area to be displayed of the content via the first display area, advancing the area to be displayed of the content when the first command is received, and returning the area to be displayed of the content when the second command is received, and a second display control unit (1110) for outputting a first effect via the second display area on the basis of the image group when the first command is received, and outputting a second effect via the second display area on the basis of the image group when the second command is received.

Description

Content display device, content display method, and computer-readable recording medium for recording content display program

The present invention relates to a content display device that displays content such as text and images, a content display method, and a computer-readable recording medium that records a content display program, and in particular, intuitively conveys the progress of content to a user. The present invention relates to a content display device capable of performing the above, a content display method, and a computer-readable recording medium for recording a content display program.

Electronic devices (content display devices) that can display (reproduce) text, images, and the like are known. Some such content display devices have a display panel and an operation panel. More specifically, the content display device displays text and images to the user via the display panel, and accepts various commands from the user via an operation panel equipped with a tablet function and the like. Examples of such content display devices include notebook personal computers and mobile phones.

For example, Japanese Patent Laid-Open No. 2000-163193 (Patent Document 1) discloses an electronic book that can be realized by a user-friendly human interface for operation input such as page turning. According to Japanese Unexamined Patent Publication No. 2000-163193 (Patent Document 1), an electronic book can output information to which a page number is assigned as image information on a page basis to a display unit. The electronic book has a finger input detection unit that detects the contact pressure, contact area, and contact position of the finger that has contacted the display unit, and the page based on the detected finger contact pressure, contact area, contact position, and finger movement direction. Finger input determination unit that determines turning input, continuous page turning input, bookmark insertion, bookmark insertion location reference input, etc., page image update control unit that performs page image update control, and bookmark processing that performs bookmark insertion and reference processing And an image generation unit that generates an image to be displayed on the display unit.

JP 2000-163193 A

Since the content display device realizes a virtual book, a virtual album, or the like, the user intuitively recognizes the processing performed by the content display device and the operation to be performed by the user on the content display device. Preferably it can be done. For example, the content display device can make the user recognize that the book has advanced to the next page by displaying a moving image in which the page of the book being browsed is turned. However, the content display apparatus needs to perform complicated image processing in order to display a moving image that advances the book being browsed to the next page. That is, there is a problem that the content display device needs to perform many calculations in order to make the user intuitively recognize the progress state of the content.

The present invention has been made to solve such a problem, and an object of the present invention is to provide a content display device, a content display method, and a content that allow a user to intuitively recognize the progress state of the content through simple processing. A computer-readable recording medium for recording a display program is provided.

According to an aspect of the present invention, a content display device is provided. The content display device includes a first and second display area, a memory for storing at least one content and at least one image group, and a reception unit for receiving one of the first and second instructions. The display target area of the content is displayed via the first display area, the display target area of the content is advanced when the first command is received, and the display target area of the content is received when the second command is received A first display control unit for returning the first effect, and when the first command is received, the first effect is output via the second display area based on at least one image group, and the second command is output And a second display control unit for outputting the second effect via the second display area based on at least one image group when received.

Preferably, when the second display control unit receives the first command, the second display control unit outputs the first effect by causing the second display area to display at least one image group in the first order. When the second command is received, the second effect is output by displaying any one of the at least one image group in the second display area in the second order opposite to the first order. .

Preferably, the at least one image group includes a first image group and a second image group. When receiving the first command, the second display control unit outputs the first effect by sequentially displaying the first image group in the second display area, and receives the second command. Sometimes, the second effect is output by sequentially displaying the second image group in the second display area.

Preferably, the content display device further includes a touch panel including a second display area. The accepting unit accepts either the first command or the second command based on a touch operation on the second display area.

Preferably, the content display device further includes a display including a first display area.

Preferably, the touch panel includes a first display area.
Preferably, the memory stores each of the plurality of image groups in association with the attribute, and stores each of the plurality of contents in association with the attribute. The second display control unit displays an image group corresponding to the content displayed in the first display area in the second display area.

Preferably, the memory stores a plurality of contents as a series of content groups. The first display control unit refers to a series of content groups and displays each content based on a corresponding attribute.

According to another aspect of the present invention, content is displayed on a computer including an arithmetic processing unit, first and second display areas, and a memory for storing at least one content and at least one image group. A computer-readable recording medium for recording a content display program is provided. The content display program receives the first instruction and the step of receiving one of the first and second instructions in the arithmetic processing unit, the step of displaying the display target area of the content via the first display area, and the first instruction. A step of advancing the content display target area when the second command is received, a step of returning the content display target area when the second command is received, and a step based on at least one image group when the first command is received. Outputting a first effect via two display areas, and outputting a second effect via the second display area based on at least one image group when a second command is received And execute.

Preferably, the step of outputting the first effect includes a step of displaying at least one of the image groups in the second display area in the first order. The step of outputting the second effect includes a step of displaying any one of the at least one image group in the second display area in a second order opposite to the first order.

Preferably, the at least one image group includes a first image group and a second image group. The step of outputting the first effect includes a step of sequentially displaying the first image group in the second display area. The step of outputting the second effect includes a step of sequentially displaying the second image group in the second display area.

Preferably, the computer further includes a touch panel including a second display area. The step of accepting either the first or second command includes the step of accepting either the first or second command based on a touch operation on the second display area.

Preferably, the computer further includes a display including the first display area.
Preferably, the touch panel includes a first display area.

Preferably, the memory stores each of the plurality of image groups in association with the attribute, and stores each of the plurality of contents in association with the attribute. The step of outputting the first effect includes a step of displaying an image group corresponding to the content displayed in the first display area in the second display area. The step of outputting the second effect includes a step of displaying an image group corresponding to the content displayed in the first display area in the second display area.

Preferably, the memory stores a plurality of contents as a series of content groups. The step of displaying the display target area includes a step of displaying each of the contents in the first display area based on a corresponding attribute with reference to a series of content groups.

According to another aspect of the present invention, there is provided a content display method in a computer including an arithmetic processing unit, first and second display areas, and a memory for storing at least one content and at least one image group. Provided. The content display method includes: a step in which the arithmetic processing unit accepts one of the first and second instructions; a step in which the arithmetic processing unit displays a content display target area via the first display area; A step in which the processing unit advances the content display target area when receiving the first command; a step in which the arithmetic processing unit returns the content display target area when receiving the second command; When the first instruction is received, the step of outputting the first effect via the second display area based on at least one image group, and the arithmetic processing unit receives the second instruction And outputting a second effect via the second display area based on at least one image group.

As described above, according to the present invention, there is provided a content display device, a content display method, and a computer-readable recording medium for recording a content display program that allow a user to intuitively recognize the progress state of content by simple processing. Provided.

It is a 1st image figure which shows the electronic device at the time of the page switching process which concerns on this Embodiment. It is a 1st image figure which shows the 2nd display panel at the time of the page switching process which concerns on this Embodiment. It is a 2nd image figure which shows the 2nd display panel at the time of the page switching process which concerns on this Embodiment. It is an image figure which shows the external appearance of the electronic device at the time of the row switching process which concerns on this Embodiment. It is a 1st image figure which shows the 2nd display panel at the time of the row switching process which concerns on this Embodiment. It is a 2nd image figure which shows the 2nd display panel at the time of the row switching process which concerns on this Embodiment. It is an image figure which shows the external appearance of the electronic device at the time of the image switching process which concerns on this Embodiment. It is a 1st image figure which shows the 2nd display panel at the time of the image switching process which concerns on this Embodiment. It is a 2nd image figure which shows the 2nd display panel at the time of the image switching process which concerns on this Embodiment. It is a block diagram showing the hardware constitutions of an electronic device. It is the figure which showed the structure of the display panel, and the peripheral circuit of the said display panel. It is sectional drawing of a display panel and a backlight. It is the figure which showed the timing chart at the time of operating an optical sensor circuit. It is sectional drawing which showed the structure which a photodiode receives the light from a backlight in the case of a scan. It is the figure which showed schematic structure of the command. It is a figure for demonstrating the command of classification "000". FIG. 10 is a diagram for explaining a command of type “001”. It is a figure for demonstrating the command of classification "010". FIG. 10 is a diagram for explaining a command of type “011”. It is a figure for demonstrating the command of classification "100". FIG. 10 is a diagram for explaining a command of type “101”. It is the figure which showed schematic structure of the response data. It is the figure which showed the image (scanned image) obtained by scanning a finger | toe. It is sectional drawing which showed the structure which a photodiode receives external light in the case of a scan. It is a block diagram which shows the function structure of the electronic device which concerns on this Embodiment. It is an image figure which shows the effect image group which RAM memorize | stores concerning this Embodiment. It is an image figure which shows the data structure of the attribute data of the content contained in the content data which concerns on this Embodiment. It is an image figure which shows the data structure of the display mode data which concern on this Embodiment. It is an image figure which shows the data structure of the command data for a display which concerns on this Embodiment. It is a 2nd image figure which shows the electronic device at the time of the page switching process which concerns on this Embodiment. It is an image figure which shows an electronic device when the search result in selection which concerns on this Embodiment switches. It is a 1st image figure which shows the 2nd display panel when the search result in selection which concerns on this Embodiment switches. It is a 2nd image figure which shows the 2nd display panel when the search result in selection which concerns on this Embodiment switches. It is an image figure which shows the electronic device at the time of the search result page switching process which concerns on this Embodiment. It is a 1st image figure which shows the 2nd display panel at the time of the search result page switching process which concerns on this Embodiment. It is a 2nd image figure which shows the 2nd display panel at the time of the search result page switching process based on this Embodiment. It is an image figure which shows the electronic device at the time of the switching process of the display area of the map which concerns on this Embodiment. It is a 1st image figure which shows the 2nd display panel at the time of the switching process of the display area of the map which concerns on this Embodiment. It is a 2nd image figure which shows the 2nd display panel at the time of the switching process of the display area of the map which concerns on this Embodiment. It is a flowchart which shows the process sequence of the content display process in the electronic device which concerns on this Embodiment. It is a 1st image figure which shows the data structure of the book data containing multiple types of content. It is a 2nd image figure which shows the data structure of the book data containing multiple types of content. It is a 1st flowchart which shows the detailed process sequence of the process for switching and displaying the multiple types of content in the electronic device which concerns on this Embodiment. It is a 2nd flowchart which shows the detailed process sequence of the process for switching and displaying the multiple types of content in the electronic device which concerns on this Embodiment. It is a 3rd flowchart which shows the detailed process sequence of the process for switching and displaying the multiple types of content in the electronic device which concerns on this Embodiment. It is a 1st image figure which shows the 2nd display panel which shows the 1st additional function of the electronic device 100 which concerns on this Embodiment. It is a 2nd image figure which shows the 2nd display panel which shows the 1st additional function of the electronic device 100 which concerns on this Embodiment. It is an image figure which shows the 2nd display panel which shows the 2nd additional function of the electronic device 100 which concerns on this Embodiment. It is an image figure which shows the 2nd display panel which shows the 3rd additional function of the electronic device 100 which concerns on this Embodiment. It is an image figure which shows the 2nd display panel which shows the 4th additional function of the electronic device 100 which concerns on this Embodiment. It is an image figure which shows the 1st display panel which concerns on the modification of a display area.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description, the same parts are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed description thereof will not be repeated.

<Overall Configuration of Electronic Device 100>
First, an overall configuration of electronic device 100 that is an example of the content display device according to the present embodiment will be described.

FIG. 1 is a first image diagram showing electronic device 100 at the time of page switching processing according to the present embodiment. More specifically, FIG. 1A is an image diagram of the electronic device 100 in a state where the first display panel 140 (display) displays the third page of text in horizontal writing. FIG. 1B is an image diagram of the electronic device 100 when a right touch operation is input to the second display panel 240 (touch panel). FIG. 1C is an image diagram of the electronic device 100 in a state where the text is returned by one page. FIG. 1D is an image diagram of the electronic device 100 when a left touch operation is input to the second display panel 240. FIG. 1E is an image diagram of the electronic device 100 in a state where the text is advanced by one page.

Referring to FIG. 1, electronic device 100 includes a first housing 100A and a second housing 100B. The first casing 100A and the second casing 100B are foldably connected by a hinge 100C. The first housing 100A includes a first photosensor built-in liquid crystal panel 140 (hereinafter also referred to as a first display panel 140). Second housing 100B includes an operation key and a second liquid crystal panel 240 with a built-in photosensor (hereinafter also referred to as second display panel 240).

As described above, the electronic device 100 according to the present embodiment includes two liquid crystal panels with a built-in optical sensor. The electronic device 100 is a portable device having a display function, such as a PDA (Personal Digital Assistant), a notebook personal computer, a portable phone, and an electronic dictionary.

Electronic device 100 according to the present embodiment displays content such as text and images using first display panel 140, and also allows an operation from the user via operation keys 177 and second display panel 240. Accept instructions. The second display panel 240 receives a command for moving a pointer displayed on the first display panel 140 and a command for controlling display of content displayed on the first display panel 140.

In the present embodiment, the first display panel 140 does not need to be a photosensor built-in liquid crystal panel except for a modified example (see FIG. 50) described later, and may be a display capable of displaying content. That's fine. And in this Embodiment, since the 2nd display panel 240 needs to be able to detect the touch operation by a user except the modification (refer FIG. 50) mentioned later, a touch panel which has a tablet function, A liquid crystal panel with a built-in optical sensor is preferable.

<Operation overview when text is displayed horizontally>
In the following, referring to FIGS. 1 to 3, operation when electronic device 100 according to the present embodiment switches text in “page units” when first display panel 140 displays text is performed. An outline will be described. FIG. 2 is a first image diagram showing the second display panel 240 during the page switching process according to the present embodiment. FIG. 3 is a second image diagram showing the second display panel during the page switching process according to the present embodiment. Here, the case where electronic device 100 displays text in horizontal writing will be described.

Referring to FIG. 1 (A), electronic device 100 causes first page of text to be displayed horizontally on first display panel 140. More specifically, the electronic device 100 accepts an operation from the user and causes the first display panel 140 to display a page desired by the user. Electronic device 100 accepts a page switching command from the user via second display panel 240.

Referring to FIG. 1B, electronic device 100 accepts a touch operation in the right direction from the user via second display panel 240. That is, the user slides the finger 900 rightward in a state where the finger 900 or the like is in contact with the second display panel 240. More specifically, electronic device 100 receives a touch operation in the right direction by detecting a contact position between second display panel 240 and finger 900.

Referring to FIG. 1C, electronic device 100 causes second page of text to be displayed on first display panel 140. That is, when displaying the text in horizontal writing, electronic device 100 returns the page of the displayed text to the front in response to a touch operation in the right direction. At this time, as shown in FIGS. 2A to 2H, the electronic device 100 according to the present embodiment is configured to perform the second processing based on the effect image data for books stored in advance in a RAM or the like. Are displayed in the first order on the display panel 240.

Referring to FIG. 1D, electronic device 100 accepts a leftward touch operation from the user via second display panel 240. That is, the user slides the finger 900 leftward with the finger 900 or the like being in contact with the second display panel 240. More specifically, electronic device 100 receives a touch operation in the left direction by detecting a contact position between second display panel 240 and finger 900.

1E, electronic device 100 causes first display panel 140 to display the fourth page of text. That is, when displaying text in horizontal writing, electronic device 100 advances the page of the displayed text first in response to a touch operation in the left direction. At this time, as shown in FIGS. 3A to 3H, the electronic device 100 according to the present embodiment performs the second processing based on the effect image data for books stored in advance in a RAM or the like. On the display panel 240, the effect image for turning the book is displayed in the reverse order (second order) of the first order.

In the following, referring to FIGS. 4 to 6, operation when electronic device 100 according to the present embodiment switches text in “line units” when first display panel 140 displays text. An outline will be described. Here, the case where the electronic device 100 displays the text in horizontal writing will be described.

FIG. 4 is an image diagram showing electronic device 100 at the time of row switching processing according to the present embodiment. More specifically, FIG. 4A is an image diagram of the electronic device 100 in a state where the first display panel 140 displays the third page of text in horizontal writing. FIG. 4B is an image diagram of the electronic device 100 when a downward touch operation is input to the second display panel 240. FIG. 4C is an image diagram of the electronic device 100 in a state where the text is returned by one line. FIG. 4D is an image diagram of the electronic device 100 when an upward touch operation is input to the second display panel 240. FIG. 4E is an image diagram of the electronic device 100 in a state where the text is advanced by one line.

FIG. 5 is a first image diagram showing the second display panel 240 during the row switching process according to the present embodiment. FIG. 6 is a second image diagram showing the second display panel during the row switching process according to the present embodiment.

4A, electronic device 100 causes first display panel 140 to display the third page of text in horizontal writing. More specifically, the electronic device 100 accepts an operation from the user and causes the first display panel 140 to display a page desired by the user. Electronic device 100 accepts a page switching command from the user via second display panel 240.

Referring to FIG. 4B, electronic device 100 accepts a downward touch operation from the user via second display panel 240. That is, the user slides the finger 900 downward with the finger 900 or the like being in contact with the second display panel 240. More specifically, electronic device 100 accepts a downward touch operation by detecting a contact position between second display panel 240 and finger 900.

Referring to FIG. 4C, electronic device 100 causes first display panel 140 to display a new text obtained by returning the original text by one line. That is, when displaying the text in horizontal writing, electronic device 100 returns the displayed text by one line in response to a downward touch operation. At this time, as shown in FIGS. 5A to 5C, electronic device 100 according to the present embodiment is based on effect image data in which text stored in advance in a RAM or the like moves line by line. Thus, an effect image in which the text moves line by line is displayed on the second display panel 240.

Referring to FIG. 4D, electronic device 100 accepts an upward touch operation from the user via second display panel 240. That is, the user slides the finger 900 upward in a state where the finger 900 or the like is in contact with the second display panel 240. More specifically, electronic device 100 accepts an upward touch operation by detecting a contact position between second display panel 240 and finger 900.

Referring to FIG. 4E, electronic device 100 causes first display panel 140 to display a new text obtained by advancing the original text by one line. That is, when displaying the text in horizontal writing, electronic device 100 advances the displayed text by one line in response to an upward touch operation. At this time, electronic device 100 according to the present embodiment is based on effect image data in which text stored in advance in a RAM or the like moves in units of lines, as shown in FIGS. Thus, an effect image in which the text moves line by line is displayed on the second display panel 240.

As described above, electronic device 100 according to the present embodiment displays the second effect image different between the case where the displayed book page advances and the case where the book page advances, and the case where the book row advances and the case where the book row returns. Display on the display panel 240. In particular, the electronic device 100 sequentially displays pre-stored effect images via a display area (second display area) different from the content display area (first display area). It is not necessary to perform image processing according to the above. As a result, the electronic device 100 can make the user intuitively recognize the progress state of the content with a simple process.

<Overview of operation when an image is displayed>
Furthermore, electronic device 100 according to the present embodiment switches effect images according to the type of content. That is, when the electronic device 100 displays text on the first display panel 140, the electronic device 100 displays an effect image for turning the book on the second display panel 240 and displays the photo on the first display panel 140. When an image such as an animation or a comic is displayed, an effect image in which the film advances is displayed on the second display panel 240.

Hereinafter, with reference to FIGS. 7 to 9, an outline of the operation of the electronic device 100 according to the present embodiment when the first display panel 140 displays a photograph will be described.

FIG. 7 is a first image diagram showing electronic device 100 at the time of image switching processing according to the present embodiment. More specifically, FIG. 7A is an image diagram of the electronic device 100 in a state where the first display panel 140 displays the second image. FIG. 7B is an image diagram of the electronic device 100 when a downward touch operation is input to the second display panel 240 (touch panel). FIG. 7C is an image diagram of the electronic device 100 in a state where one image is returned. FIG. 7D is an image diagram of the electronic device 100 when an upward touch operation is input to the second display panel 240. FIG. 7E is an image diagram of the electronic device 100 in a state where one image is advanced.

FIG. 8 is a first image diagram showing the second display panel 240 during the image switching process according to the present embodiment. FIG. 9 is a second image diagram showing the second display panel during the image switching process according to the present embodiment.

Referring to FIG. 7A, electronic device 100 displays the second photograph on first display panel 140. More specifically, electronic device 100 accepts an operation from the user and causes first display panel 140 to display a photograph desired by the user. Electronic device 100 accepts a photo switching command from the user via second display panel 240.

Referring to FIG. 7B, electronic device 100 accepts a downward touch operation from the user via second display panel 240. That is, the user slides the finger 900 downward with the finger 900 or the like being in contact with the second display panel 240. More specifically, electronic device 100 accepts a downward touch operation by detecting a contact position between second display panel 240 and finger 900.

Referring to FIG. 7C, electronic device 100 causes first display panel 140 to display the first photograph. In other words, electronic device 100 displays a photo preceding the currently displayed photo in response to a downward touch operation. At this time, as shown in FIGS. 8A to 8F, the electronic device 100 according to the present embodiment performs the second processing based on the effect image data for film stored in advance in a RAM or the like. Are displayed in the first order (forward direction) on the display panel 240.

Referring to FIG. 7D, electronic device 100 accepts an upward touch operation from the user via second display panel 240. That is, the user slides the finger 900 upward in a state where the finger 900 or the like is in contact with the second display panel 240. More specifically, electronic device 100 accepts an upward touch operation by detecting a contact position between second display panel 240 and finger 900.

Referring to FIG. 7 (E), electronic device 100 causes first display panel 140 to display the third photograph. In other words, electronic device 100 displays a photo next to the currently displayed photo in response to an upward touch operation. At this time, as shown in FIGS. 9A to 9F, the electronic device 100 according to the present embodiment performs the second processing based on the effect image data for the film stored in advance in the RAM or the like. Are displayed in the second order (reverse direction) on the display panel 240.

Thus, electronic device 100 according to the present embodiment switches the display order of effect images or the type of effect image according to the type of content, so that the user can intuitively recognize the progress of content. it can.

Hereinafter, a configuration for realizing such a function will be described in detail. The relationship between the direction of the touch operation, the advance / retreat of the content, and the advance / retreat of the effect image is not limited to that according to this embodiment. For example, the relationship between the direction of the touch operation and the content advance / retreat, the relationship between the content advance / retreat and the effect image, or the relationship between the effect image and the direction of the touch operation may be reversed.

<Hardware Configuration of Electronic Device 100>
Next, an aspect of a specific configuration of the electronic device 100 will be described with reference to FIG. FIG. 10 is a block diagram illustrating a hardware configuration of electronic device 100.

The electronic device 100 includes a first unit 1001 (main device) and a second unit 1002. The second unit 1002 is detachably connected to the first unit 1001 from the electronic device 100. The first unit 1001 includes a main body device 101 and a display device 102. The second unit 1002 includes a display device 103 and a main device 104.

The first housing 100A includes a display device 102. Second housing 100B includes main device 101. The second housing 100B includes a second unit 1002.

(About the first unit)
The main unit 101 includes a CPU (Central Processing Unit) 110, a RAM (Random Access Memory) 171, a ROM (Read-Only Memory) 172, a memory card reader / writer 173, an external communication unit 174, a microphone 175, A speaker 176, an operation key 177, a power switch 191, a power circuit 192, a power detection unit 193, a USB (Universal Serial Bus) connector 194, an antenna 195, and a LAN (Local Area Network) connector 196 are included. . Each component (110, 171 to 177, 193) is connected to each other by a data bus DB1. A memory card 1731 is attached to the memory card reader / writer 173.

CPU 110 executes the program. The operation key 177 receives an instruction input from the user of the electronic device 100. The RAM 171 stores data generated by the execution of the program by the CPU 110 or data input via the operation keys 177 in a volatile manner. The ROM 172 stores data in a nonvolatile manner. The ROM 172 is a ROM capable of writing and erasing data such as an EPROM (Erasable Programmable Read-Only Memory) and a flash memory.

External communication unit 174 communicates with other electronic devices. Specifically, the external communication unit 174 communicates with, for example, the second unit 1002 via the USB connector 194. The external communication unit 174 performs wireless communication with the second unit 1002 via the antenna 195, for example. Furthermore, the external communication unit 174 performs wired communication with other electronic devices via the LAN connector 196.

The main unit 101 may communicate with other electronic devices by wireless communication other than Bluetooth (registered trademark). For example, the external communication unit 174 may perform wireless communication with another electronic device connected to the LAN via a wireless LAN antenna (not shown). Alternatively, wireless communication may be performed with another electronic device via an infrared port (not shown).

The power switch 191 is a switch for starting up the electronic device 100.
When the power switch 191 is turned on, the power supply circuit 192 supplies power to each component connected to the data bus DB1 and the display device 102 via the power detection unit 193. When the power switch 191 is turned on, the power circuit 192 supplies power to the external communication unit 174 without going through the power detection unit 193.

The power supply detection unit 193 detects the output from the power supply circuit 192. In addition, the power supply detection unit 193 sends information (for example, a voltage value and a current value) regarding the detected output to the CPU 110.

The USB connector 194 is used to connect the first unit 1001 to the second unit 1002. Note that the main device 101 may include other USB connectors in addition to the USB connector 194.

The first unit 1001 transmits data to the second unit 1002 via the USB connector 194. The first unit 1001 receives data from the second unit 1002 via the USB connector 194. Further, the first unit 1001 supplies power to the second unit 1002 via the USB connector 194.

The antenna 195 is used for communication in accordance with the Bluetooth (registered trademark) standard between the first unit 1001 and another communication device (for example, the second unit 1002). The LAN connector 196 is used to connect the electronic device 100 to the LAN.

The display device 102 includes a driver 130, a photosensor built-in liquid crystal panel 140 (hereinafter referred to as a display panel 140), an internal IF 178, a backlight 179, and an image processing engine 180.

The driver 130 is a drive circuit for driving the display panel 140 and the backlight 179. Various drive circuits included in the driver 130 will be described later.

The display panel 140 is a device having a liquid crystal display function and an optical sensor function. That is, the display panel 140 can perform image display using liquid crystal and sensing using an optical sensor. Details of the display panel 140 will be described later.

The internal IF (Interface) 178 mediates exchange of data between the main device 101 and the display device 102.

The backlight 179 is a light source disposed on the back surface of the display panel 140. The backlight 179 irradiates the back surface with uniform light.

The image processing engine 180 controls the operation of the display panel 140 via the driver 130. Here, the control is performed based on various data sent from the main apparatus 101 via the internal IF 178. Note that the various data includes commands to be described later. In addition, the image processing engine 180 processes data output from the display panel 140 and sends the processed data to the main apparatus 101 via the internal IF 178. Further, the image processing engine 180 includes a driver control unit 181, a timer 182, and a signal processing unit 183.

The driver control unit 181 controls the operation of the driver 130 by sending a control signal to the driver 130. In addition, the driver control unit 181 analyzes a command transmitted from the main device 101. Then, the driver control unit 181 sends a control signal based on the analysis result to the driver 130. Details of the operation of the driver 130 will be described later.

The timer 182 generates time information and sends the time information to the signal processing unit 183.
The signal processing unit 183 receives data output from the optical sensor. Here, since the data output from the optical sensor is analog data, the signal processing unit 183 first converts the analog data into digital data. Further, the signal processing unit 183 performs data processing on the digital data in accordance with the content of the command sent from the main device 101. Then, the signal processing unit 183 sends data including the data after the above data processing and the time information acquired from the timer 182 (hereinafter referred to as response data) to the main unit 101. The signal processing unit 183 includes a RAM (not shown) that can store a plurality of scan data, which will be described later, continuously.

The command includes a sensing command for instructing sensing by the optical sensor. Details of the sensing command and the response data will be described later (FIGS. 15, 16, and 22).

Note that the timer 182 is not necessarily provided in the image processing engine 180. For example, the timer 182 may be provided outside the image processing engine 180 in the display device 102. Alternatively, the timer 182 may be provided in the main body device 101. Further, the microphone 175 and the speaker 176 are not always provided in the electronic device 100, and may be configured so as not to include either or both of the microphone 175 and the speaker 176 depending on the embodiment of the electronic device 100.

Here, the display device 102 includes a system liquid crystal. The system liquid crystal is a device obtained by integrally forming peripheral devices of the display panel 140 on the glass substrate of the display panel 140. In the present embodiment, the driver 130 (excluding the circuit that drives the backlight 179), the internal IF 178, and the image processing engine 180 are integrally formed on the glass substrate of the display panel 140. Note that the display device 102 is not necessarily configured using the system liquid crystal, and the driver 130 (excluding a circuit that drives the backlight 179), the internal IF 178, and the image processing engine 180 are included in the glass substrate. Other substrates may be configured.

(About the second unit)
The second unit 1002 receives power supply from the first unit 1001. Specifically, the second unit 1002 is supplied with power from the power supply circuit 192 of the first unit 1001 by connecting a USB connector 294 described later and the USB connector 194 of the first unit 1001.

The main body device 104 includes a CPU 210, a RAM 271, a ROM 272, an external communication unit 274, a power supply detection unit 293, a USB connector 294, an antenna 295, and a signal strength detection unit 297. Each component (210, 271, 272, 274, 293) is connected to each other by a data bus DB2.

CPU 210 executes a program. The RAM 271 stores data generated by the execution of the program by the CPU 210 in a volatile manner. The ROM 272 stores data in a nonvolatile manner. The ROM 272 is a ROM capable of writing and erasing data such as an EPROM (Erasable Programmable Read-Only Memory) and a flash memory.

The external communication unit 274 communicates with other electronic devices. Specifically, the external communication unit 274 communicates with, for example, the first unit 1001 via the USB connector 294. The external communication unit 274 communicates with the first unit 1001 through the antenna 295, for example.

The main body device 104 may communicate with another electronic device (for example, the first unit 1001) by wireless communication other than Bluetooth (registered trademark). For example, the external communication unit 274 may perform wireless communication with other electronic devices via an infrared port (not shown).

The signal strength detection unit 297 detects the strength of the signal received via the antenna 295. Then, the signal strength detection unit 297 sends the detected strength to the external communication unit 274.

The USB connector 294 is used to connect the second unit 1002 to the first unit 1001.

The second unit 1002 transmits data to the first unit 1001 via the USB connector 294. The second unit 1002 receives data from the first unit 1001 via the USB connector 294. Furthermore, the second unit 1002 receives power supply from the first unit 1001 via the USB connector 294 as described above. The second unit 1002 stores the electric power supplied from the first unit 1001 in a battery (not shown).

The antenna 295 is used for communication according to the Bluetooth (registered trademark) standard between the second unit 1002 and the first unit 1001, for example.

The power detection unit 293 detects the power supplied via the USB connector 294. In addition, the power supply detection unit 293 sends information about the detected power to the CPU 210.

The main device 104 may have a function of performing infrared communication.
The display device 103 includes a driver 230, an optical sensor built-in liquid crystal panel 240 (hereinafter referred to as “display panel 240”), an internal IF 278, a backlight 279, and an image processing engine 280. The image processing engine 280 includes a driver control unit 281, a timer 282, and a signal processing unit 283.

The display device 103 has the same configuration as the display device 102. That is, the driver 230, the display panel 240, the internal IF 278, the backlight 279, and the image processing engine 280 have the same configuration as the driver 130, the display panel 140, the internal IF 178, the backlight 179, and the image processing engine 180 in the display device 102. Have each. The driver control unit 281, the timer 282, and the signal processing unit 283 have the same configurations as the driver control unit 181, the timer 182, and the signal processing unit 183 in the display device 102, respectively. Therefore, description of each functional block included in display device 103 will not be repeated.

Incidentally, the processing in the electronic device 100 is realized by software executed by each hardware and the CPU 110. Such software may be stored in the ROM 172 in advance. The software may be stored in a memory card 1731 or other storage medium and distributed as a program product. Alternatively, the software may be provided as a program product that can be downloaded by an information provider connected to the so-called Internet. Such software is read from the storage medium by the memory card reader / writer 173 or other reading device, or downloaded via the communication unit 174 or communication IF (not shown), and then temporarily stored in the ROM 172. The The software is read from the ROM 172 by the CPU 110 and stored in the RAM 171 in the form of an executable program. CPU 110 executes the program.

Each component constituting the main device 101 of the electronic device 100 shown in FIG. 10 is a general one. Therefore, it can be said that the essential part of the present invention is the software stored in the RAM 171, the ROM 172, the memory card 1731 and other storage media, or the software downloadable via the network. Since the hardware operation of main device 101 of electronic device 100 is well known, detailed description will not be repeated.

The storage medium is not limited to a memory card, but is a CD-ROM, FD (Flexible Disk), hard disk, magnetic tape, cassette tape, optical disk (MO (Magnetic Optical Disc) / MD (Mini Disc) / DVD (Digital Versatile). Disc)), IC (Integrated Circuit) card (excluding memory card), optical card, mask ROM, EPROM, EEPROM (Electronically Erasable Programmable Read-Only Memory), semiconductor memory such as flash ROM, etc. It may be a medium to be used.

The program here includes not only a program directly executable by the CPU but also a program in a source program format, a compressed program, an encrypted program, and the like.

<Configuration and drive of liquid crystal panel with built-in optical sensor>
Next, the configuration of the display panel 140 and the configuration of peripheral circuits of the display panel 140 will be described. FIG. 11 is a diagram illustrating a configuration of the display panel 140 and peripheral circuits of the display panel 140.

Referring to FIG. 11, the display panel 140 includes a pixel circuit 141, an optical sensor circuit 144, a scanning signal line Gi, a data signal line SRj, a data signal line SGj, a data signal line SBj, and a sensor signal line. SSj, sensor signal line SDj, read signal line RWi, and reset signal line RSi are included. Note that i is a natural number satisfying 1 ≦ i ≦ m, and j is a natural number satisfying 1 ≦ j ≦ n.

Further, the driver 130 of the display device 102 illustrated in FIG. 10 includes, as peripheral circuits of the display panel 140, a scanning signal line driving circuit 131, a data signal line driving circuit 132, an optical sensor driving circuit 133, a switch 134, And an amplifier 135.

The scanning signal line drive circuit 131 receives the control signal TC1 from the driver control unit 181 shown in FIG. The scanning signal line drive circuit 131 applies a predetermined voltage in order from the scanning signal line G1 to each scanning signal line (G1 to Gm) based on the control signal TC1. More specifically, the scanning signal line driving circuit 131 sequentially selects one scanning signal line from the scanning signal lines (G1 to Gm) per unit time, and a TFT (to be described later) with respect to the selected scanning signal line. A voltage that can turn on the gate of the thin film transistor 142 (hereinafter referred to as a high level voltage) is applied. Note that a low level voltage is applied to a scanning signal line that is not selected without applying a high level voltage.

The data signal line driving circuit 132 receives image data (DR, DG, DB) from the driver control unit 181 shown in FIG. The data signal line driving circuit 132 applies a voltage corresponding to one row of image data to the 3n data signal lines (SR1 to SRn, SG1 to SGn, SB1 to SBn) for each unit time. Apply sequentially.

In addition, although it demonstrated using the drive system called what is called a line sequential system here, a drive system is not limited to this.

The pixel circuit 141 is a circuit for setting the luminance (transmittance) of one pixel. Further, m × n pixel circuits 141 are arranged in a matrix. More specifically, m pixel circuits 141 are arranged in the vertical direction in FIG. 11 and n pixel circuits in the horizontal direction.

The pixel circuit 141 includes an R subpixel circuit 141r, a G subpixel circuit 141g, and a B subpixel circuit 141b. Each of these three circuits (141r, 141g, 141b) includes a TFT 142, a pair of electrode pairs 143 including a pixel electrode and a counter electrode, and a capacitor (not shown).

In addition, CMOS (Complementary Metal Oxide Semiconductor) that can make n-type transistors and p-type transistors can be realized, and the movement speed of carriers (electrons or holes) is several hundreds compared to amorphous silicon thin film transistors (a-Si TFTs). For example, a polycrystalline silicon thin film transistor (p-Si TFT) is used as the TFT 142 in the display device 102 because it is twice as fast. Note that the TFT 142 will be described as an n-channel field effect transistor. However, the TFT 142 may be a p-type channel field effect transistor.

The source of the TFT 142 in the R subpixel circuit 141r is connected to the data signal line SRj. The gate of the TFT 142 is connected to the scanning signal line Gi. Further, the drain of the TFT 142 is connected to the pixel electrode of the electrode pair 143. A liquid crystal is disposed between the pixel electrode and the counter electrode. The G sub-pixel circuit 141g and the B sub-pixel circuit 141b have the same configuration as the R sub-pixel circuit 141r except that the data signal line to which the source of each TFT 142 is connected is different. Therefore, description of these two circuits (141g, 141b) will not be repeated.

Here, the setting of the luminance in the pixel circuit 141 will be described. First, the high level voltage is applied to the scanning signal line Gi. By the application of the high level voltage, the gate of the TFT 142 is turned on. In this manner, with the gate of the TFT 142 turned on, a specified voltage (voltage corresponding to image data for one pixel) is applied to each data signal line (SRj, SGj, SBj). Thereby, a voltage based on the designated voltage is applied to the pixel electrode. As a result, a potential difference is generated between the pixel electrode and the counter electrode. Based on this potential difference, the liquid crystal responds and the luminance of the pixel is set to a predetermined luminance. Note that the potential difference is held by the capacitor (auxiliary capacitor) (not shown) until the scanning signal line Gi is selected in the next frame period.

The optical sensor driving circuit 133 receives the control signal TC2 from the driver control unit 181 shown in FIG.

Then, the optical sensor drive circuit 133 sequentially selects one signal line from the reset signal lines (RS1 to RSm) for each unit time based on the control signal TC2, and determines in advance for the selected signal line. At a given timing, the voltage VDDR that is higher than usual is applied. Note that a voltage VSSR lower than the voltage applied to the selected reset signal line is kept applied to the unselected reset signal line. For example, the voltage VDDR may be set to 0V and the voltage VSSR may be set to −5V.

In addition, the photosensor driving circuit 133 sequentially selects one signal line from the readout signal lines (RW1 to RWm) per unit time based on the control signal TC2, and determines in advance for the selected signal line. At a given timing, a voltage VDD higher than usual is applied. Note that the voltage VSSR is applied to the read signal line that is not selected. For example, the value of VDD may be set to 8V.

The timing for applying the voltage VDDR and the timing for applying the voltage VDD will be described later.

The optical sensor circuit 144 includes a photodiode 145, a capacitor 146, and a TFT 147. In the following description, it is assumed that the TFT 147 is an n-type channel field effect transistor. However, the TFT 147 may be a p-type channel field effect transistor.

The anode of the photodiode 145 is connected to the reset signal line RSi. On the other hand, the cathode of the photodiode 145 is connected to one electrode of the capacitor 146. The other electrode of the capacitor 146 is connected to the read signal line RWi. Hereinafter, a connection point between the photodiode 145 and the capacitor 146 is referred to as a node N.

The gate of the TFT 147 is connected to the node N. The drain of the TFT 147 is connected to the sensor signal line SDj. Further, the source of the TFT 147 is connected to the sensor signal line SSj. Details of sensing using the optical sensor circuit 144 will be described later.

The switch 134 is a switch provided for switching whether or not to apply a predetermined voltage to the sensor signal lines (SD1 to SDn). The switching operation of the switch 134 is performed by the optical sensor driving circuit 133. The voltage applied to the sensor signal lines (SD1 to SDn) when the switch 134 is turned on will be described later.

The amplifier 135 amplifies the voltage output from each sensor signal line (SS1 to SSn). The amplified voltage is sent to the signal processing unit 183 shown in FIG.

Note that the image processing engine 180 controls the timing at which an image is displayed on the display panel 140 using the pixel circuit 141 and the timing at which sensing is performed using the optical sensor circuit 144.

FIG. 12 is a cross-sectional view of the display panel 140 and the backlight 179. Referring to FIG. 12, display panel 140 includes an active matrix substrate 151A, a counter substrate 151B, and a liquid crystal layer 152. The counter substrate 151B is disposed to face the active matrix substrate 151A. The liquid crystal layer 152 is sandwiched between the active matrix substrate 151A and the counter substrate 151B. The backlight 179 is disposed on the opposite side of the liquid crystal layer 152 with respect to the active matrix substrate 151A.

The active matrix substrate 151A includes a polarizing filter 161, a glass substrate 162, a pixel electrode 143a constituting an electrode pair 143, a photodiode 145, a data signal line 157, and an alignment film 164. Further, although not shown in FIG. 12, the active matrix substrate 151A includes the capacitor 146, the TFT 147, the TFT 142, and the scanning signal line Gi shown in FIG.

In the active matrix substrate 151A, the polarizing filter 161, the glass substrate 162, the pixel electrode 143a, and the alignment film 164 are arranged in this order from the backlight 179 side. The photodiode 145 and the data signal line 157 are formed on the liquid crystal layer 152 side of the glass substrate 162.

The counter substrate 151B includes a polarizing filter 161, a glass substrate 162, a light shielding film 163, color filters (153r, 153g, 153b), a counter electrode 143b constituting an electrode pair 143, and an alignment film 164.

In the counter substrate 151B, the alignment film 164, the counter electrode 143b, the color filters (153r, 153g, 153b), the glass substrate 162, and the polarizing filter 161 are arranged in this order from the liquid crystal layer 152 side. The light shielding film 163 is formed in the same layer as the color filters (153r, 153g, 153b).

The color filter 153r is a filter that transmits light having a red wavelength. The color filter 153g is a filter that transmits light having a green wavelength. The color filter 153b is a filter that transmits light having a blue wavelength. Here, the photodiode 145 is arranged at a position facing the color filter 153b.

The display panel 140 displays an image by blocking or transmitting light emitted by a light source such as external light or a backlight 179. Specifically, the display panel 140 changes the direction of the liquid crystal molecules of the liquid crystal layer 152 by applying a voltage between the pixel electrode 143a and the counter electrode 143b, thereby blocking or transmitting the light. . However, since the light cannot be completely blocked by the liquid crystal alone, a polarizing filter 161 that transmits only light having a specific polarization direction is provided.

Note that the position of the photodiode 145 is not limited to the above position, and may be provided at a position facing the color filter 153r or a position facing the color filter 153g.

Here, the operation of the optical sensor circuit 144 will be described. FIG. 13 is a timing chart when the optical sensor circuit 144 is operated. In FIG. 13, a voltage VINT indicates a potential at the node N in the photosensor circuit 144. The voltage VPIX is an output voltage from the sensor signal line SSj shown in FIG. 11 and is a voltage before being amplified by the amplifier 135.

Hereinafter, the description will be divided into a reset period for resetting the optical sensor circuit 144, a sensing period for sensing light using the optical sensor circuit 144, and a readout period for reading the sensing result.

First, the reset period will be described. In the reset period, the voltage applied to the reset signal line RSi is instantaneously switched from the low level (voltage VSSR) to the high level (voltage VDDR). On the other hand, the voltage applied to the read signal line RWi is kept at the low level (voltage VSSR). As described above, by applying the high-level voltage to the reset signal line RSi, a current starts to flow in the forward direction (from the anode side to the cathode side) of the photodiode 145. As a result, the voltage VINT which is the potential of the node N has a value represented by the following expression (1). In Equation (1), the forward voltage drop amount in the photodiode 145 is Vf.

VINT = VSSR + | VDDR−VSSR | −Vf (1)
Therefore, the potential of the node N is a value smaller by Vf than the voltage VDDR as shown in FIG.

Here, since the voltage VINT is not more than the threshold value for turning on the gate of the TFT 147, there is no output from the sensor signal line SSj. For this reason, the voltage VPIX does not change. Further, a difference corresponding to the voltage VINT occurs between the electrodes of the capacitor 146. For this reason, the capacitor 146 accumulates charges corresponding to the difference.

Next, the sensing period will be described. In the sensing period following the reset period, the voltage applied to the reset signal line RSi is instantaneously switched from the high level (voltage VDDR) to the low level (voltage VSSR). On the other hand, the voltage applied to the read signal line RWi is kept at the low level (voltage VSSR).

Thus, by changing the voltage applied to the reset signal line RSi to the low level, the potential of the node N becomes higher than the voltage of the reset signal line RSi and the voltage of the read signal line RWi. For this reason, in the photodiode 145, the voltage on the cathode side becomes higher than the voltage on the anode side. That is, the photodiode 145 is in a reverse bias state. In such a reverse bias state, when the photodiode 145 receives light from the light source, current starts to flow from the cathode side to the anode side of the photodiode 145. As a result, as shown in FIG. 13, the potential of the node N (that is, the voltage VINT) becomes lower with the passage of time.

Note that since the voltage VINT continues to decrease in this way, the gate of the TFT 147 does not turn on. Therefore, there is no output from the sensor signal line SSj. For this reason, the voltage VPIX does not change.

Next, the reading period will be described. In the readout period following the sensing period, the voltage applied to the reset signal line RSi is kept at the low level (voltage VSSR). On the other hand, the voltage applied to the read signal line RWi is instantaneously switched from the low level (voltage VSSR) to the high level (voltage VDD). Here, the voltage VDD is higher than the voltage VDDR.

In this way, by instantaneously applying a high level voltage to the read signal line RWi, the potential of the node N is raised through the capacitor 146 as shown in FIG. Note that the increase width of the potential of the node N is a value corresponding to the voltage applied to the read signal line RWi. Here, since the potential of the node N (that is, the voltage VINT) is raised to a threshold value that turns on the gate of the TFT 147, the gate of the TFT 147 is turned on.

At this time, if a constant voltage is applied in advance to the sensor signal line SDj (see FIG. 11) connected to the drain side of the TFT 147, the sensor signal line SSj connected to the source side of the TFT 147 will cause the VPIX in FIG. As shown in the graph, a voltage corresponding to the potential of the node N is output.

Here, if the amount of light received by the photodiode 145 (hereinafter referred to as the amount of received light) is small, the slope of the straight line shown in the VINT graph of FIG. 13 becomes gentle. As a result, the voltage VPIX is higher than when the amount of received light is large. As described above, the optical sensor circuit 144 changes the value of the voltage output to the sensor signal line SSj in accordance with the amount of light received by the photodiode 145.

By the way, in the above description, the operation has been described focusing on one optical sensor circuit 144 among the m × n optical sensor circuits. Below, operation | movement of each photosensor circuit in the display panel 140 is demonstrated.

First, the optical sensor driving circuit 133 applies a predetermined voltage to all n sensor signal lines (SD1 to SDn). Next, the photosensor drive circuit 133 applies a voltage VDDR that is higher than normal to the reset signal line RS1. The other reset signal lines (RS2 to RSm) and read signal lines (RW1 to RWm) are kept in a state where a low level voltage is applied. As a result, the n photosensor circuits in the first row in FIG. 11 enter the reset period described above. Thereafter, the n photosensor circuits in the first row enter a sensing period. Further, thereafter, the n photosensor circuits in the first row enter a reading period.

Note that the timing for applying a predetermined voltage to all n sensor signal lines (SD1 to SDn) is not limited to the above timing, and may be any timing that is applied at least before the readout period. .

When the reading period of the n photosensor circuits in the first row is completed, the photosensor drive circuit 133 applies a voltage VDDR that is higher than usual to the reset signal line RS2. That is, the reset period of the n photosensor circuits in the second row starts. When the reset period ends, the n photosensor circuits in the second row enter a sensing period, and thereafter enter a reading period.

Thereafter, the above-described processing is sequentially performed on the n photosensor circuits in the third row, the n photosensor circuits in the fourth row,..., The n photosensor circuits in the m row. As a result, the sensor signal lines (SS1 to SSn) output the first row sensing result, the second row sensing result,..., The mth row sensing result in this order.

In the display device 102, sensing is performed for each row as described above, and a sensing result is output from the display panel 140 for each row. For this reason, hereinafter, the data after the signal processing unit 183 performs the above-described data processing on the data regarding the voltage for m rows from the first row to the m-th row output from the display panel 140, This is called “scan data”. That is, scan data refers to image data obtained by scanning a scan target (for example, a user's finger). An image displayed based on the scan data is referred to as a “scanned image”. Further, in the following, sensing is referred to as “scan”.

In the above description, the configuration in which scanning is performed using all m × n photosensor circuits has been described as an example, but the configuration is not limited thereto. Scanning may be performed on a partial region of the surface of the display panel 140 using a photosensor circuit selected in advance.

In the following, it is assumed that the electronic device 100 can take either configuration. Further, switching between the components is performed by a command sent from the main body apparatus 101 based on an input via the operation key 177 or the like. Note that when scanning is performed on a partial area on the surface of the display panel 140, the image processing engine 180 sets a scan target area. The setting of the area may be configured to be specified by the user via the operation key 177.

As described above, when scanning is performed on a partial region of the surface of the display panel 140, there are the following modes of use for displaying an image. The first is a mode in which an image is displayed in a surface area other than the partial area (hereinafter referred to as a scan area). The second is a mode in which no image is displayed in the surface area other than the scan area. Which mode is used is based on a command sent from the main apparatus 101 to the image processing engine 180.

FIG. 14 is a cross-sectional view of the display panel 140 and the backlight 179 and shows a configuration in which the photodiode 145 receives light from the backlight 179 during scanning.

Referring to FIG. 14, when the user's finger 900 is in contact with the surface of the display panel 140, a part of the light emitted from the backlight 179 is part of the user's finger 900 (abbreviated in the contacted area). Reflected on the plane). The photodiode 145 receives the reflected light.

Further, even in a region where the finger 900 is not in contact, a part of the light emitted from the backlight 179 is reflected by the user's finger 900. Even in this case, the photodiode 145 receives the reflected light. However, since the finger 900 is not in contact with the surface of the display panel 140 in the region, the amount of light received by the photodiode 145 is smaller than the region in which the finger 900 is in contact. Note that the photodiode 145 cannot receive most of the light emitted from the backlight 179 that does not reach the user's finger 900.

Here, by turning on the backlight 179 at least during the sensing period, the optical sensor circuit 144 can output a voltage corresponding to the amount of light reflected by the user's finger 900 from the sensor signal line SSj. it can. In this manner, by controlling lighting and extinguishing of the backlight 179, the display panel 140 has a contact position of the finger 900, a contact range of the finger 900 (determined by the pressing force of the finger 900), and the display panel 140. The voltage output from the sensor signal lines (SS1 to SSn) varies depending on the direction of the finger 900 with respect to the surface of the sensor.

As described above, the display device 102 can scan an image (hereinafter also referred to as a reflected image) obtained by reflecting light with the finger 900.

Note that examples of the scan object other than the finger 900 include a stylus.
By the way, in the present embodiment, a display panel is described as an example of the display device of electronic device 100, but another panel such as an organic EL (Electro-Luminescence) panel is used instead of the display panel. Also good.

<About data>
Next, an example of commands exchanged between the first unit 1001 and the second unit 1002 and an example of commands exchanged between the main body device 101 and the display device 102 in the first unit 1001 will be described. .

FIG. 15 is a diagram showing a schematic configuration of a command. Referring to FIG. 15, the command includes header DA01, first field DA02, second field DA03, third field DA04, fourth field DA05, fifth field DA06, and spare data area DA07. including.

FIG. 16 is a diagram for explaining a command of type “000” (that is, a sensing command). The CPU 110 sends a command of type “000” (hereinafter referred to as “first command”) from the main unit 101 of the first unit 1001 to the second unit 1002. Alternatively, the CPU 110 sends the first command from the main body device 101 to the display device 102. In the following, the case where the CPU 110 sends the first command from the main unit 101 of the first unit 1001 to the second unit 1002 will be described as an example.

The CPU 110 writes the command type (“000”), the command transmission destination, and the like in the header DA01. The CPU 110 writes the value of the timing whose number is “1” in the first field DA02. The CPU 110 writes the value of the data type with the number “2” in the second field DA03. The CPU 110 writes the value of the reading method whose number is “3” in the third field DA04. The CPU 110 writes the value of the image gradation number “4” in the fourth field DA05. The CPU 110 writes the value of the resolution with the number “5” in the fifth field DA06.

The first command in which “00” is set in the first field DA02 requests the image processing engine 280 to transmit scan data at that time. That is, the sensing first command requests transmission of scan data obtained by scanning using the optical sensor circuit of the display panel 240 after the image processing engine 280 receives the first command. Further, the first command in which “01” is set in the first field DA02 requests transmission of scan data when the scan result is changed. Further, the first command in which “10” is set in the first field DA02 requests transmission of scan data at regular intervals.

The first command in which “001” is set in the second field DA03 requests transmission of the coordinate value of the center coordinate in the partial image. In addition, the first command in which “010” is set in the second field DA03 requests transmission of only the partial image whose scan result has changed. Note that the change in the scan result indicates that the previous scan result is different from the current scan result. Furthermore, the first command in which “100” is set in the second field DA03 requests transmission of the entire image.

Here, the “whole image” is an image generated by the image processing engine 280 based on the voltage output from each optical sensor circuit when scanning using m × n optical sensor circuits. . The “partial image” is a part of the entire image. The reason why the partial image is requested to transmit only the partial image whose scan result has changed will be described later.

Note that the coordinate value and the partial image or the entire image may be requested at the same time. Further, in the case of a configuration in which a partial area on the surface of the display panel 240 is scanned, the entire image is an image corresponding to the area to be scanned.

The first sensing command in which “00” is set in the third field DA04 requests that the backlight 279 be turned on to scan. In addition, the first command in which “01” is set in the third field DA04 requests that the backlight 279 be turned off to perform scanning. A configuration for scanning with the backlight 279 off will be described later (FIG. 24). Furthermore, the first command in which “10” is set in the third field DA04 requests scanning using both reflection and transmission. Note that the combined use of reflection and transmission refers to scanning a scan object by switching between a method of scanning with the backlight 279 turned on and a method of scanning with the backlight turned off.

The first command in which “00” is set in the fourth field DA05 requests black and white binary image data. Further, the first command in which “01” is set in the fourth field DA05 requests multi-gradation image data. Further, the first command in which “10” is set in the fourth field DA05 requests RGB color image data.

The first command in which “0” is set in the fifth field DA06 requests image data with high resolution. The first command in which “1” is set in the fifth field DA06 requests image data with a low resolution.

In addition to the data shown in FIG. 16, the first command includes designation of a region to be scanned (region of pixels that drive the optical sensor circuit 144), timing to perform scanning, timing to turn on the backlight 179, and the like. Is described.

Note that the image processing engine 280 analyzes the content of the first command, and sends back data (that is, response data) according to the result of the analysis to the main unit 101.

FIG. 17 is a diagram for explaining a command of type “001” (hereinafter referred to as “second command”). The CPU 110 sends a second command from the main unit 101 of the first unit 1001 to the second unit 1002.

The CPU 110 writes the command type (“001”), the command transmission destination, and the like in the header DA01. The CPU 110 writes the value of the display request with the number “1” in the first field DA02. CPU 110 writes information on the number / type of number “2” in second field DA03. The CPU 110 writes the value of the display range whose number is “3” in the third field DA04. The CPU 110 writes information related to the image data with the number “4” in the fourth field DA05.

The second command in which “001” is set in the first field DA02 requests the image processing engine 280 to display an image on the display panel 240. The second command in which “010” is set in the first field DA02 requests the image processing engine 280 to display an icon on the display panel 240. Further, the second command in which “011” is set in the first field DA02 requests the image processing engine 280 to display the handwriting area on the display panel 240.

The second field DA03 stores the number of images to be displayed on the display panel 240 and a number for designating the type of handwriting language. The image processing engine 280 performs processing according to the number of the images or the language type.

The second command in which “01” is set in the third field DA04 requests the image processing engine 280 to designate the display range on the display panel 240 by coordinates. Further, the second command in which “10” is set in the third field DA04 requests the image processing engine 280 to set the display range on the display panel 240 to the entire display area.

The fourth field DA05 stores image data to be displayed on the display panel 240 and position information for displaying the image data. The image processing engine 280 performs processing for displaying the image data at a position specified by the position information.

FIG. 18 is a diagram for explaining a command of type “010” (hereinafter referred to as “third command”). The CPU 110 sends the third command from the main unit 101 of the first unit 1001 to the second unit 1002. Alternatively, the CPU 210 sends a third command from the main body device 104 of the second unit 1002 to the first unit 1001.

The

CPUs

110 and 210 write the command type (“010”), the command transmission destination, and the like in the header DA01. The

CPUs

110 and 210 write the value of the OS (Operating System) processing request with the number “1” in the first field DA02. The

CPUs

110 and 210 write the value of the OS information with the number “2” in the second field DA03.

The third command in which “01” or “10” is set in the first field DA02 is transmitted from the second unit 1002 to the first unit 1001.

The third command in which “01” is set in the first field DA02 requests the first unit 1001 to transmit information indicating the OS type of the first unit 1001. The third command in which “10” is set in the first field DA02 requests the first unit 1001 to start the OS specified by the OS information.

The third command in which “000”, “001”, or “010” is set in the second field DA03 is transmitted from the second unit 1002 to the first unit 1001.

The third command in which “000” is set in the second field DA03 does not request the activation of the OS in the first unit 1001. The third command in which “001” is set in the second field DA03 indicates that the second unit 1002 has selected to start the first OS. Furthermore, the third command in which “010” is set in the second field DA03 indicates that the second unit 1002 has selected to start the second OS.

FIG. 19 is a diagram for explaining a command of type “011” (hereinafter referred to as “fourth command”). The CPU 210 sends a fourth command from the main body device 104 of the second unit 1002 to the first unit 1001.

The CPU 210 writes the command type (“011”), the command transmission destination, and the like in the header DA01. The CPU 210 writes information related to the activated application with the number “1” in the first field DA02. The CPU 210 writes the startup information with the number “2” in the second field DA03.

In the first field DA02, information specifying an application to be activated in the first unit 1001 is stored. The second field DA03 stores information used at the time of activation setting and information used after activation.

FIG. 20 is a diagram for explaining a command of type “100” (hereinafter referred to as “fifth command”). The CPU 210 sends the fifth command from the main body device 104 of the second unit 1002 to the first unit 1001.

The CPU 210 writes the command type (“100”), the command transmission destination, and the like in the header DA01. The CPU 210 writes information related to the reception request with the number “1” in the first field DA02. The CPU 210 writes information relating to the number “2” in the second field DA03. The CPU 210 writes information related to the file with the number “3” in the third field DA04.

The fifth command in which “01” is set in the first field DA02 requests the first unit 1001 to receive a file. In the second field DA03, the number of files transmitted from the second unit 1002 to the first unit 1001 is stored. Further, a file transmitted from the second unit 1002 to the first unit 1001 is stored in the third field DA04.

FIG. 21 is a diagram for explaining a command of type “101” (hereinafter referred to as “sixth command”). The CPU 110 sends the sixth command from the main unit 101 of the first unit 1001 to the second unit 1002. Alternatively, the CPU 210 sends the sixth command from the main body device 104 of the second unit 1002 to the first unit 1001.

The

CPUs

110 and 210 write the command type (“101”), the command transmission destination, and the like in the header DA01. The

CPUs

110 and 210 write the value of the communication type with the number “1” in the first field DA02. The

CPUs

110 and 210 write the value of the connection destination with the number “2” in the second field DA03. The

CPUs

110 and 210 write the value of the transfer destination with the number “3” in the third field DA04. The

CPUs

110 and 210 write the value of the signal strength acquisition timing number “4” in the fourth field DA05.

The sixth command in which “001” is set in the first field DA02 requests the counterpart device to perform infrared communication. The sixth command in which “010” is set in the first field DA02 requests the counterpart device to perform wireless communication by Bluetooth (registered trademark). Furthermore, the sixth command in which “011” is set in the first field DA02 requests the counterpart device to perform LAN communication.

The sixth command in which “000” is set in the second field DA03 indicates that it does not have information specifying the communication connection destination.

In addition, the sixth command in which “001” is set in the second field DA03 is transmitted by the first unit 1001 to the device to which the first unit 1001 is connected. Such a sixth command requests transmission of information related to a device to which the first unit 1001 is connected.

Furthermore, the sixth command in which “010” is set in the second field DA03 is transmitted by the second unit 1002 to the first unit 1001 to which the second unit 1002 is connected. Such a sixth command requests transmission of information regarding the first unit 1001 to which the second unit 1002 is connected.

The sixth command in which “011” is set in the second field DA03 is transmitted by the second unit 1002 to the first unit 1001 to which the second unit 1002 is connected. Such a sixth command requests that information regarding the second unit 1002 be set as connection destination device information.

Further, the sixth command in which “100” is set in the second field DA03 is transmitted by the first unit 1001 to the device to which the first unit 1001 is connected (for example, the second unit 1002). Such a sixth command requests that information regarding the first unit 1001 be set as connection destination device information.

The sixth command in which “000” is set in the third field DA04 indicates that it does not have information specifying the transfer destination of data (for example, a file).

The sixth command in which “001” is set in the third field DA04 is transmitted by the first unit 1001 to the data transfer destination device. Such a sixth command requests transmission of information relating to the data transfer destination device.

Furthermore, the sixth command in which “010” is set in the third field DA04 is transmitted by the second unit 1002 to the first unit 1001 that is the data transfer destination. Such a sixth command requests transmission of information related to the first unit 1001 of the data transfer destination.

Further, the sixth command in which “011” is set in the third field DA04 is transmitted by the second unit 1002 to the first unit 1001 that is the data transfer destination. Such a sixth command requests that information regarding the second unit 1002 be set as device information of the data transfer source.

Further, the sixth command in which “100” is set in the third field DA04 is transmitted by the first unit 1001 to the data transfer destination device (for example, the second unit 1002). Such a sixth command requests that information regarding the first unit 1001 be set as device information of the data transfer source.

The sixth command in which “00”, “01”, “10”, or “11” is set in the fourth field DA05 is transmitted by the first unit 1001 to the second unit 1002.

The sixth command in which “00” is set in the fourth field DA05 does not request the second unit 1002 to transmit data indicating the signal strength. Also, the sixth command in which “01” is set in the fourth field DA05 requests the signal strength detection unit 297 to transmit data indicating the signal strength at that time. Furthermore, the sixth command in which “10” is set in the fourth field DA05 requests transmission of data indicating the signal strength when the signal strength is changed. Further, the sixth command in which “11” is set in the fourth field DA05 requests the transmission of data indicating the signal strength at regular intervals.

FIG. 22 is a diagram showing a schematic configuration of response data. The response data is data corresponding to the content of the first command (sensing command).

When the first command is transmitted from the main device 101 to the second unit 1002, the CPU 210 transmits response data from the display device 103 to the first unit 1001. When the first command is transmitted from the main device 101 to the display device 102 of the first unit 1001, the image processing engine 180 transmits response data from the image processing engine 180 to the main device 101. Hereinafter, a case where the first command is transmitted from the main device 101 to the second unit 1002 will be described as an example.

Referring to FIG. 22, the response data includes a header data area DA11, a coordinate data area DA12, a time data area DA13, and an image data area DA14. Here, the value of the center coordinates of the partial image is written in the data area DA12 indicating the coordinates. The time information acquired from the timer 282 of the image processing engine 280 is written in the data area indicating the time. Further, image data (that is, scan data) after being processed by the image processing engine 280 is written in the data area indicating the image.

FIG. 23 is a diagram showing an image obtained by scanning the finger 900 (that is, a scanned image). Referring to FIG. 23, an image of a region W1 surrounded by a thick solid line is an entire image, and an image of a region P1 surrounded by a broken line is a partial image. The center point C1 of the cross indicated by a thick line is the center coordinate.

In the present embodiment, a pixel (that is, a predetermined gradation or a pixel having a photosensor circuit which is a rectangular region and whose output voltage from the sensor signal line SSj is equal to or higher than a predetermined value). An area including all of the pixels having a luminance equal to or higher than a predetermined luminance is set as a partial image area.

Also, the center coordinates are coordinates determined in consideration of the gradation of each pixel in the partial image area. Specifically, for each pixel in the partial image, the center coordinate is determined by performing a weighting process based on the gradation of the pixel and the distance between the pixel and the rectangular center point (that is, the centroid). Is done. That is, the center coordinates do not necessarily match the centroid of the partial image.

However, the position of the center coordinates is not necessarily limited to the above, and the center coordinates may be the coordinates of the centroid or the coordinates near the centroid.

When “001” is set in the data area indicating the data type of the first command, the image processing engine 280 writes the value of the central coordinate in the data area DA12 indicating the coordinate. In this case, the image processing engine 280 does not write image data in the data area DA14 indicating an image. After writing the value of the center coordinate, the image processing engine 280 sends response data including the value of the center coordinate to the main body device 104. The main device 104 sends response data including the value of the center coordinate to the main device 101 of the first unit 1001. As described above, when “001” is set in the data area indicating the data type, the first command does not request output of the image data but requests output of the value of the center coordinate.

When “010” is set in the data area indicating the data type of the first command, the image processing engine 280 stores the image data of the partial image whose scan result has changed in the data area DA14 indicating the image. Write. In this case, the image processing engine 280 does not write the value of the center coordinate in the data area DA12 indicating the coordinate. The image processing engine 280 writes the image data of the partial image whose scan result has changed, and then sends response data including the image data of the partial image to the main body device 104. The main device 104 sends response data including image data of the partial image to the main device 101 of the first unit 1001. Thus, when “010” is set in the data area indicating the data type, the first command does not request the output of the value of the center coordinate, and the image data of the partial image whose scan result has changed. Request output.

As described above, the reason for requesting transmission of only the partial image whose scan result has changed is that the scan data of the partial image area of the scan data is more important than the scan data of the other area. This is because the data is high, and the scan data in the region corresponding to the region of the partial image in the scan data is likely to change due to the contact state with the scan object such as the finger 900.

When “011” is set in the data area indicating the data type of the first command, the image processing engine 280 writes the value of the center coordinate in the data area DA12 indicating the coordinate and also indicates the data indicating the image. The image data of the partial image whose scan result has changed is written in the area DA14. Thereafter, the image processing engine 280 sends response data including the value of the center coordinate and the image data of the partial image to the main body device 104. The main device 104 sends response data including the value of the center coordinate and the image data of the partial image to the main device 101 of the first unit 1001. Thus, when “011” is set in the data area indicating the data type, the first command outputs the value of the center coordinate and the output of the image data of the partial image whose scan result has changed. Request.

When “100” is set in the data area indicating the data type of the first command, the image processing engine 280 displays the entire image in the data area DA14 indicating the image of the response data shown in FIG. Write image data. In this case, the image processing engine 280 does not write the value of the center coordinate in the data area DA12 indicating the coordinate. After writing the image data of the whole image, the image processing engine 280 sends response data including the image data of the whole image to the main body device 104. The main device 104 sends response data including image data of the entire image to the main device 101 of the first unit 1001. As described above, when “100” is set in the data area indicating the data type, the first command requests the output of the image data of the entire image without requesting the output of the center coordinate value. .

When “101” is set in the data area indicating the data type of the first command, the image processing engine 280 writes the value of the central coordinate in the data area DA12 indicating the coordinate and the data indicating the image. The image data of the entire image is written in the area DA14. Thereafter, the image processing engine 280 sends response data including the value of the center coordinate and the image data of the entire image to the main body device 104. The main device 104 sends response data including the value of the center coordinate and the image data of the entire image to the main device 101 of the first unit 1001. Thus, when “101” is set in the data area indicating the data type, the first command requests the output of the value of the center coordinate and the output of the image data of the entire image.

<Regarding First Modification of Configuration>
Next, with reference to FIG. 24, a scanning method different from the above-described scanning method (that is, the method of scanning the reflected image in FIG. 14) will be described.

FIG. 24 is a cross-sectional view showing a configuration in which a photodiode receives external light during scanning. As shown in the figure, part of the external light is blocked by the finger 900. Therefore, the photodiode disposed under the surface area of the display panel 140 that is in contact with the finger 900 can hardly receive external light. In addition, although the photodiodes disposed under the surface area where the shadow of the finger 900 is formed can receive a certain amount of external light, the amount of external light received is small compared to the surface area where no shadow is formed.

Here, by turning off the backlight 179 at least during the sensing period, the optical sensor circuit 144 can output a voltage corresponding to the position of the finger 900 relative to the surface of the display panel 140 from the sensor signal line SSj. it can. In this manner, by controlling the backlight 179 to be turned on and off, the display panel 140 has a contact position of the finger 900, a contact range of the finger 900 (determined by the pressing force of the finger 900), and the display panel 140. The voltage output from the sensor signal lines (SS1 to SSn) varies depending on the direction of the finger 900 with respect to the surface of the sensor.

As described above, the display device 102 can scan an image (hereinafter also referred to as a shadow image) obtained by blocking external light by the finger 900.

Furthermore, the display device 102 may be configured to perform scanning by turning on the backlight 179 and then performing scanning again by turning off the backlight 179. Alternatively, the display device 102 may be configured to perform scanning by turning off the backlight 179 and then performing scanning again by turning on the backlight 179.

In this case, since two scanning methods are used together, two scan data can be obtained. Therefore, it is possible to obtain a highly accurate result as compared with the case of scanning using only one scanning method.

<About display devices>
The operation of the display device 103 is controlled in accordance with a command (for example, a first command) from the main body device 101 as in the operation of the display device 102. The display device 103 has the same configuration as the display device 102. Therefore, when the display device 103 receives the same command as the display device 102 from the main body device 101, the display device 103 performs the same operation as the display device 102. For this reason, description of the configuration and operation of the display device 103 will not be repeated.

Note that the main device 101 can send commands having different commands to the display device 102 and the display device 103. In this case, the display device 102 and the display device 103 perform different operations. Further, the main device 101 may send a command to either the display device 102 or the display device 103. In this case, only one display device performs an operation according to the command. Further, the main device 101 may send a command having the same command to the display device 102 and the display device 103. In this case, the display device 102 and the display device 103 perform the same operation.

Note that the size of the display panel 140 of the display device 102 and the size of the display panel 240 of the display device 103 may be the same or different. Further, the resolution of the display panel 140 and the resolution of the display panel 240 may be the same or different.

<Functional configuration of electronic device 100 according to the present embodiment>
Hereinafter, the functional configuration of electronic apparatus 100 according to the present embodiment will be described with reference to FIGS. FIG. 25 is a block diagram showing a functional configuration of electronic device 100 according to the present embodiment.

The electronic device 100 according to the present embodiment includes a reception unit 1101, a first display control unit 1103, and a second display control unit 1110. In addition, as described with reference to FIG. 10, electronic device 100 includes RAM 171, first display panel 140, and second display panel 240.

First, the first display panel 140 emits visible light based on the content data 1711 from the first display control unit 1103, that is, based on an output signal from the CPU 110. More specifically, the first display panel 140 uses light from the backlight 179 based on the content data 1711 from the first display control unit 1103 via the image processing engine 180 (FIG. 10) or the like. While displaying text and images.

Each of the plurality of photosensor circuits 244 of the second display panel 240 receives incident light and generates an electrical signal corresponding to the incident light. The plurality of optical sensor circuits 244 as a whole input an electrical signal corresponding to incident light to the reception unit 1101 via the image processing engine 280 (FIG. 10) or the like. Note that the plurality of optical sensor circuits 244 may read the contact position of the finger 900, the stylus pen, or the like with the backlight 179 turned off as shown in FIG.

As described above, the plurality of photosensor circuits 244 and the image processing engine 280 according to the present embodiment realize an operation unit as a whole. Then, the operation unit receives an operation command for controlling the display of the text displayed on the first display panel 140 via the second display panel 240 or is displayed on the first display panel 140. Or accepting a move command to move the current pointer.

Each of the plurality of pixel circuits 241 of the second display panel 240 emits visible light based on the effect image data from the second display control unit 1110, that is, based on the output signal from the CPU 110. More specifically, the plurality of pixel circuits 241 as a whole emit light from the backlight 179 based on the effect image data from the second display control unit 1110 via the image processing engine 280 (FIG. 10) or the like. Display effect images sequentially while using.

First, the RAM 171

stores content data

1711 and 1712 for displaying content and effect

image data groups

171A and 171B for displaying effect images.

Here, with reference to FIG. 26, the effect

image data groups

171A and 171B stored in the RAM 171 will be described. FIG. 26 is an image diagram showing effect

image data groups

171A, 171B, and 171C stored in the RAM 171 according to the present embodiment.

The RAM 171 stores an effect image data group 171A for each content type (content attribute). Each of the effect image data groups 171A according to the present embodiment includes a series of effect image data. That is, the effect image data group constitutes a moving image as a whole.

For example, the text effect image data group 171A includes a plurality (a series) of effect image data 171A-1, 171A-2, and 171A-3. The second display control unit 1110 sequentially displays the effect image data 171A-1, 171A-2, 171A-3 on the second display panel 240 when the first display control unit 1103 switches the text in units of pages. Let

For example, the image effect image data group 171B includes a plurality (a series) of effect image data 171B-1, 171B-2, and 171B-3. The second display control unit 1110 sequentially applies the effect image data 171B-1, 171B-2, 171B-3 to the second display panel 240 when the first display control unit 1103 moves the text in units of lines. Display.

For example, the image effect image data group 171C includes a plurality (a series) of effect image data 171C-1, 171C-2, and 171C-3. The second display control unit 1110 causes the second display panel 240 to sequentially display the effect image data 171C-1, 171C-2, and 171C-3 when the first display control unit 1103 switches images.

Next, the

content data

1711 and 1712 will be described with reference to FIG. FIG. 27 is an image diagram showing the data structure of content attribute data included in

content data

1711 and 1712 according to the present embodiment. More specifically, FIG. 27 shows a data structure of attribute data corresponding to a horizontal book.

The attribute data includes an area DA31 in which nothing is set, a content display direction DA32, and a content attribute DA33. The content display direction DA32 includes, for example, information “0” for designating horizontal writing and information “1” for designating vertical writing when the content is text.

The content attribute DA33 includes, for example, information “0” indicating that the content is a normal book, information “1” indicating that the content is a comic, and information “1” indicating that the content is a screen for inputting a search condition. 2 ”, information“ 3 ”indicating that the content is a search result list, information“ 4 ”indicating that the content is a search result, information“ 5 ”indicating that the content is a page image, and the like.

Referring back to FIG. 25, the reception unit 1101, the first display control unit 1103, and the second display control unit 1110 are functions realized by the CPU 110 and the like. More specifically, each function of the CPU 110 is a function realized by the CPU 110 executing a control program stored in the RAM 171 or the like and controlling each hardware shown in FIG.

Alternatively, any of the reception unit 1101, the second display control unit 1110, and the first display control unit 1103 may be realized by the CPU 210 or the like. More specifically, the CPU 210 executes a control program stored in the RAM 271 and the like to control each hardware shown in FIG. 10, whereby the receiving unit 1101, the second display control unit 1110, and the first display Any one of the control units 1103 may be realized.

First, the reception unit 1101 receives an operation command, a movement command, or the like input to the second display panel 240 based on an electrical signal input from the plurality of optical sensor circuits 244 via the image processing engine 280. More specifically, the reception unit 1101 acquires image data output from the image processing engine 280 of the second display panel 240 at every sensing time, and generates an operation command, a movement command, and the like based on the image data. To do.

The receiving unit 1101 may store the image data output from the image processing engine 280 in the RAM 271. That is, the reception unit 1101 may constantly update the image data in the RAM 271 to the latest image data. Note that the reception unit 1101 may have a function realized by the CPU 210 and the plurality of photosensor circuits 244 of the second display panel 240. That is, the accepting unit 1101 may be a concept indicating a functional block including a part of the function of the CPU 210 and the light receiving function of the second display panel 240.

That is, the reception unit 1101 detects a touch operation on the second display panel 240 based on the electrical signal from the second display panel 240. And the reception part 1101 recognizes the operation command from a user according to touch operation.

More specifically, the reception unit 1101 includes a direction determination unit 1104 that determines the moving direction of the contact position between the finger 900 detected via the second display panel 240 and the second display panel 240. The direction determination unit 1104 determines whether or not the contact position has moved to the right. The direction determination unit 1104 determines whether or not the contact position has moved leftward. Furthermore, the direction determination unit 1104 determines whether or not the contact position has moved downward. The direction determination unit 1104 determines whether or not the contact position has moved upward.

The reception unit 1101 includes a command determination unit 1105 that determines an operation command from the user based on the determination result from the direction determination unit 1104. As shown in FIG. 1B, the command determination unit 1105 determines that the first operation command has been received when it is determined that the contact position has moved to the right. When the command determination unit 1105 acquires the first operation command, the command determination unit 1105 delivers the first operation command to the first display control unit 1103 and the second display control unit 1110. As shown in FIG. 1D, the command determining unit 1105 determines that the second operation command has been received when it is determined that the contact position has moved leftward. When the command determination unit 1105 receives the second operation command, the command determination unit 1105 delivers the second operation command to the first display control unit 1103 and the second display control unit 1110.

Furthermore, as shown in FIGS. 4B and 7B, the command determination unit 1105 determines that the third operation command has been received when it is determined that the contact position has moved downward. When the command determination unit 1105 acquires the third operation command, the command determination unit 1105 delivers the third operation command to the first display control unit 1103 and the second display control unit 1110. As shown in FIGS. 4D and 7D, the command determination unit 1105 determines that the fourth operation command has been received when it is determined that the contact position has moved upward. When receiving the fourth operation command, the command determination unit 1105 transfers the fourth operation command to the first display control unit 1103 and the second display control unit 1110.

The first display control unit 1103 reads the content data 1711 from the RAM 171 based on the operation command (display command) from the reception unit 1101. The first display control unit 1103 displays content on the first display panel 140 based on the attribute data shown in FIG. For example, as shown in FIGS. 1A and 4A, the first display control unit 1103 causes the first display panel 140 to display the third page of text.

More specifically, the first display control unit 1103 designates a display target area of content to be displayed by the first display control unit 1103. That is, when the text is displayed in horizontal writing, the first display control unit 1103 receives the first operation command from the reception unit 1101 and displays the display target area (display) to be displayed by the first display control unit 1103. Move the target page back by one page. For example, the first display control unit 1103 receives a first operation command and generates information indicating a page, a line, and the like to be displayed by the first display control unit 1103. Alternatively, the first display control unit 1103 may generate a first command to display the page before the currently displayed page.

When the text is displayed in horizontal writing, the first display control unit 1103 receives a second operation command from the reception unit 1101 and displays a display target area (display target page) to be displayed by the first display control unit 1103. ) Go forward by one page. For example, the first display control unit 1103 receives the second operation command and generates information indicating a page, a line, and the like to be displayed by the first display control unit 1103. Alternatively, the first display control unit 1103 may generate a second command to display the next page after the currently displayed page.

On the other hand, when the text is displayed in horizontal writing, the first display control unit 1103 receives the third operation command from the reception unit 1101 and displays the display target area (display) to be displayed by the first display control unit 1103. Move the target page back by one line. For example, the first display control unit 1103 receives a first operation command and generates information indicating a page, a line, and the like to be displayed by the first display control unit 1103.

When the text is displayed in horizontal writing, the first display control unit 1103 receives a fourth operation command from the reception unit 1101 and displays a display target area (display target page) to be displayed by the first display control unit 1103. ) Go forward by one line. For example, the first display control unit 1103 receives a fourth operation command and generates information indicating a page, a line, and the like to be displayed by the first display control unit 1103.

The first display control unit 1103 displays a part of the text via the first display panel 140 based on the content data 1711 in accordance with the designation of the display target area (display target page). For example, the first display control unit 1103 displays the second page of text as shown in FIG. 1C in response to the first operation command (first command). The first display control unit 1103 displays the fourth page of the text as shown in FIG. 1E in response to the second operation command (second command). In response to the third operation command (third command), the first display control unit 1103 returns the text to be displayed by one line as shown in FIG. The first display control unit 1103 advances the text to be displayed by one line as illustrated in FIG. 2E in response to the fourth operation command (fourth command).

The first display control unit 1103 displays information (display mode data) indicating what type (attribute) of the content displayed via the first display panel 140 is the second display. The data is transferred to the control unit 1110. FIG. 28 is a first image diagram showing a data structure of display mode data according to the present embodiment. More specifically, FIG. 28 shows a data structure of display mode data when the first display control unit 1103 displays a horizontally written normal book via the first display panel 140.

The display mode data includes an area DA41 in which nothing is set and a content display mode DA42. The content display mode DA42 is, for example, information “0” indicating that the first display control unit 1103 has returned the display target area by one page, and the first display control unit 1103 advances the display target area by one page. Information “1” indicating “1”, information “2” indicating that the first display control unit 1103 has returned the display target area by one line, and indicating that the first display control unit 1103 has advanced the display target area by one line. Information “3”, information “4” indicating that the first display control unit 1103 has returned the search result, information “5” indicating that the first display control unit 1103 has advanced the search result , Information “6” indicating that the first display control unit 1103 has returned the selected search result row to the previous, and indicates that the first display control unit 1103 has advanced the selected search result row. Information “7”, the first display control unit 1103 Information indicating that did not returned before the Ceiling "8", etc. information "9" indicating that the first display control unit 1103 has not advanced beyond the content.

Note that the display command that the reception unit 1101 passes to the first display control unit 1103 has a data structure as shown in FIG. 29, for example. FIG. 29 is an image diagram showing a data structure of command data for display according to the present embodiment. More specifically, FIG. 29 shows a data structure of display command data when the first display control unit 1103 displays a horizontally written normal book via the first display panel 140.

The display command data includes an area DA51 in which nothing is set and a display command area DA52, similarly to the display mode data. The display instruction area DA52 includes, for example, information “0” indicating an instruction to return the display target area by one page, information “1” indicating an instruction to advance the display target area by one page, and information “1” indicating an instruction to return the display target area by one line. 2 ”, information“ 3 ”indicating an instruction to advance the display target area by one line, information“ 4 ”indicating an instruction to return the search result to the previous position, information“ 5 ”indicating an instruction to advance the search result forward, search being selected Information “6” indicating an instruction to return the result row to the front, information “7” indicating an instruction to advance the selected search result row, and the like are included.

The second display control unit 1110 receives the first operation command from the reception unit 1101 and the display mode data (text is displayed) from the first display control unit 1103, and performs the second display. An effect image for sending a page is displayed via the panel 240. More specifically, the second display control unit 1110 reads the effect image data group 171A corresponding to the display mode data in the first order in response to the first operation command, and applies the effect to the second display panel 240. The images are displayed in the first order.

The second display control unit 1110 receives the second operation command from the reception unit 1101 and the display mode data (that text is displayed) from the first display control unit 1103, and performs the second display. An effect image for returning the page is displayed via the panel 240. More specifically, the second display control unit 1110 reads the effect image data group 171A corresponding to the display mode data in the second order in accordance with the second operation command, and applies the effect to the second display panel 240. The images are displayed in the second order.

Further, the second display control unit 1110 receives the third operation command from the reception unit 1101 and the display mode data (text is displayed) from the first display control unit 1103, and receives the second operation command. The effect image for sending the page is displayed via the display panel 240. More specifically, the second display control unit 1110 reads the effect image data group 171B corresponding to the display mode data in accordance with the third operation command, and sequentially applies the effect images to the second display panel 240. Display.

The second display control unit 1110 receives the fourth operation command from the reception unit 1101 and the display mode data (that text is displayed) from the first display control unit 1103, and performs the second display. An effect image for returning the page is displayed via the panel 240. More specifically, the second display control unit 1110 reads the effect image data group 171C corresponding to the display mode data in accordance with the fourth operation command, and sequentially applies the effect images to the second display panel 240. Display.

When the first display control unit 1103 displays an image such as a photograph or a search result on the first display panel 140, the second display control unit 1110 displays the film on the second display panel 240. Display effect images such as cards and cards. Since the function of the second display control unit 1110 in such a case is the same as that in the case of the effect image of the book, description thereof will not be repeated here.

The accepting unit 1101 may deliver the display mode data to the first display control unit 1103 and the second display control unit 1110 in response to a touch operation on the second display panel 240. Then, the first display control unit 1103 may update the content to be displayed based on the display mode data from the reception unit 1101. The second display control unit 1110 may display the effect image based on the display mode data from the reception unit 1101.

In the present embodiment, the reception unit 1101, the first display control unit 1103, and the second display control unit 1110 have been described as different functional blocks. However, two or more of them may be combined into one functional block, or one of them may be divided into two or more functional blocks.

For example, the reception unit 1101 and the second display control unit 1110 may be configured as one module realized by the CPU 210, and the first display control unit 1103 may be configured as one module realized by the CPU 110.

<Operation overview when the text is displayed vertically>
Since electronic device 100 according to the present embodiment has the above-described configuration, it can also realize the following applied functions. Below, with reference to FIG. 30, FIG. 2, FIG. 3, the operation | movement outline | summary of the electronic device 100 which concerns on this Embodiment when a text is displayed vertically is demonstrated. Here, a case where electronic device 100 displays text in vertical writing will be described.

FIG. 30 is a second image diagram showing electronic apparatus 100 at the time of page switching processing according to the present embodiment. More specifically, FIG. 30A is an image diagram of the electronic device 100 in a state where the first display panel 140 displays the third page of text in vertical writing. FIG. 30B is an image diagram of the electronic device 100 when a right touch operation is input to the second display panel 240. FIG. 30C is an image diagram of the electronic device 100 in a state where the text is advanced by one page. FIG. 30D is an image diagram of the electronic device 100 when a left touch operation is input to the second display panel 240. FIG. 30E is an image diagram of the electronic device 100 in a state where the text is returned by one page.

Referring to FIG. 30 (A), electronic device 100 causes first page of text to be displayed on first display panel 140 in vertical writing. More specifically, the electronic device 100 accepts an operation from the user and causes the first display panel 140 to display a page desired by the user. Electronic device 100 accepts a page switching command from the user via second display panel 240.

Referring to FIG. 30 (B), electronic device 100 accepts a right touch operation from the user via second display panel 240. That is, the user slides the finger 900 or the stylus pen to the right while the finger 900 or a stylus pen (not shown) is in contact with the second display panel 240. More specifically, electronic device 100 receives a touch operation in the right direction by detecting a contact position between second display panel 240 and finger 900.

Referring to FIG. 30C, electronic device 100 causes first display panel 140 to display the fourth page of text. That is, when displaying the text in vertical writing, electronic device 100 advances the displayed text first (next) in response to a touch operation in the right direction. At this time, as shown in FIGS. 2A to 2H, the electronic device 100 according to the present embodiment is configured to perform the second processing based on the effect image data for books stored in advance in a RAM or the like. Are displayed in the first order on the display panel 240.

Referring to FIG. 30D, electronic device 100 accepts a leftward touch operation from the user via second display panel 240. That is, the user slides the finger 900 leftward with the finger 900 or the like being in contact with the second display panel 240. More specifically, electronic device 100 receives a touch operation in the left direction by detecting a contact position between second display panel 240 and finger 900.

Referring to FIG. 30 (E), electronic device 100 causes second page of text to be displayed on first display panel 140. That is, when displaying the text in vertical writing, electronic device 100 returns the displayed text to the front in response to a leftward touch operation. At this time, as shown in FIGS. 3A to 3H, the electronic device 100 according to the present embodiment performs the second processing based on the effect image data for books stored in advance in a RAM or the like. On the display panel 240, the effect image for turning the book is displayed in the reverse order (second order) of the first order.

As described above, electronic device 100 according to the present embodiment executes electronic device 100 according to an input operation when the text is displayed in vertical writing and when the text is displayed in horizontal writing. Processing is different. This configuration usually comes from the fact that pages in real vertical writing progress from right to left, and pages in real horizontal writing progress from left to right. That is, since the electronic device 100 realizes a virtual book (text), it is preferable that the user's operation and the processing of the electronic device 100 are associated with each other based on the arrangement of pages in the actual book. .

When electronic device 100 according to the present embodiment receives a touch operation in the right direction on first display panel 140, second display panel 240 does not display horizontally or vertically. The effect image shown in FIG. 2 is displayed. On the other hand, when a right touch operation is input to the first display panel 140, the electronic device 100 displays different pages on the first display panel 140 for horizontal writing and vertical writing. Let

As a result, in electronic device 100 according to the present embodiment, the user can easily associate operations performed on the electronic device with processing executed in the electronic device.

<Operation overview when the search result list is displayed>
Furthermore, electronic device 100 according to the present embodiment can also realize the following applied functions. Hereinafter, with reference to FIG. 31 to FIG. 33, an outline of operation of electronic device 100 according to the present embodiment when first display panel 140 displays a search result list will be described.

FIG. 31 is an image diagram showing the electronic device 100 when the search result being selected according to the present embodiment is switched. More specifically, FIG. 31A is an image diagram of the electronic device 100 in a state where the first display panel 140 displays the search result list. FIG. 31B is an image diagram of the electronic device 100 when an upward touch operation is input to the second display panel 240. FIG. 31C is an image diagram of electronic device 100 in a state where the previous search result is selected. FIG. 31D is an image diagram of the electronic device 100 when a downward touch operation is input to the second display panel 240. FIG. 31E is an image diagram of the electronic device 100 in a state where the next search result is selected.

FIG. 32 is a first image diagram showing the second display panel 240 when the search result being selected according to the present embodiment is switched. FIG. 33 is a second image diagram showing the second display panel when the search result being selected according to the present embodiment is switched.

Referring to FIG. 31A, electronic device 100 causes first display panel 140 to display the search result list in horizontal writing. More specifically, electronic device 100 according to the present embodiment has an electronic dictionary function. The electronic device 100 receives a search condition from the user, and causes the first display panel 140 to display a search result that matches the search condition.

For example, the accepting unit 1101 accepts input of a character string via the second display panel 240 or the operation key 177. The first display control unit 1103 refers to the database of the RAM 171 and causes the first display panel 140 to display a list of headings including the character string as search results. That is, the first display control unit 1103 causes the first display panel 140 to display a search result list including at least one search result. At this time, the first display control unit 1103 puts one search result in the search result list into a selected state, for example, by highlighting it.

Referring to FIG. 31B, electronic device 100 accepts an upward touch operation from the user via second display panel 240. That is, the user slides the finger 900 upward in a state where the finger 900 or the like is in contact with the second display panel 240. More specifically, electronic device 100 accepts an upward touch operation by detecting a contact position between second display panel 240 and finger 900.

Referring to FIG. 31C, electronic device 100 causes first display panel 140 to display the previous search result in a selected state. In other words, when displaying the search result list, electronic device 100 returns the selected search result to the previous one in response to an upward touch operation. At this time, as shown in FIGS. 32A to 32C, electronic device 100 according to the present embodiment has an effect image data group in which the search results being selected that are stored in advance in RAM or the like are switched. Based on this, the second display panel 240 sequentially displays effect images in which the selected search results are switched.

Referring to FIG. 31D, electronic device 100 accepts a downward touch operation from the user via second display panel 240. That is, the user slides the finger 900 downward with the finger 900 or the like being in contact with the second display panel 240. More specifically, electronic device 100 accepts a downward touch operation by detecting a contact position between second display panel 240 and finger 900.

Referring to FIG. 31 (E), electronic device 100 causes first display panel 140 to display the next search result in a selected state. That is, when displaying the search result list, electronic device 100 advances the selected search result by one in response to a downward touch operation. At this time, as shown in FIGS. 33A to 33C, electronic device 100 according to the present embodiment has an effect image data group in which the search results being selected stored in advance in RAM or the like are switched. Based on this, the second display panel 240 sequentially displays effect images in which the selected search results are switched.

As described above, electronic device 100 according to the present embodiment displays different effect images on second display panel 240 when the search result selection is advanced and when the search result is returned. In particular, the electronic device 100 sequentially displays previously stored effect images via a display area (second display area) different from the content display area (first display area). There is no need to perform image processing according to the headline. As a result, the electronic device 100 can make the user intuitively recognize the progress state of the content with a simple process.

<Overview of operation when detailed screen of search results is displayed>
Furthermore, electronic device 100 according to the present embodiment can also realize the following applied functions. Hereinafter, with reference to FIGS. 34 to 36, an outline of operation of electronic device 100 according to the present embodiment when first display panel 140 displays a detailed screen of search results will be described.

FIG. 34 is an image diagram showing an electronic device during search result page switching processing according to the present embodiment. More specifically, FIG. 34A is an image diagram of the electronic device 100 in a state where the first display panel 140 displays a detailed search result screen. FIG. 34B is an image diagram of the electronic device 100 when a touch operation in the upper right direction is input to the second display panel 240. FIG. 34C is an image diagram of the electronic device 100 in a state where the previous search result is displayed. FIG. 34D is an image diagram of the electronic device 100 when a lower left touch operation is input to the second display panel 240. FIG. 34E is an image diagram of the electronic device 100 in a state in which the next search result is displayed.

FIG. 35 is a first image diagram showing the second display panel during the search result page switching process according to the present embodiment. FIG. 36 is a second image diagram showing the second display panel during the search result page switching process according to the present embodiment.

Referring to FIG. 34A, electronic device 100 causes first display panel 140 to display detailed search results in horizontal writing. As described above, electronic device 100 according to the present embodiment has an electronic dictionary function. The electronic device 100 accepts a search result selection command from the user and displays the detailed contents of the search result on the first display panel 140.

For example, in the state shown in FIG. 31A, FIG. 31C, or FIG. 31E, the accepting unit 1101 accepts an instruction via the second display panel 240 or the operation key 177. Then, the first display control unit 1103 refers to the database of the RAM 171 and causes the first display panel 140 to display detailed search results as shown in FIG.

Referring to FIG. 34 (B), electronic device 100 accepts a touch operation in the upper right direction from the user via second display panel 240. That is, the user slides the finger 900 in the upper right direction with the finger 900 or the like being in contact with the second display panel 240. More specifically, electronic device 100 accepts a touch operation in the upper right direction by detecting a contact position between second display panel 240 and finger 900.

Referring to FIG. 34C, electronic device 100 causes first display panel 140 to display the detailed contents of the next search result. That is, when displaying the detailed content of the search result, electronic device 100 advances the displayed search result by one in response to a touch operation in the upper right direction. At this time, as shown in FIGS. 35 (A) to 35 (E), electronic device 100 according to the present embodiment turns effect image data for turning one card indicating a search result stored in advance in a RAM or the like. Based on the group, the second display panel 240 displays an effect image for turning one card indicating the search result.

Referring to FIG. 34 (D), electronic device 100 accepts a touch operation in the lower left direction from the user via second display panel 240. That is, the user slides the finger 900 in the lower left direction with the finger 900 or the like being in contact with the second display panel 240. More specifically, electronic device 100 accepts a touch operation in the lower left direction by detecting a contact position between second display panel 240 and finger 900.

Referring to FIG. 34 (E), electronic device 100 causes first display panel 140 to display the detailed contents of the previous search result. That is, when the detailed content of the search result is displayed, electronic device 100 restores only one search result being displayed in response to a touch operation in the lower left direction. At this time, as shown in FIGS. 36 (A) to 36 (E), electronic device 100 according to the present embodiment covers effect image data on which one card indicating a search result stored in advance in a RAM or the like is placed. Based on the group, the second display panel 240 displays an effect image that covers one card indicating the search result.

Thus, electronic device 100 according to the present embodiment displays a series of different effect images on second display panel 240 when the detailed search result being displayed is advanced and when it is returned. In particular, the electronic device 100 sequentially displays previously stored effect images via a display area (second display area) different from the content display area (first display area). There is no need to perform image processing according to the contents. As a result, the electronic device 100 can make the user intuitively recognize the progress state of the content with a simple process.

<Overview of operation when a map image is displayed>
Furthermore, the electronic device 100 according to the present embodiment can also realize the following applied functions. Hereinafter, with reference to FIGS. 37 to 39, an outline of operation of electronic device 100 according to the present embodiment when first display panel 140 displays a map image will be described.

FIG. 37 is an image diagram showing electronic device 100 at the time of the switching process of the map display area according to the present embodiment. More specifically, FIG. 37A is an image diagram of the electronic device 100 in a state where the first display panel 140 displays the first area of the map image. FIG. 37B is an image diagram of the electronic device 100 when a right touch operation is input to the second display panel 240. FIG. 37C is an image diagram of the electronic device 100 in a state where the first display panel 140 displays the second area of the map image. FIG. 37D is an image diagram of the electronic device 100 when a downward touch operation is input to the second display panel 240. FIG. 37E is an image diagram of the electronic device 100 in a state where the first display panel 140 displays the third area of the map image.

FIG. 38 is a first image diagram showing the second display panel 240 during the map display area switching process according to the present embodiment. FIG. 39 is a second image diagram showing the second display panel during the map display area switching process according to the present embodiment.

Referring to FIG. 37 (A), electronic device 100 causes first display panel 140 to display the first area of the map image. More specifically, electronic device 100 according to the present embodiment receives a display position designation command from the user, and causes first display panel 140 to display a first area centered on the display position.

Referring to FIG. 37 (B), electronic device 100 receives a touch operation in the right direction from the user via second display panel 240. That is, the user slides the finger 900 rightward in a state where the finger 900 or the like is in contact with the second display panel 240. More specifically, electronic device 100 receives a touch operation in the right direction by detecting a contact position between second display panel 240 and finger 900.

Referring to FIG. 37 (C), electronic device 100 causes first display panel 140 to display a second area located on the left side of the first area in the map image. That is, when displaying the map image, electronic device 100 moves the displayed area to the left in response to a touch operation in the right direction. At this time, as shown in FIGS. 38A to 38C, electronic device 100 according to the present embodiment has effect image data in which a map sheet stored in advance in RAM or the like moves to the right. Based on the group, the second display panel 240 displays an effect image in which the map sheet moves to the right.

37 (D), electronic device 100 accepts a downward touch operation from the user via second display panel 240. That is, the user slides the finger 900 downward with the finger 900 or the like being in contact with the second display panel 240. More specifically, electronic device 100 accepts a downward touch operation by detecting a contact position between second display panel 240 and finger 900.

Referring to FIG. 37 (E), electronic device 100 causes first display panel 140 to display a third area located above the first area of the map image. That is, when displaying the map image, electronic device 100 moves the displayed area upward in response to a downward touch operation. At this time, as shown in FIGS. 39A to 39C, the electronic device 100 according to the present embodiment has effect image data in which a map sheet stored in advance in a RAM or the like moves downward. Based on the group, an effect image in which the map sheet moves downward is displayed on the second display panel 240.

Here, the case where the electronic device 100 receives a right touch operation and a downward touch operation has been described, but the above description also applies when the electronic device 100 receives a left touch operation and an upward touch operation. Works as well. Since the operation when electronic device 100 accepts a leftward touch operation is the same as the operation when a rightward touch operation is accepted, description thereof will not be repeated here. The operation when electronic device 100 accepts an upward touch operation is the same as the operation when a downward touch operation is accepted, and thus the description thereof will not be repeated here.

The electronic device 100 according to the present embodiment moves the map image by an amount corresponding to half of the display area according to the direction of the touch operation, but is not limited to such a configuration. For example, the electronic device 100 may move the map image by an amount corresponding to the entire display area.

Furthermore, the electronic device 100 may change the movement amount of the map image according to the distance of the touch operation. In this case, the reception unit 1101 or the direction determination unit 1104 includes a distance determination unit, and the distance determination unit determines whether the movement distance of the touch operation is equal to or greater than a predetermined value. Then, the command determination unit 1105 receives a display area switching command from the user based on the determination results from the direction determination unit 1104 and the distance determination unit.

As described above, electronic device 100 according to the present embodiment displays different effect images on second display panel 240 in accordance with the direction in which the map image display area moves. In particular, the electronic device 100 sequentially displays the effect images stored in advance through a display area (second display area) different from the map image display area (first display area). It is not necessary to perform image processing according to the contents of As a result, the electronic device 100 can make the user intuitively recognize the progress state of the content with a simple process.

<Content display processing>
Next, content display processing in electronic device 100 according to the present embodiment will be described with reference to FIGS. FIG. 40 is a flowchart showing a processing procedure of content display processing in electronic device 100 according to the present embodiment.

First, the CPU 110 functioning as the first display control unit 1103 accepts a command for displaying an electronic book from the user via the second display panel 240 or the operation key 177 and receives content data (book data) from the RAM 171. Is read (step S102). CPU 110 creates initial display layout information based on the content data (step S104). CPU 110 recognizes the display type (first display type) based on the content data (step S106). More specifically, the CPU 110 functioning as the first display control unit 1103 notifies the CPU 210 (or CPU 110) functioning as the second display control unit 1110 of the first display type.

CPU 110 displays content on first display panel 140 based on the layout information (step S108). CPU 110 determines whether or not a touch operation (for example, a page switching command) has been received via second display panel 240 (step S110). CPU110 repeats the process of step S110, when a touch operation is not received (when it is NO in step S110).

When CPU 110 accepts the touch operation (YES in step S110), CPU 110 creates new layout information in accordance with the touch operation based on the content data (step S112). CPU 110 recognizes a new display type (second display type) based on the content data (step S114). More specifically, the CPU 110 functioning as the first display control unit 1103 notifies the CPU 210 functioning as the second display control unit 1110 of the second display type. The CPU 110 redisplays the content via the first display panel 140 based on the new layout information.

On the other hand, after step S106, the CPU 210 functioning as the second display control unit 1110 stores (updates) the first display type in the memory area used by itself (step S118). The CPU 210 selects an operation screen based on the first display type (step S120), and displays the operation screen via the second display panel 240 (step S122). For example, when the first display panel 140 displays text, the CPU 210 causes the second display panel 240 to display a book image as shown in FIG. When the first display panel 140 displays an image, the CPU 210 causes the second display panel 240 to display a film image as shown in FIG.

CPU 210 (or CPU 110) functioning as reception unit 1101 determines whether or not a touch operation has been received via second display panel 240 (step S124). CPU210 repeats the process of step S124, when a touch operation is not received (when it is NO in step S124).

CPU210 acquires a 1st-4th operation command based on a touch operation, when a touch operation is received (when it is YES in step S124) (step S126). The CPU 210 delivers the first to fourth operation commands to the CPU 110 functioning as the first display control unit 1103 (step S128).

The CPU 210 functioning as the second display control unit 1110 extracts the effect image data group 171A corresponding to the first display type from the RAM 171 (step S130). As shown in FIG. 2, the CPU 210 sequentially displays effect images on the second display panel 240 (step S132). The CPU 210 compares the first display type with the second display type (step S134). The CPU 210 determines whether or not the display types are different (step S136). That is, the CPU 210 determines whether or not the content type has been switched.

CPU210 repeats the process from step S124, when a display classification is the same (when it is NO in step S136). If the display types are different (YES in step S136), CPU 210 updates the first display type to the second display type (step S138). The CPU 210 selects an operation screen based on the updated second display type (step S140), and displays the operation screen via the second display panel 240 (step S142). CPU110 repeats the process from step S124.

<Modification of processing procedure>
Next, as a modification of the processing procedure in the present embodiment, the processing procedure in electronic device 100 in the case where a plurality of types of

content data

1711 and 1712 such as text and comics are included in one book data (content group). Will be described in more detail. Below, the data structure of book data is demonstrated. FIG. 41 is a first image diagram illustrating a data structure of book data 171X including a plurality of types of contents. FIG. 42 is a second image diagram illustrating a data structure of book data 171X including a plurality of types of contents.

Referring to FIG. 41, each of the book data 171X includes first information 171Y and second information 171Z. Note that the book data 171X is stored in the RAM 171 or the like. The first information 171Y includes information related to the entire content. The first information 171Y includes information indicating the type of data present in the content, information indicating the number of data, and information indicating an initial layout when the book is opened. The second information 171Z includes information indicating a pointer to an area that holds each layout of data and information indicating the type of content.

Or, referring to FIG. 42, the book data 171X alternately includes information indicating the type (attribute) of the content and information necessary for laying out the content. That is, information indicating the type of content and information necessary for laying out the content are included in units of layout (for each pointer).

For example, the book data 171X includes type data DA61 indicating text and layout data DA62 as the first content data. Layout data DA62 includes text data. The book data 171X includes type data DA63 indicating that it is a comic and layout data DA64 as the second content data. Layout data DA64 includes image data. The book data 171X includes type data DA65 indicating comics and layout data DA66 as third content data. Layout data DA66 includes image data.

Next, with reference to FIG. 1 to FIG. 39 and FIG. 41 to FIG. 45, a process for switching and displaying a plurality of types of contents in electronic device 100 according to the present embodiment will be described in more detail. FIG. 43 is a first flowchart illustrating a processing procedure of processing for switching and displaying a plurality of types of content in electronic device 100 according to the present modification. FIG. 44 is a second flowchart illustrating a processing procedure of processing for switching and displaying a plurality of types of content in electronic device 100 according to the present modification. FIG. 45 is a third flowchart illustrating a processing procedure of processing for switching and displaying a plurality of types of content in electronic device 100 according to the present modification.

Referring to FIG. 43, CPU 110 functioning as first display control unit 1103 accepts a command for displaying an electronic book from the user via second display panel 240 or operation key 177 and reads the book from RAM 171. Read data 171X. CPU110 acquires layout data DA62 of 1st content data from book data 171X (step S202).

CPU 110 creates an initial display layout (first layout information) based on layout data DA62 (step S204). Based on the first content data type data DA61, the CPU 110 recognizes that the first display type corresponding to the first content is text (step S206). More specifically, the CPU 110 functioning as the first display control unit 1103 notifies the CPU 210 (or CPU 110) functioning as the second display control unit 1110 of the first display type. CPU110 displays a text on the 1st display panel 140 based on layout information (step S208).

On the other hand, after step S206, the CPU 210 functioning as the second display control unit 1110 sets (updates) the first display type in the memory area used by itself (step S218). The CPU 210 selects an operation screen based on the first display type (step S220), and displays the operation screen via the second display panel 240 (step S222). Here, since the first display panel 140 displays text, the CPU 210 causes the second display panel 240 to display a book image as shown in FIG.

Next, referring to FIG. 44, CPU 110 determines whether or not a touch operation (for example, a page switching command) has been received via second display panel 240 (step S210). CPU110 repeats the process of step S210, when a touch operation is not received (when it is NO in step S210).

CPU110 produces the layout information of the following page, when a touch operation is received (when it is YES in step S210) (step S212). That is, the CPU 110 creates the second layout information based on the layout data DA64 of the second content data. The CPU 110 recognizes a new display type (second display type) based on the second content data type data DA63 (step S214). More specifically, the CPU 110 functioning as the first display control unit 1103 notifies the CPU 110 functioning as the second display control unit 1110 of the second display type. The CPU 110 displays the content again via the first display panel 140 based on the second layout information (step S216).

On the other hand, the CPU 210 (or the CPU 110) functioning as the reception unit 1101 determines whether or not a touch operation has been received via the second display panel 240 (step S224). CPU210 repeats the process of step S224, when a touch operation is not received (when it is NO in step S224).

CPU210 acquires a 1st-4th operation command based on a touch operation, when a touch operation is received (when it is YES in step S224) (step S226). When the first display panel 140 displays text (usually a book) in horizontal writing, the CPU 210 acquires a page turn command (second operation command) based on a leftward touch operation. The CPU 210 delivers a page feed command to the CPU 110 functioning as the first display control unit 1103 (step S228).

The CPU 210 functioning as the second display control unit 1110 extracts the effect image data group 171A corresponding to the text from the RAM 171 (step S230). As shown in FIGS. 3A to 3H, the CPU 210 sequentially displays the effect images whose pages are turned on the second display panel 240 (step S232). CPU 210 compares the first display type corresponding to the text with the second display type corresponding to the comic (step S234).

Referring to FIG. 45, CPU 210 determines whether or not the display types are different (step S236). CPU210 repeats the process from step S224, when a display classification is the same (when it is NO in step S236).

Here, since the first display type is different from the second display type (in the case of YES in step S236), the display type corresponding to the currently displayed content is updated to the second display type corresponding to the comic. (Step S238). The CPU 210 selects an operation screen corresponding to the comic (step S240), and displays an operation screen corresponding to the comic as shown in FIG. 8 via the second display panel 240 (step S242). CPU110 repeats the process from step S224.

<First additional function>
Next, a first additional function of electronic device 100 according to the present embodiment will be described with reference to FIGS. 46A and 46B. 46A and 46B are image diagrams showing a second display panel showing the first additional function of electronic device 100 according to the present embodiment.

The electronic device 100 according to the present embodiment displays a manual image for advising the operation method on the second display panel 240 when, for example, text is displayed on the first display panel 140.

For example, in the state shown in FIGS. 2 and 4, as shown in FIG. 46A, the second display control unit 1110 of the electronic device 100 returns the text by one page via the second display panel 240. An arrow indicating the direction of the touch operation, an arrow indicating the direction of the touch operation for advancing the text by one page, an arrow indicating the direction of the touch operation for advancing the text by one line, and a touch operation for advancing the text by one line An arrow indicating the direction of the is displayed.

For example, in the state illustrated in FIG. 30, as illustrated in FIG. 46B, the second display control unit 1110 of the electronic device 100 performs a touch operation for returning the text by one page via the second display panel 240. An arrow indicating the direction, an arrow indicating the direction of the touch operation for advancing the text by one page, an arrow indicating the direction of the touch operation for advancing the text by one line, and the direction of the touch operation for advancing the text by one line Display arrows and so on.

Thereby, the user can easily know the touch operation necessary for causing the electronic device 100 to execute a desired process.

<Second additional function>
Next, a second additional function of electronic device 100 according to the present embodiment will be described with reference to FIG. FIG. 47 is a conceptual diagram showing a second display panel showing a second additional function of electronic device 100 according to the present embodiment.

In the

content data

1711 and 1712 stored in the electronic device 100, an event such as a URL (Uniform Resource Locator) indicating a web page address or a link jump to another page in the same book data 171X may be set. is there. For example, the above event may be set in a character string or an image included in the text.

Creators of content often want to display the content they created without affecting the above event information. In order to inform the user that the above event is set in a character string or image included in text, a normal electronic device adds a “ruled line” to the character string or displays the image as a “thick frame”. Or surround with.

The electronic device 100 according to the present embodiment performs the following operation when the above event is set in a character string or an image included in the text. That is, as shown in FIG. 47, the electronic device 100 has a case where the pointer is positioned on the character string or the image, or when the character string or the image is positioned at the center of the first display panel 140. The fact that an event is set for them is displayed via the second display panel 240. For example, the electronic device 100 may display a character string indicating the content of the event on the second display panel 240 or display an exclamation mark indicating that the event is set on the second display panel 240. May be.

Electronic device 100 accepts execution from the user via second display panel 240 or operation key 177 when second display panel 240 displays a character string or an exclamation mark indicating the content of the event. The process corresponding to the event is executed.

<Third additional function>
Next, with reference to FIG. 48, the 3rd additional function of the electronic device 100 which concerns on this Embodiment is demonstrated. FIG. 48 is an image diagram showing a second display panel showing a third additional function of electronic device 100 according to the present embodiment.

The electronic device 100 sets a bookmark position in the book data 171X (or the content data 1711 and 1712). Electronic device 100 may set a bookmark at a predetermined position in book data 171X (or content data 1711 and 1712) in advance, or may set a bookmark position based on a command from the user.

48. As shown in FIG. 48, the electronic device 100 causes the second display panel 240 to display an image of a book in which a bookmark is sandwiched at a location corresponding to the bookmark position. Electronic device 100 accepts a touch operation on a bookmark image via second display panel 240. In response to the touch operation, electronic device 100 displays a page corresponding to the bookmark position on first display panel 140.

<Fourth additional function>
Next, with reference to FIG. 49, the 4th additional function of the electronic device 100 which concerns on this Embodiment is demonstrated. FIG. 49 is an image diagram showing a second display panel showing a fourth additional function of electronic device 100 according to the present embodiment.

When electronic device 100 accepts an instruction to move to the next page (next line) while first display panel 140 displays the last page of the book, electronic device 100 may not proceed to the next page. The first screen shown is displayed on the second display panel 240. Conversely, when electronic device 100 accepts a command to move to the previous page (previous row) while first display panel 140 displays the first page of the book, electronic device 100 advances to the previous page. A second screen indicating the absence is displayed on the second display panel 240.

<Modification of display area>
Electronic device 100 according to the present embodiment displays content via first display panel 140 and displays an operation screen and effect images via second display panel 240. That is, the first display panel 140 realizes a first display area for displaying content, and the second display panel 240 realizes a second display area for displaying an operation screen and an effect image. However, it is not limited to such a configuration.

FIG. 50 is an image diagram showing a first display panel according to a modification of electronic device 100 according to the present embodiment. Referring to FIG. 50, in the present modification, first display panel 140 realizes a first display area for displaying content and a second display area for displaying an operation screen and an effect image.

The electronic device according to this modification uses the first display panel 140 capable of detecting a touch operation, so that the right side of the first display panel 140 functions as a first display area for displaying content. The left side of one display panel 140 functions as a second display area for displaying an operation screen and an effect image. That is, the electronic device may not have the second display panel 240.

More specifically, electronic device 100 displays content in the first display area of first display panel 140 and also displays effect image 140A in the second display area of first display panel 140. Good. The electronic device 100 may accept touch operations corresponding to the first to fourth operation commands via the second display area of the first display panel 140.

Furthermore, in the electronic device 100 according to the above-described embodiment, the second display area (second display panel 240) detects a touch operation, but receives a page switching command via the operation key 177 or the like. It may be a thing. That is, even if one or both of the first display panel 140 and the second display panel 240 is an electronic device that cannot detect a touch operation, the technology of the electronic device 100 according to the above embodiment can be used. .

<Other embodiments>
It goes without saying that the present invention can also be applied to a case where it is achieved by supplying a program to a system or apparatus. Then, a storage medium storing a program represented by software for achieving the present invention is supplied to the system or apparatus, and the computer (or CPU or MPU) of the system or apparatus stores the program code stored in the storage medium It is possible to enjoy the effects of the present invention also by reading and executing.

In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiment, and the storage medium storing the program code constitutes the present invention.

As a storage medium for supplying the program code, for example, hard disk, optical disk, magneto-optical disk, CD-ROM, CD-R, magnetic tape, nonvolatile memory card (IC memory card), ROM (mask ROM, flash) EEPROM, etc.) can be used.

Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an OS (operating system) running on the computer based on the instruction of the program code However, it is needless to say that a case where the function of the above-described embodiment is realized by performing part or all of the actual processing and the processing is included.

Further, after the program code read from the storage medium is written to a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, the function expansion is performed based on the instruction of the program code. It goes without saying that the CPU or the like provided in the board or the function expansion unit performs part or all of the actual processing and the functions of the above-described embodiments are realized by the processing.

The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

100 electronic device, 101 main unit, 102, 103 display device, 110 CPU, 1101 accepting unit, 1103 first display control unit, 1104 direction judging unit, 1105 command judging unit, 1110 second display control unit, 130 driver, 131 Scanning signal line driving circuit, 132 Data signal line driving circuit, 133 Photo sensor driving circuit, 134 Switch, 135 amplifier, 140 First liquid crystal panel with built-in photo sensor (first display panel), 141, 241 Pixel circuit, 141b , 141g, 141r subpixel circuit, 143 electrode pair, 143a pixel electrode, 143b counter electrode, 144, 244 photosensor circuit, 145, 145b, 145g, 145r photodiode, 146 capacitor, 151A active matrix substrate, 1 1B counter substrate, 152 liquid crystal layer, 153b, 153g, 153r color filter, 157 data signal line, 161 polarizing filter, 162 glass substrate, 163 light shielding film, 164 alignment film, 171 RAM, 1711, 1712 content data, 171A, 171B effect Image data group, 173, memory card reader / writer, 174 communication device, 175 microphone, 176 speaker, 177 operation key, 179 backlight, 180 image processing engine, 181 driver control unit, 182 timer, 183 signal processing unit, 240 second Photosensor built-in liquid crystal panel (second display panel), 1001 first unit, 1002 second unit, 1731 memory card, DB1, DB2 data bus.

Claims

First and second display areas (140, 240);
A memory (171) for storing at least one content (1711) and at least one image group (171A);
An accepting unit (1101) for accepting one of the first and second instructions;
The display target area of the content is displayed via the first display area, the display target area of the content is advanced when the first command is received, and the content is displayed when the second command is received. A first display control unit (1103) for returning the display target area of the content;
The first effect is output via the second display area based on the at least one image group when the first command is received, and the at least one when the second command is received A content display device comprising: a second display control unit (1110) for outputting a second effect via the second display area based on an image group.
The second display controller is
When the first instruction is received, the first effect is output by causing the second display area to display any one of the at least one image group in a first order,
When the second command is received, the second effect is displayed by causing the second display area to display any one of the at least one image group in a second order opposite to the first order. The content display device according to claim 1, wherein:
The at least one image group includes a first image group and a second image group;
The second display controller is
When the first command is received, the first effect is output by sequentially displaying the first image group in the second display area,
The content display according to claim 1, wherein when receiving the second command, the second effect is output by sequentially displaying the second group of images in the second display area. apparatus.
A touch panel (140, 240) including the second display area;
The content display device according to claim 1, wherein the reception unit receives one of the first and second instructions based on a touch operation on the second display area.
The content display device according to claim 4, further comprising a display (140, 240) including the first display area.
The content display device according to claim 4, wherein the touch panel includes the first display area.
The memory stores each of the plurality of image groups in association with an attribute, and stores each of the plurality of contents in association with the attribute,
The content display according to claim 1, wherein the second display control unit displays the image group corresponding to the content displayed in the first display area in the second display area. apparatus.
The memory stores a plurality of the contents as a series of content groups,
The content display device according to claim 7, wherein the first display control unit displays each of the contents based on the corresponding attribute with reference to the series of content groups.
A computer that records a content display program for displaying content on a computer including an arithmetic processing unit, first and second display areas, and a memory for storing at least one content and at least one image group A readable recording medium,
The content display program is stored in the arithmetic processing unit.
Receiving either of the first and second instructions;
Displaying a display target area of the content via the first display area;
Advancing the display target area of the content when receiving the first instruction;
Returning the display target area of the content when the second command is received;
Outputting a first effect via the second display area based on the at least one image group when receiving the first command;
A computer-readable recording of a content display program for executing a step of outputting a second effect via the second display area based on the at least one image group when the second command is received Recording medium.
The step of outputting the first effect includes a step of displaying any of the at least one image group in the first display order in the second display area,
The step of outputting the second effect includes a step of causing the second display area to display any one of the at least one image group in a second order opposite to the first order. A computer-readable recording medium for recording the content display program according to item 9.
The at least one image group includes a first image group and a second image group;
The step of outputting the first effect includes a step of sequentially displaying the first group of images in the second display area,
The computer-readable recording medium according to claim 9, wherein the step of outputting the second effect includes a step of sequentially displaying the second group of images in the second display area. Recording medium.
The computer further includes a touch panel including the second display area,
The step of accepting any of the first and second instructions includes a step of accepting any of the first and second instructions based on a touch operation on the second display area. A computer-readable recording medium for recording the content display program according to item 9.
The computer-readable recording medium for recording the content display program according to claim 12, further comprising a display including the first display area.
13. The computer-readable recording medium for recording the content display program according to claim 12, wherein the touch panel includes the first display area.
The memory stores each of the plurality of image groups in association with an attribute, and stores each of the plurality of contents in association with the attribute,
Outputting the first effect includes displaying the image group corresponding to the content displayed in the first display area in the second display area;
The step of outputting the second effect includes the step of displaying the image group corresponding to the content displayed in the first display area in the second display area. A computer-readable recording medium for recording the content display program described in 1.
The memory stores a plurality of the contents as a series of content groups,
The step of displaying the display target area includes a step of displaying each of the contents in the first display area based on the corresponding attribute with reference to the series of contents group. A computer-readable recording medium for recording the content display program according to the item.
A content display method in a computer including an arithmetic processing unit, first and second display areas, and a memory for storing at least one content and at least one image group,
The arithmetic processing unit accepting either of the first and second instructions;
The arithmetic processing unit displaying a display target area of the content via the first display area;
A step of advancing the display target area of the content when the arithmetic processing unit receives the first instruction;
Returning the display target area of the content when the arithmetic processing unit receives the second command;
Outputting a first effect through the second display area based on the at least one image group when the arithmetic processing unit receives the first command;
A content display method comprising: a step of outputting a second effect via the second display area based on the at least one image group when the arithmetic processing unit receives the second command.