JP5154368B2 - COMMUNICATION DEVICE AND ITS CONTROL METHOD - Google Patents

Description

  The present invention relates to a communication apparatus that performs communication using a plurality of communication means.

  In order to share content (such as image data) between users, the content is often transmitted and received between communication devices. In that case, some content processing may be performed, the processed content may be saved as a file, and the file may be transferred for sharing.

  In order to transmit and receive content more reliably and more conveniently, a communication device including a plurality of communication means has been proposed.

  Patent Documents 1 to 4 disclose an information processing apparatus including a main display device and a sub display device.

In addition, a personal computer (for example, DynaBook G8 series (manufactured by Toshiba)) equipped with a clear pad whose touch pad is a liquid crystal display is commercially available.
JP 2004-5105 A (published January 8, 2004) JP 2004-5212 A (published January 8, 2004) JP 2004-234504 A (published on August 19, 2004) JP 2000-339097 A (released on December 8, 2000)

  However, in a communication device including a communication unit that can operate independently, it has not been proposed so far how to link these communication units to realize an efficient communication mode.

  The present invention has been made to solve the above problems, and an object of the present invention is to provide a communication apparatus capable of realizing an efficient communication mode in which a plurality of independent communication units are linked. is there.

  In order to solve the above problems, a communication device according to the present invention is a communication device including a first communication unit and a second communication unit that can operate independently of the first communication unit. The first communication unit includes a storage unit that stores address information in which identification information of a counterpart device that is a communication destination device is associated with a user name of the counterpart device, and the second communication unit communicates with itself. When a possible counterpart device is detected and the identification information of the counterpart device is obtained, the target user name that is the name of the user corresponding to the identification information is requested to the first communication unit, and the first communication unit , By referring to the address information stored in the storage unit, the information including the target user name requested from the second communication unit is transmitted to the second communication unit.

  In order to solve the above-described problem, a control method according to the present invention controls a communication apparatus including a first communication unit and a second communication unit that can operate independently of the first communication unit. The first communication unit includes a storage unit that stores address information in which identification information of a counterpart device that is a communication destination device is associated with a user name of the counterpart device, and the second communication unit. Requesting the first communication unit to request the target user name, which is the name of the user corresponding to the identification information, when a partner device communicable with itself is detected and the identification information of the partner device is acquired. And the first communication unit refers to the address information stored in the storage unit, thereby obtaining information including the target user name requested from the second communication unit. It is characterized in that it comprises a target user name transmission step of transmitting.

  According to the above configuration, the first communication unit and the second communication unit can operate independently of each other. The second communication unit detects a partner device that can communicate with the device itself, and acquires identification information (for example, Bluetooth (registered trademark) device address) of the partner device.

  Since the first communication unit includes the storage unit that stores the address information in which the identification information of the counterpart device and the name of the user of the counterpart device are associated with each other, the second communication unit corresponds to the acquired identification information. A request for acquiring a user name (target user name) is transmitted to the first communication unit. In this case, the second communication unit may request the entire address information or may request partial address information including at least the target user name.

  When the request is received from the second communication unit, the first communication unit refers to the address information stored in the storage unit, thereby obtaining information including the target user name requested from the second communication unit. Send to.

  The second communication unit can notify the user by displaying the target user name transmitted from the first communication unit on the display unit.

  Therefore, even when the second communication unit does not include a storage unit that stores address information, by using the storage unit included in the first communication unit, it is possible to communicate with the counterpart device, and a plurality of independent communication units are provided. It is possible to realize an efficient communication form linked together.

  Further, the second communication unit transmits the acquired identification information to the first communication unit as the request, and the first communication unit refers to the address information stored in the storage unit, thereby It is preferable that partial address information in which the name of the user corresponding to the identification information transmitted from the second communication unit is associated with the identification information is generated and transmitted to the second communication unit.

  According to the above configuration, the second communication unit requests partial address information by transmitting the acquired identification information to the first communication unit. The first communication unit generates partial address information in which the name of the user corresponding to the identification information is associated with the identification information from the address information stored in the storage unit, and the generated partial address information is the second Send to the communication unit.

  Therefore, the processing time required for transmitting the address information can be shortened compared to transmitting the entire address information from the first communication unit to the second communication unit.

  The first communication unit includes a first display unit that displays an entire image, and the second communication unit includes a second display unit that displays a partial image of the entire image displayed on the first display unit. It is preferable to transmit the partial image displayed on the second display unit.

  According to said structure, the 1st communication unit is provided with the 1st display part, and the 2nd communication unit is provided with the 2nd display part. The second display unit displays a partial image of the entire image displayed on the first display unit. Then, the second communication unit transmits the partial image displayed on the second display unit to the counterpart device.

  Therefore, different images can be displayed on the first display unit and the second display unit, and one of these images can be transmitted from the second communication unit to the counterpart device. Moreover, since the second communication unit transmits a partial image and has a small data capacity, it is possible to reduce the processing amount required for image transmission.

  The first or second communication unit transmits the partial image together with partial image identification information for identifying the partial image to the counterpart device, and the first communication unit transmits the image together with the partial image identification information When received from the counterpart device, it is preferable that the image is transmitted to the second communication unit, and the second communication unit displays the image transmitted from the first communication unit on the second display unit.

  According to said structure, when a 1st communication unit or a 2nd communication unit transmits a partial image to a partner apparatus, a partial image is transmitted with the partial image identification information for identifying the said partial image. When the first communication unit receives a partial image from the counterpart device together with the partial image identification information, the first communication unit transmits the partial image to the second communication unit. The second communication unit displays the partial image on the second display unit.

  Therefore, a partial image that is preferably displayed on the second display unit can be displayed on the second display unit.

  Moreover, it is preferable that the said 2nd communication unit is further provided with the process means which processes the partial image displayed on the said 2nd display part.

  With the above configuration, the partial image displayed on the second display unit can be processed.

  Preferably, the first communication unit further includes a combining unit that combines the partial image processed by the processing unit and the whole image, and transmits the combined image combined by the combining unit to the partner apparatus. .

  According to said structure, a synthetic | combination means synthesize | combines the partial image and whole image which were processed by the process means, and the synthesized image is transmitted to the other apparatus by the 1st communication unit.

  Therefore, when it is preferable to transmit not only the processed partial image but also the entire image to the counterpart apparatus, a composite image including both can be transmitted.

  The second communication unit further includes image determining means for determining whether to transmit the partial image or the composite image to the partner device based on the display size of the display unit of the partner device. Preferably, the image determined by the image determining means is transmitted to the counterpart device.

  According to said structure, an image determination means determines whether a partial image is transmitted to the said other party apparatus, or a synthesized image based on the display size of the display part which the other party apparatus has. Then, the first or second communication unit transmits the partial image or the entire image determined by the image determining means to the counterpart device.

  For example, when the display size is equal to or smaller than a predetermined size, the partial image is transmitted to the partner apparatus. When the display size is larger than the predetermined size, the composite image is transmitted.

  Therefore, a composite image or a partial image can be sent in accordance with the display size of the display unit of the counterpart device, and an easy-to-see image can be sent on the counterpart device.

  The first communication unit performs first data communication, the second communication unit performs second data communication in a manner different from the first data communication, and the second communication unit performs the second data communication. The first communication unit preferably communicates with the counterpart device by the first data communication when communication with the counterpart device cannot be performed by data communication.

  According to said structure, a 1st communication unit and a 2nd communication unit communicate by a mutually different communication style. When the second communication unit cannot detect a partner device that can communicate with itself through the second data communication, the first communication unit communicates with the partner device through the first data communication.

  Therefore, a different communication mode can be selected according to the communication environment with the counterpart device, and communication can be performed more reliably.

  The first data communication is preferably wide-area communication, and the second data communication is preferably short-range wireless communication.

  With the above configuration, when communication by near field communication (for example, Bluetooth (registered trademark)) cannot be performed, communication with a partner apparatus can be performed by wide area communication (for example, the Internet). Therefore, it can be used together with the convenience of near field communication and the certainty of wide area communication.

  The second communication unit is preferably provided so as to be removable.

  With the above configuration, the second communication unit can be removed from the first communication unit and carried around, or the second communication unit can be replaced with another second communication unit.

  As described above, in the communication device according to the present invention, the first communication unit stores the address information in which the identification information of the counterpart device that is the communication destination device is associated with the name of the user of the counterpart device. When the second communication unit detects a partner device that can communicate with itself and acquires identification information of the partner device, the second communication unit sets the target user name that is the name of the user corresponding to the identification information to the first communication unit. The first communication unit makes a request to the communication unit, and the first communication unit refers to the address information stored in the storage unit, so that the information including the target user name requested from the second communication unit is sent to the second communication unit. It is the structure which transmits.

  In the control method according to the present invention, the first communication unit includes a storage unit that stores address information in which identification information of a counterpart device that is a communication destination device is associated with a name of a user of the counterpart device. When the communication unit detects a partner device communicable with itself and acquires identification information of the partner device, the communication unit requests the first communication unit for a target user name that is a user name corresponding to the identification information. A requesting step and a target to which the first communication unit transmits information including the target user name requested from the second communication unit to the second communication unit by referring to the address information stored in the storage unit And a user name transmission step.

  Therefore, even when the second communication unit does not include a storage unit that stores address information, by using the storage unit included in the first communication unit, it is possible to communicate with the counterpart device, and a plurality of independent communication units are provided. There is an effect that it is possible to realize a coordinated and efficient communication mode.

[Embodiment 1]
One embodiment of the present invention is described below with reference to FIGS. First, an outline of the two-screen display device (communication device) 200 of the present embodiment will be described. FIG. 17 is a diagram for explaining an outline of the two-screen display device 200. The two-screen display device 200 includes a main display device (first communication unit) 210 and a sub-display device with sensor (second communication unit) 230, as shown in FIGS. 2 (a), 2 (b) and FIG. ing. The main display device 210 and the sub display device 230 can operate independently of each other.

  The main display device 210 includes a liquid crystal panel (first display unit) 311 for displaying an image (entire image), and the sub display device 230 includes a sensor built-in liquid crystal panel (second display unit) for displaying a partial image of the entire image. 301 is provided. The sensor built-in liquid crystal panel 301 has a smaller display area (screen) than the liquid crystal panel 311. The sub display device 230 transmits the partial image displayed on the sensor built-in liquid crystal panel 301 to a partner terminal (partner device) which is a communication destination terminal device.

  The whole image is, for example, a map image, but may be any image.

  The sensor built-in liquid crystal panel 301 incorporates a plurality of optical sensor circuits (imaging sensors) 32, and the sensor built-in liquid crystal panel 301 images the fingertip of the user touched on the sensor built-in liquid crystal panel 301. . The user can write a line (line 23 in FIG. 17) along the trajectory of the fingertip in the partial image by moving the fingertip touching the sensor built-in liquid crystal panel 301.

  Therefore, the user displays a part of the entire image displayed on the liquid crystal panel 311 on the sensor built-in liquid crystal panel 301, processes the partial image displayed on the sensor built-in liquid crystal panel 301, and displays the processed partial image on the sub display device. 230 can be transmitted to the partner terminal (the personal computer 21 and the portable terminal 22).

  The sub display device 230 can also transmit the partial image to the counterpart terminal via the main display device 210.

  When the counterpart device has a relatively large display unit such as a PC, the main display device 210 generates a composite image by combining the entire image and the processed partial image, and the composite image is displayed. You may transmit to a partner apparatus.

  Next, the configuration of the two-screen display device 200 will be described in detail.

(Outline of the two-screen display device)
The two-screen display device 200 includes a main display device 210 including a liquid crystal panel 311 capable of displaying data, and a sensor built-in liquid crystal panel 301 (described later) capable of detecting an image of a nearby object in addition to displaying data. This is a device having a sub-display device 230 with a sensor.

  The main display device 210 and the sensor-equipped sub display device 230 are detachable. When the sensor-equipped sub-display device 230 is mounted on the main display device 210, it becomes possible to perform wired communication with each other, and by transmitting an “inter-device command” to the partner device, the partner device Can be controlled. Details of the inter-device command will be described later.

  Each of the main display device 210 and the sensor-equipped sub display device 230 includes a control unit that controls the device itself. Therefore, in a state where the main display device 210 and the sensor-equipped sub display device 230 are separated from each other, each can operate independently.

  A specific example of the two-screen display device 200 will be described with reference to FIG. FIG. 2A is a schematic diagram illustrating a specific example of the two-screen display device 200. As shown in the figure, in the present embodiment, a notebook PC (Personal Computer) including a sensor-equipped sub display device 230 instead of a conventional touch pad is assumed as the two-screen display device 200. In this case, a portion other than the sensor-equipped sub display device 230 (a portion including the display unit 310 and the keyboard 920) corresponds to the main display device 210. The state shown in FIG. 2A is a state where the sub display device with sensor 230 is mounted inside the main display device 210. The portion corresponding to the touch pad of the main display device 210 is transparent, and the user can operate the sensor built-in liquid crystal panel 301 of the sub display device with sensor 230 mounted inside.

  FIG. 2B shows the notebook PC shown in FIG. 2A in which the sub display device with sensor 230 is taken out from the main display device 210 and the main display device 210 and the sub display device with sensor 230 are separated. It is a schematic diagram which shows a state. As described above, the main display device 210 and the sensor-equipped sub-display device 230 can each operate alone. In this state, the main display device 210 can operate as a notebook PC having no touch pad function. It is. The sensor-equipped sub-display device 230 can operate as, for example, a remote controller including a sensor-equipped liquid crystal panel 301 or a PDA (Personal Digital Assistant).

(Outline of LCD panel with built-in sensor)
As described above, the sensor built-in liquid crystal panel 301 included in the sensor-equipped sub-display device 230 is a liquid crystal panel capable of detecting an image of a nearby object in addition to displaying data. Here, the image detection of the object is, for example, detection of a position pointed (touched) by the user with a finger or a pen, or reading (scanning) of an image of a printed material or the like. The device used for display is not limited to a liquid crystal panel, and may be an organic EL (Electro Luminescence) panel or the like.

  The structure of the sensor built-in liquid crystal panel 301 will be described with reference to FIG. FIG. 3 is a diagram schematically showing a cross section of the sensor built-in liquid crystal panel 301. The sensor built-in liquid crystal panel 301 described here is an example, and any structure can be used as long as the display surface and the reading surface are shared.

  As shown in the figure, the sensor built-in liquid crystal panel 301 includes an active matrix substrate 51A disposed on the back surface side and a counter substrate 51B disposed on the front surface side, and has a structure in which a liquid crystal layer 52 is sandwiched between these substrates. doing. The active matrix substrate 51A is provided with a pixel electrode 56, a data signal line 57, an optical sensor circuit 32 (not shown), an alignment film 58, a polarizing plate 59, and the like. The counter substrate 51B is provided with color filters 53r (red), 53g (green), 53b (blue), a light shielding film 54, a counter electrode 55, an alignment film 58, a polarizing plate 59, and the like. In addition, a backlight 307 is provided on the back surface of the sensor built-in liquid crystal panel 301.

  The photodiode 6 included in the photosensor circuit 32 is provided in the vicinity of the pixel electrode 56 provided with the blue color filter 53b, but is not limited to this configuration. It may be provided in the vicinity of the pixel electrode 56 provided with the red color filter 53r, or may be provided in the vicinity of the pixel electrode 56 provided with the green color filter 53g.

  Next, referring to FIGS. 4A and 4B, a case where the user touches the sensor built-in liquid crystal panel 301 with a finger or a pen is taken as an example, and the touched position is detected. The types of methods will be described.

  FIG. 4A is a schematic diagram showing how the position touched by the user is detected by detecting the reflected image. When the light 63 is emitted from the backlight 307, the optical sensor circuit 32 including the photodiode 6 detects the light 63 reflected by the object 64 such as a finger. Thereby, the reflected image of the target object 64 can be detected. Thus, the sensor built-in liquid crystal panel 301 can detect the touched position by detecting the reflected image.

  FIG. 4B is a schematic diagram showing a state in which a position touched by the user is detected by detecting a shadow image. As shown in FIG. 4B, the optical sensor circuit 32 including the photodiode 6 detects the external light 61 transmitted through the counter substrate 51B and the like. However, when there is an object 62 such as a pen, the incident of the external light 61 is hindered, so that the amount of light detected by the optical sensor circuit 32 is reduced. Thereby, a shadow image of the object 62 can be detected. Thus, the sensor built-in liquid crystal panel 301 can also detect a touched position by detecting a shadow image.

  As described above, the photodiode 6 may detect reflected light (shadow image) of the light emitted from the backlight 307 or may detect a shadow image caused by external light. Further, the two types of detection methods may be used in combination to detect both a shadow image and a reflected image at the same time.

(Main components of main display device and sub display device with sensor)
Next, the main configuration of the main display device 210 and the sensor-equipped sub display device 230 will be described with reference to FIG. FIG. 5 is a block diagram illustrating a main configuration of the main display device 210 and the sub display device with sensor 230.

(Main components of the sub display device with sensor)
First, the configuration of the main part of the sensor-equipped sub display device 230 will be described. As illustrated, the sensor-equipped sub display device 230 includes a display / light sensor unit (second display unit) 330, a display / light sensor circuit control unit 630, a data processing unit 730, a sub-side main control unit 830, and a storage unit 931. A primary storage unit 932, a communication control unit (second communication control means) 933, a wired communication unit 934, a short-range wireless communication unit 935, an electric field strength detection unit 936, a battery 937, and a power supply switching unit 938.

  The display / light sensor unit 330 is a so-called liquid crystal display device with a built-in light sensor. The display / light sensor unit 330 includes a sensor built-in liquid crystal panel 301, a backlight 307, and a peripheral circuit 309 for driving them. A detailed configuration of the display / light sensor unit 330 will be described later.

  Next, the display / light sensor circuit control unit 630 has a function as a device driver that controls the peripheral circuit 309 of the display / light sensor unit 330. A detailed configuration of the display / light sensor circuit control unit 630 will be described later.

  Next, the data processing unit 730 has a function as middleware that gives an instruction to the display / light sensor circuit control unit 630 based on the “in-sub display device command” received from the sub-side main control unit 830. . A detailed configuration of the data processing unit 730 will be described later. Details of the sub display device commands will also be described later.

  Next, the sub-side main control unit 830 controls the operation of each unit included in the sensor-equipped sub display device 230. The sub-side main control unit 830 reads various programs stored in the storage unit 931, controls each unit of the sensor-equipped sub display device 230, and implements various functions included in the sensor-equipped sub display device 230.

  The sub-side main control unit 830 causes the data processing unit 730 to display the display data on the sensor built-in liquid crystal panel 301 and to scan the object in the sensor built-in liquid crystal panel 301. Send commands and display data. When “data type” is designated for the command within the sub display device, at least one of the whole image data, the partial image data, and the coordinate data is sent as a response to the command within the sub display device. Receive from 730.

  In addition, the sub-side main control unit 830 receives control from the main display device 210. Specifically, the inter-device command transmitted from the main-side main control unit 810 of the main display device 210 is received, and one of the following (1) to (3) is selected according to the type of the received inter-device command. The process shown is executed.

  First, (1) if the received inter-device command is a “scan command” (details will be described later) that designates scanning of an object located in the vicinity of the display / light sensor unit 330, the sub-side The main control unit 830 converts the received scan command into a sub display device command, and transmits the converted sub display device command to the data processing unit 730. The above conversion is performed by removing a “0th” field (described later) from the scan command.

  The sub-side main control unit 830 receives at least one of the whole image data, the partial image data, and the coordinate data from the data processing unit 730 as a response to the command within the sub-display device after the conversion. The received data is transmitted to the main main control unit 810.

  Next, (2) when the received inter-device command is a “display command” (details will be described later) for designating display / light sensor unit 330 to display image data such as a photograph or an icon. The sub-side main control unit 830 generates display data based on the information specified in the display command, and transmits the generated display data to the data processing unit 730.

  Next, (3) when the received inter-device command is a “processing command” (details will be described later) that designates execution of a predetermined process, the sub-side main control unit 830 is designated as the processing command. The predetermined processing is executed.

  The predetermined processing executed by the sub-side main control unit 830 includes, for example, character recognition processing and image processing for image data obtained from the data processing unit 730, and communication using the communication control unit 933 of the sensor-equipped sub display device 230. This is an execution process of various applications, middleware, and the like included in the sensor-equipped sub display device 230 such as a connection process and a file transfer process.

  Further, the sub-side main control unit 830 instructs the main display device 210 to execute a predetermined process. Specifically, an inter-device command as a processing command is transmitted to the main side main control unit 810 of the main display device 210. For example, the OS (Operating System) activation process of the main display device 210 and the communication control unit (first communication control means) 913 of the main display device 210 are used as the predetermined processing executed by the main main control unit 810. This is an execution process of various applications, middleware, etc. of the main display device 210 such as a communication connection process and a file transfer process.

  The display / light sensor circuit control unit 630, the data processing unit 730, and the sub-side main control unit 830 can be configured by a CPU (Central Processing Unit), a memory, and the like, respectively. The data processing unit 730 may be configured by a circuit such as an ASIC (application specific integrate circuit).

  Next, the storage unit 931 reads (1) the control program of each unit, (2) the OS program, (3) the application program, and (4) the program executed by the sub-side main control unit 830. Records various data. The storage unit 931 is configured by a nonvolatile storage device such as a flash memory.

  The primary storage unit 932 is configured by a volatile storage device such as a RAM (Random Access Memory), and temporarily holds data in the process in which the sub-side main control unit 830 executes the various programs described above. Used as a work area.

  Next, the communication control unit 933 allows the sensor-equipped sub display device 230 to communicate with either the wired communication unit 934 for communicating with the main display device 210, and the main display device 210 or an external device (not shown). The short-range wireless communication unit 935 is controlled.

  That is, the communication control unit 933 transmits an inter-device command and various data transmitted from the main display device 210 via either the wired communication unit 934 or the short-range wireless communication unit 935 to the sub-side main control unit 830. . Also, the communication control unit 933 sends inter-device commands and various data transmitted from the sub-side main control unit 830 in response to an instruction from the sub-side main control unit 830, and the wired communication unit 934 and the short-range wireless communication unit 935. Is transmitted to the main display device 210.

  Further, the communication control unit 933 exchanges various data with an external device via the short-range wireless communication unit 935.

  The wired communication unit 934 can be connected to the wired communication unit 914 of the main display device 210. In the present embodiment, it is assumed that the wired communication unit 934 and the wired communication unit 914 of the main display device 210 are connected by USB (universal serial bus).

  At the time of connection, the wired communication unit 934 can transmit and receive signals to and from the wired communication unit 914 of the main display device 210 using the communication line, and the wired communication of the main display device 210 using the power line. Power can be supplied from the unit 914. When the power supply is received from the wired communication unit 914 of the main display device 210, the wired communication unit 934 supplies power to the power supply switching unit 938.

  The short-range wireless communication unit 935 can communicate with each other using a wireless communication path formed with the short-range wireless communication unit 915 of the main display device 210 and with an external device. Yes. In the present embodiment, it is assumed that Bluetooth (registered trademark) communication (hereinafter simply referred to as Bluetooth) communication is applied to the wireless communication path. That is, the short-range wireless communication unit 935 is a communication unit that can transmit and receive signals by Bluetooth communication. Note that the wireless communication path is not particularly limited as long as it can transmit and receive signals, and IEEE802.11 wireless, ZigBee (registered trademark), or the like may be applied.

  Next, the electric field strength detection unit 936 detects the strength of the signal received by the short-range wireless communication unit 935 (RSSI (Received Signal Strength Indication)) in response to an instruction from the sub-side main control unit 830. To do. Then, the detection result is transmitted to the sub-side main control unit 830.

  Next, when the wired communication unit 934 is supplied with power from the wired communication unit 914 of the main display device 210, the power supply switching unit 938 supplies the power to the respective units included in the sensor-equipped sub display device 230 from the main display device 210. Supply the power that is. On the other hand, when the wired communication unit 934 does not receive power from the wired communication unit 914 of the main display device 210, the power supply switching unit 938 supplies the power of the battery 937 to each unit included in the sensor-equipped sub display device 230. .

(Main display unit configuration)
Next, a configuration of main parts of the main display device 210 will be described. As illustrated, the main display device 210 includes a display unit (first display unit) 310, a display circuit control unit 610, a display data processing unit 710, a main main control unit 810, a storage unit 911, a primary storage unit 912, a communication. A control unit 913, a wired communication unit 914, a short-range wireless communication unit 915, a wired / wireless network communication unit 916, a power source 917, an operation unit 919, an audio output unit 924, and an audio input unit 925 are provided.

  The display unit 310 is for displaying data, and includes a liquid crystal panel 311, a backlight 317, and a peripheral circuit 319 for driving them. The display circuit control unit 610 has a function as a device driver that controls the peripheral circuit 319. The display data processing unit 710 has a function as middleware that gives an instruction to the display circuit control unit 610 based on the display data received from the main main control unit 810. The display unit 310, the display circuit control unit 610, and the display data processing unit 710 can be configured using a generally known technique.

  Next, the main main control unit 810 controls the operation of each unit included in the main display device 210. The main main control unit 810 reads various programs stored in the storage unit 911, controls each unit of the main display device 210, and realizes various functions included in the main display device 210.

  The main-side main control unit 810 transmits an inter-device command to the sub-side main control unit 830 of the sensor-equipped sub display device 230, and causes the sensor-equipped sub display device 230 to execute predetermined processing. When “data type” (described later) is specified for a scan command that is one of the inter-device commands, at least one of whole image data, partial image data, and coordinate data is returned as a response to the command. Is received from the sub-side main control unit 830.

  The main main control unit 810 receives a processing command that is one of the inter-device commands from the sub-side main control unit 830 of the sensor-equipped sub display device 230, and performs a predetermined process specified in the received inter-device command. (OS startup processing, execution processing of various applications, middleware, etc.) is performed.

  The display circuit control unit 610, the display data processing unit 710, and the main side main control unit 810 can each be configured with a CPU, a memory, and the like.

  Next, the storage unit 911 reads (1) the control program for each unit, (2) the OS program, (3) the application program, and (4) the program executed by the main-side main control unit 810. Records various data. The storage unit 911 is configured by a nonvolatile storage device such as a flash memory.

  The primary storage unit 912 is configured by a volatile storage device such as a RAM, and serves as a work area for temporarily storing data in the course of executing the above-described various programs by the main main control unit 810. used.

  The communication control unit 913 is a short-range wireless communication for the main display device 210 to communicate with the wired communication unit 914 for communicating with the sensor-equipped sub-display device 230, the sensor-equipped sub-display device 230, and an external device (not shown). Control unit 915 and wired / wireless network communication unit 916 for communicating with an external device (not shown).

  That is, the communication control unit 913 sends inter-device commands and various data transmitted from the sensor-equipped sub display device 230 via either the wired communication unit 914 or the short-range wireless communication unit 915 to the main main control unit 810. Send. In addition, the communication control unit 913 receives inter-device commands and various data transmitted from the main main control unit 810 in response to an instruction from the main main control unit 810, the wired communication unit 914, and the short-range wireless communication unit 915. Is transmitted to the sensor-equipped sub-display device 230.

  In addition, the communication control unit 933 exchanges various data with an external device via either the short-range wireless communication unit 935 or the wired / wireless network communication unit 916.

  As described above, the wired communication unit 914 can be connected to the wired communication unit 934 of the sensor-equipped sub display device 230. The wired communication unit 914 and the wired communication unit 934 of the sensor-equipped sub display device 230 Assumes a USB connection.

  The short-range wireless communication unit 915 is configured to be able to communicate with each other using a wireless communication path formed between the short-range wireless communication unit 935 of the sensor-equipped sub display device 230 and an external device. It is assumed that Bluetooth communication is applied to the wireless communication path. In other words, the short-range wireless communication unit 915 is a communication unit that can transmit and receive signals by Bluetooth communication.

  The wired / wireless network communication unit 916 is configured to be communicably connected to an external communication network, and performs communication with an external device. The communication network is not particularly limited. For example, the Internet, intranet, extranet, LAN, ISDN, VAN, CATV communication network, virtual private network, telephone line network, mobile communication network, satellite communication. A net or the like is available.

  As described above, the sub display device 230 does not include a wired / wireless network communication unit, and in this respect, the main display device 210 and the sub display device 230 are different from each other in communication mode.

  The power source 917 supplies power to each unit included in the main display device 210. When the wired communication unit 914 is connected to the wired communication unit 934 of the sensor-equipped sub display device 230, power is also supplied to the sensor-equipped sub display device 230 via the wired communication unit 914. The power can be turned on / off by the user operating the power switch 922.

  Next, the operation unit 919 receives an input operation of the user of the main display device 210. The operation unit 919 includes, for example, an input device such as a keyboard 920, a switch, a mouse, and a remote controller. Then, the operation unit 919 generates a control signal corresponding to the input operation of the user of the main display device 210, and transmits the generated control signal to the main side main control unit 810.

  As an example of the switch, a hinge switch 921 for detecting the open / closed state of the casing, a power switch 922 for switching on / off of the power, A hardware switch such as a user switch 923 to which a function is assigned is assumed.

  In addition, the main display device 210 includes an audio output unit 924 such as a speaker for outputting audio and an audio input unit 925 such as a microphone for inputting an audio signal.

(Detailed configuration of sensor-equipped sub-display device)
Next, detailed configurations of the display / light sensor unit 330, the display / light sensor circuit control unit 630, and the data processing unit 730 of the sub display device with sensor 230 will be described with reference to FIG. FIG. 6 is a block diagram showing more detailed configurations of the display / light sensor unit 330, the display / light sensor circuit control unit 630, and the data processing unit 730.

  First, a more detailed configuration of the display / light sensor unit 330 will be described. As shown in the figure, the display / light sensor unit 330 includes a sensor built-in liquid crystal panel 301, a backlight 307, and a peripheral circuit 309 for driving them.

  The sensor built-in liquid crystal panel 301 includes a plurality of pixel circuits 31 and photosensor circuits 32 arranged in a matrix. The detailed configuration of the sensor built-in liquid crystal panel 301 will be described later.

  The peripheral circuit 309 includes a liquid crystal panel drive circuit 304, an optical sensor drive circuit 305, a signal conversion circuit 306, and a backlight drive circuit 308.

  The liquid crystal panel drive circuit 304 outputs a control signal (G) and a data signal (S) in accordance with the timing control signal (TC1) and the data signal (D) from the liquid crystal panel control unit 601 of the display / light sensor circuit control unit 630. In this case, the pixel circuit 31 is driven. Details of the driving method of the pixel circuit 31 will be described later.

  The optical sensor driving circuit 305 is a circuit that drives the optical sensor circuit 32 by applying a voltage to the signal line (R) in accordance with the timing control signal (TC2) from the sensor control unit 602 of the display / optical sensor circuit control unit 630. is there. Details of the driving method of the optical sensor circuit 32 will be described later.

  The signal conversion circuit 306 is a circuit that converts the sensor output signal (SS) output from the optical sensor circuit 32 into a digital signal (DS) and transmits the converted signal to the sensor control unit 602.

  The backlight 307 includes a plurality of white LEDs (Light Emitting Diodes) and is disposed on the back surface of the sensor built-in liquid crystal panel 301. When a power supply voltage is applied from the backlight drive circuit 308, the backlight 307 is turned on and irradiates the sensor built-in liquid crystal panel 301 with light. Note that the backlight 307 is not limited to white LEDs, and may include LEDs of other colors. The backlight 307 may include, for example, a cold cathode fluorescent lamp (CCFL) instead of the LED.

  The backlight drive circuit 308 applies a power supply voltage to the backlight 307 when the control signal (BK) from the backlight control unit 603 of the display / light sensor circuit control unit 630 is at a high level. When the control signal from the light control unit 603 is at a low level, no power supply voltage is applied to the backlight 307.

  Next, the display / light sensor circuit control unit 630 will be described. The display / light sensor circuit control unit 630 includes a liquid crystal panel control unit 601, a sensor control unit 602, a backlight control unit 603, and a display data storage unit 604.

  The liquid crystal panel control unit 601 receives display data from the display data processing unit 701 of the data processing unit 730 and, in accordance with an instruction from the display data processing unit 701, sends a timing to the liquid crystal panel drive circuit 304 of the display / light sensor unit 330. A control signal (TC1) and a data signal (D) are transmitted, and the received display data is displayed on the sensor built-in liquid crystal panel 301.

  Note that the liquid crystal panel control unit 601 primarily stores the display data received from the display data processing unit 701 in the display data storage unit 604. Then, a data signal (D) is generated based on the primary stored display data. The display data storage unit 604 is, for example, a video random access memory (VRAM).

  The sensor control unit 602 transmits a timing control signal (TC2) to the optical sensor driving circuit 305 of the display / optical sensor unit 330 in accordance with an instruction from the sensor data processing unit 703 of the data processing unit 730, and the sensor built-in liquid crystal panel 301. Run the scan with.

  In addition, the sensor control unit 602 receives a digital signal (DS) from the signal conversion circuit 306. Then, image data is generated based on the digital signal (DS) corresponding to the sensor output signal (SS) output from all the optical sensor circuits 32 included in the sensor built-in liquid crystal panel 301. That is, the image data read in the entire reading area of the sensor built-in liquid crystal panel 301 is generated. Then, the generated image data is transmitted to the sensor data processing unit 703.

  The backlight control unit 603 transmits a control signal (BK) to the backlight drive circuit 308 of the display / light sensor unit 330 in accordance with instructions from the display data processing unit 701 and the sensor data processing unit 703 to drive the backlight 307. Let

  Next, the data processing unit 730 will be described. The data processing unit 730 includes a display data processing unit 701 and a sensor data processing unit 703.

  The display data processing unit 701 receives display data from the sub-side main control unit 830 and gives instructions to the liquid crystal panel control unit 601 and the backlight control unit 603 according to the sub-display device command received by the data processing unit 730. The received display data is displayed on the sensor built-in liquid crystal panel 301.

  The sensor data processing unit 703 gives instructions to the sensor control unit 602 and the backlight control unit 603 in accordance with the sub-display device command received by the data processing unit 730.

  The sensor data processing unit 703 receives image data from the sensor control unit 602 and stores the image data in the image data buffer 704 as it is. The sensor data processing unit 703 then selects “whole image data”, “partial image data (partial) based on the image data stored in the image data buffer 704 in accordance with the sub-display device command received by the data processing unit 730. At least one of “including coordinate data of image)” and “coordinate data” is transmitted to the sub-side main control unit 830. The whole image data, partial image data, and coordinate data will be described later. The operation of the sensor data processing unit 703 according to the sub display device command will be described later.

(Detailed configuration of main display device)
As described above, the display unit 310 and the display circuit control unit 610 of the main display device 210 can be configured using a generally known technique, but these configuration examples are described with reference to FIG. A brief explanation will be given. FIG. 7 is a block diagram illustrating a configuration example of the display unit 310 and the display circuit control unit 610.

  The display unit 310 includes a liquid crystal panel 311 including a plurality of pixel circuits 38 arranged in a matrix, a plurality of white LEDs, a backlight 317 disposed on the back surface of the liquid crystal panel 311, and drives them. And a peripheral circuit 319. The backlight 317 is not limited to the white LED, and may include another color LED, or may include a cold cathode tube instead of the LED.

  The peripheral circuit 319 includes a liquid crystal panel drive circuit 314 that drives the pixel circuit 38 and a backlight drive circuit 318 that applies a power supply voltage to the backlight 317. Note that when a power supply voltage is applied from the backlight driving circuit 318, the backlight 317 is turned on and irradiates the liquid crystal panel 311 with light.

  Next, the display circuit control unit 610 includes a liquid crystal panel control unit 611, a backlight control unit 613, and a display data storage unit 614.

  The liquid crystal panel control unit 611 receives display data from the display data processing unit 710 and transmits a control signal and a data signal to the liquid crystal panel drive circuit 314 in accordance with an instruction from the display data processing unit 710. The received display data Is displayed on the liquid crystal panel 311. The liquid crystal panel control unit 611 temporarily stores the display data received from the display data processing unit 710 in the display data storage unit 614 (VRAM or the like), and generates a data signal based on the display data stored in the primary storage. To do.

  The backlight control unit 613 transmits a control signal to the backlight driving circuit 318 according to an instruction from the display data processing unit 710 to drive the backlight 317.

(Sub display device command)
Next, with reference to FIGS. 8 and 9, details of the sub display device commands transmitted from the sub-side main control unit 830 to the data processing unit 730 will be described. FIG. 8 is a diagram schematically showing an example of the frame structure of the sub-display device command. FIG. 9 is a diagram for explaining an example of values that can be specified in each field included in the sub-display device command and an outline thereof.

  As shown in FIG. 8, the sub-display device commands include “header”, “data acquisition timing”, “data type”, “scan method”, “scanned image gradation”, “scan resolution”, and “reserved”. Each field. In each field, for example, values shown in FIG. 9 can be designated.

  The “header” field is a field indicating the start of a frame of the sub display device command. As long as it is possible to identify the start of the frame, the value of the “header” field may be any value.

  Next, the “data acquisition timing” field is a field for designating timing for acquiring data from the sensor data processing unit 703. In the “data acquisition timing” field, for example, values “00” (sense), “01” (event), and “10” (all) can be specified.

  Here, “sense” designates that the latest data is immediately transmitted to the sensor data processing unit 703. Therefore, when the sensor data processing unit 703 receives a command in the sub display device whose value in the “data acquisition timing” field is “sense”, the latest data specified in the “data type” field is immediately updated. Transmit to the sub-side main control unit 830.

  “Event” designates that the sensor data processing unit 703 transmits the image data received from the sensor control unit 602 at a timing when the change occurs. Therefore, when the sensor data processing unit 703 receives a command in the sub display device whose value of the “data acquisition timing” field is “event”, the sensor control unit 602 converts the data specified in the “data type” field. Is transmitted to the sub-side main control unit 830 at a timing when a change larger than a predetermined threshold occurs in the image data received from.

  Further, “all” designates that the sensor data processing unit 703 transmits data at a predetermined cycle. Therefore, when the sensor data processing unit 703 receives a command in the sub display device having a value of “all” in the “data acquisition timing” field, the data designated in the “data type” field is Transmit to the sub-side main control unit 830. The predetermined period coincides with the period in which the optical sensor circuit 32 performs scanning.

  Next, the “data type” field is a field for designating the type of data acquired from the sensor data processing unit 703. In the “data type” field, for example, values of “001” (coordinates), “010” (partial image), and “100” (entire image) can be specified. Furthermore, by adding these values, “coordinates” and “partial image” / “whole image” can be specified simultaneously. For example, when “coordinate” and “partial image” are specified at the same time, “011” can be specified.

  When the sensor data processing unit 703 receives a command in the sub display device whose value of the “data type” field is “whole image”, the image data itself stored in the image data buffer 704 is sent to the sub-side main control unit 830. Send. The image data itself stored in the image data buffer 704 is referred to as “whole image data”.

  In addition, when the sensor data processing unit 703 receives a command in the sub display device whose value of the “data type” field is “partial image”, the image data received from the sensor control unit 602 changes more than a predetermined threshold. An area including the generated portion is extracted, and image data of the extracted area is transmitted to the sub-side main control unit 830. Here, the image data is referred to as “partial image data”.

  When a plurality of partial image data is extracted from the image data received from the sensor control unit 602, the sensor data processing unit 703 sends each of the extracted plurality of partial image data to the sub-side main control unit 830. Send. Therefore, for example, when the user touches a plurality of locations on the sensor built-in liquid crystal panel 301 with a plurality of fingers or pens at the same time, the sensor data processing unit 703 extracts each of the touched regions as partial image data, Transmit to the sub-side main control unit 830.

  Further, the sensor data processing unit 703 detects a representative point in the partial image and receives the position of the representative point in the partial image when a command in the sub display device whose value of the “data type” field is “partial image” is received. Is sent to the sub-side main control unit 830. The representative point includes, for example, the center of the partial image data, the center of gravity of the partial image data, and the like.

  When a plurality of partial image data is extracted from the image data received from the sensor control unit 602, the sensor data processing unit 703 detects each representative point of the extracted plurality of partial image data, and The coordinate data of the representative coordinates indicating the position of each detected representative point in the partial image is transmitted to the sub-side main control unit 830. Therefore, for example, when the user touches a plurality of locations on the sensor built-in liquid crystal panel 301 with a plurality of fingers or pens at the same time, the sensor data processing unit 703 displays the position of the representative point of each touched region in each region. Is transmitted to the sub-side main control unit 830.

  Next, when the sensor data processing unit 703 receives the in-sub-display device command whose value of the “data type” field is “coordinate”, the coordinate data of the representative coordinate indicating the position of the representative point in the entire image data is obtained. Transmit to the sub-side main control unit 830.

  When a plurality of partial image data is extracted from the image data received from the sensor control unit 602, the sensor data processing unit 703 displays the position of each representative point of the extracted plurality of partial images in the entire image. Is sent to the sub-side main control unit 830. Therefore, for example, when the user simultaneously touches a plurality of locations on the sensor built-in liquid crystal panel 301 with a plurality of fingers, pens, etc., the sensor data processing unit 703 displays the positions of the touched representative points in the entire image. Is transmitted to the sub-side main control unit 830.

  That is, the sensor-equipped sub-display device 230 can detect the representative point of the touched region regardless of the number of touches of the sensor-equipped liquid crystal panel 301 by the user with a finger or a pen (multi-point detection). Then, the coordinate data of the representative coordinates indicating the position of the detected representative point in the partial image and the coordinate data of the representative coordinates indicating the position in the entire image can be used as the sub-side main control unit 830.

  Specific examples of the whole image data, the partial image data, and the coordinate data will be described later with reference to schematic diagrams.

  Next, the “scan method” field is a field for designating whether or not the backlight 307 is turned on at the time of executing the scan. In the “scan method” field, for example, values of “00” (reflection), “01” (transmission), and “10” (reflection / transmission) can be designated.

  “Reflection” designates that scanning is performed with the backlight 307 turned on. Therefore, when the sensor data processing unit 703 receives a command in the sub-display device in which the value of the “scan method” field is “reflection”, the optical sensor driving circuit 305 and the backlight driving circuit 308 operate in synchronization. In addition, an instruction is given to the sensor control unit 602 and the backlight control unit 603.

  “Transmission” specifies that scanning is performed with the backlight 307 turned off. Accordingly, when the sensor data processing unit 703 receives the sub-display device command whose “scan method” field value is “transparent”, the sensor data processing unit 703 operates the optical sensor driving circuit 305 and does not operate the backlight driving circuit 308. An instruction is given to the sensor control unit 602 and the backlight control unit 603. Note that “reflection / transmission” specifies that scanning is performed using both “reflection” and “transmission”.

  Next, the “scanned image gradation” field is a field for designating gradations of the partial image data and the entire image data. In the “scanned image gradation” field, for example, values of “00” (binary) and “01” (multivalue) can be designated.

  Here, when the sensor data processing unit 703 receives a command in the sub display device having a “binary” value in the “scanned image gradation” field, the partial image data and the whole image data are converted into monochrome data, and the main data on the sub side. Transmit to the control unit 830.

  In addition, when the sensor data processing unit 703 receives a command in the sub-display device having a “scanned image gradation” field value of “multi-value”, the partial image data and the entire image data are converted into multi-gradation data, Transmit to the main control unit 830.

  Next, the “scan resolution” field is a field for designating the resolution of the partial image data and the entire image data. In the “scan resolution” field, for example, values of “0” (high) and “1” (low) can be specified.

  Here, “high” designates a high resolution. Therefore, when the sensor data processing unit 703 receives a command in the sub display device having a value of “high” in the “scan resolution” field, it transmits the partial image data and the entire image data to the sub-side main control unit 830 at a high resolution. To do. For example, it is desirable to designate “high” for image data (image data such as a fingerprint) to be subjected to image processing such as image recognition.

  “Low” designates a low resolution. Therefore, when the sensor data processing unit 703 receives the sub-display device internal command whose “scan resolution” field value is “low”, it transmits the partial image data and the entire image data to the sub-side main control unit 830 at a low resolution. To do. For example, it is desirable to designate “low” for image data (such as touched finger or hand image data) that only needs to be recognized.

  Next, the “reserved” field is a field that is appropriately specified when it is necessary to further specify information other than information that can be specified in the above-described fields.

  Note that it is not necessary to use all of the above-described fields when transmitting a sub-display device command, and invalid values (such as a NULL value) may be set for fields that are not used.

  When the user wants to acquire coordinate data of a position touched with a finger or a pen, a command in the sub display device in which “coordinate” is specified in the “data type” field is transmitted to the data processing unit 730. When it is desired to detect the movement of a finger or a pen, it is desirable to further specify “all” in the “data acquisition timing” field of the sub display device command to acquire coordinate data. In this case, since it is only necessary to obtain coordinate data of the touched position, the scanning accuracy may not be high. Therefore, it is only necessary to designate “low” as the value of the “scan resolution” field of the sub display device command.

  In addition, when “coordinate” is specified in the “data type” field of the sub display device command, for example, when the user touches the sensor built-in liquid crystal panel 301 with a plurality of fingers or pens at the same time, the touch is performed. Each coordinate data of the position can be acquired (multi-point detection).

  Further, when acquiring image data of an object such as a document, a command in the sub display device in which “whole image” is specified in the “data type” field is transmitted to the data processing unit 730. Since it is common to scan an object in a stationary state, it is not necessary to periodically scan the object. Therefore, in this case, it is desirable to designate “sense” or “event” in the “data acquisition timing” field. When scanning an object such as a document, it is desirable that the scanning accuracy is high so that the user can easily read the characters. Therefore, it is desirable to designate “high” in the “scan resolution” field.

(Inter-device command)
Next, the frame structure of the inter-device command will be described with reference to FIG. 10 and FIG. First, FIG. 10 is a schematic diagram illustrating an example of a frame structure of an inter-device command. As illustrated, the inter-device command includes a “header” field, “0th” to “fifth” fields, and a “reserved” field in order from the top.

  The “header” field is a field indicating the start of the frame of the inter-device command. As long as it is possible to identify the start of the frame, the header field value may be any value.

  Next, the “0th” field is a field for designating a “command identifier” for identifying the type of command between devices.

  The inter-device command whose command identifier value is “000” is referred to herein as a “scan command”. The scan command is sent from the main main controller 810 to the sub display main unit 810 so that the main display device 210 causes the sub display device with sensor 230 to scan an object located in the vicinity of the sensor built-in liquid crystal panel 301. It is transmitted to the control unit 830. The sub-side main control unit 830 that has received the scan command converts the scan command into a sub-display device command by removing the 0th field from the scan command, and converts the converted sub-display device command into data. The data is transmitted to the processing unit 730.

  Further, an inter-device command whose command identifier value is “001” is referred to herein as a “display command”. The display command is sent from the main main control unit 810 to the sub display device with sensor 230 so that the display / light sensor unit 330 displays image data such as photographs and icons. It is transmitted to the side main control unit 830.

  An inter-device command whose command identifier value is “010” is referred to herein as a “processing command”. The processing command is transmitted / received between the main side main control unit 810 and the sub side main control unit 830 so that either the sub display device with sensor 230 or the main display device 210 causes the other party to execute a predetermined process. The

  Next, information that can be specified in each field from “first” to “fifth” is different for each of the scan command, the display command, and the processing command. Information that can be specified in the first to fifth fields will be described later.

  Next, the “reserved” field is a field that is appropriately specified when it is necessary to further specify information other than the information that can be specified in each field described above.

  Note that it is not necessary to use all the fields described above, and invalid values (such as NULL values) may be set for fields that are not used.

  Next, information that can be specified in each field for each of the scan command, the display command, and the processing command will be described with reference to FIG. FIG. 11 is an explanatory diagram showing information that can be specified in each field for each of a scan command, a display command, and a processing command.

  As shown in the figure, in the scan command, “data acquisition timing”, “data type”, “scan method”, “scan image gradation”, and “scan resolution” can be specified in the first to fifth fields. it can. Note that the information that can be specified in the first to fifth fields of the scan command is the same as the information that can be specified in the fields of the sub display device command.

  The information that can be specified in the first to fifth fields of the scan command is the same as the information that can be specified in each field of the command in the sub display device, so that the sub-side main control unit that has received the scan command. 830 removes the 0th field of the scan command received from the main main control unit 810 so that the received scan command can be converted into a sub-display device command.

  In the display command, “display number” and “display data / display position” can be designated in the first and second fields. Note that the third to fifth fields are not used (a NULL value).

  In the processing command, “execution processing type”, “argument number”, and “argument information” can be specified in the first to third fields. Note that the fourth and fifth fields are not used (is a NULL value).

  Next, an example of values that can be specified in each field and an outline thereof will be described for each of the scan command, the display command, and the processing command with reference to FIGS.

  FIG. 12 is an explanatory diagram of an example of values that can be specified in the first to fifth fields of the scan command and an outline thereof. As shown in the figure, the first to fifth fields of the scan command are the same as the fields of the sub-display device command, and thus description thereof is omitted here.

  Next, the display command will be described. FIG. 13 is an explanatory diagram of an example of values that can be specified in the first and second fields of the display command and an outline thereof.

  The first field (display number) of the display command is a field for designating the number of data to be displayed on the display / light sensor unit 330 of the sub display device with sensor 230. That is, it indicates the number of sets designated in the second field (display data / display position).

  The second field (display data / display position) of the display command is a field indicating a set of image data itself to be displayed on the display / light sensor unit 330 of the sub-display device with sensor 230 and the display position (coordinates) of the data. It is. When a plurality of image data are displayed, a plurality of sets are designated in the second field.

  Next, the processing command will be described. FIG. 14 is an explanatory diagram of an example of values that can be specified in the first to third fields of the processing command and an outline thereof.

  The first field (execution processing type) of the processing command is a field for specifying identification information of processing (various applications, middleware, OS, etc.) to be executed by the main control unit that is the processing command transmission destination. The identification information may be any value as long as the process can be identified.

  The second field (argument number) of the processing command is a field for designating the number of various information (arguments) used during or after the processing indicated by the identification information specified in the first field. That is, it indicates the number of data specified in the third field (argument information).

  The third field (argument information) of the processing command is a field for designating various information (arguments) used during or after the processing indicated by the identification information specified in the first field.

  The information used at the time of starting includes, for example, an operation mode of a predetermined process specified in the first field, a parameter specifying a display area, and the like.

  Further, the information used after activation includes, for example, image data to be recognized by the character recognition application when a character recognition application is specified in the first field. Further, when a file transfer application is specified in the first field, a file to be transferred by the file transfer application can be cited.

(Whole image data / Partial image data / Coordinate data)
Next, the whole image data, the partial image data, and the coordinate data will be described by way of example with reference to FIG. The image data shown in FIG. 15A is image data obtained as a result of scanning the entire sensor built-in liquid crystal panel 301 when the object is not placed on the sensor built-in liquid crystal panel 301. 15B is image data obtained as a result of scanning the entire sensor-equipped liquid crystal panel 301 when the user is touching the sensor-equipped liquid crystal panel 301 with a finger.

  When the user touches the sensor built-in liquid crystal panel 301 with a finger, the amount of light received by the photosensor circuit 32 in the vicinity of the touch changes, so that the voltage output from the photosensor circuit 32 changes, and as a result, In the image data generated by the sensor control unit 602, the brightness of the pixel value of the portion touched by the user changes.

  In the image data shown in FIG. 15B, the brightness of the pixel value of the portion corresponding to the user's finger is higher than that in the image data shown in FIG. In the image data shown in FIG. 15B, the smallest rectangular area (area PP) that includes all pixel values whose lightness changes more than a predetermined threshold is “partial image data”.

  The image data indicated by the area AP is “whole image data”.

  Further, the coordinate data in the entire image data (area AP) of the representative coordinates Z of the partial image data (area PP) is (Xa, Ya), and the coordinate data in the partial image data (area PP) is (Xp, Yp). It is.

(Configuration of sensor built-in liquid crystal panel)
Next, the configuration of the sensor built-in liquid crystal panel 301 included in the sensor-equipped sub display device 230 and the configuration of the peripheral circuit 309 of the sensor built-in liquid crystal panel 301 will be described with reference to FIG. FIG. 16 is a block diagram showing the main part of the display / light sensor unit 330, particularly the configuration of the sensor built-in liquid crystal panel 301 and the configuration of the peripheral circuit 309.

  The sensor built-in liquid crystal panel 301 includes a pixel circuit 31 for setting light transmittance (brightness) and an optical sensor circuit 32 that outputs a voltage corresponding to the intensity of light received by the sensor. The pixel circuit 31 is used as a general term for the R pixel circuit 31r, the G pixel circuit 31g, and the B pixel circuit 31b corresponding to the red, green, and blue color filters, respectively.

  The pixel circuits 31 are arranged on the sensor built-in liquid crystal panel 301 in the column direction (vertical direction) and 3n in the row direction (horizontal direction). A set of the R pixel circuit 31r, the G pixel circuit 31g, and the B pixel circuit 31b is continuously arranged in the row direction (lateral direction). This set forms one pixel.

  In order to set the light transmittance of the pixel circuit 31, first, the high level voltage (TFT 33 is turned on) to the scanning signal line Gi connected to the gate terminal of the TFT (Thin Film Transistor) 33 included in the pixel circuit 31. Voltage). Thereafter, a predetermined voltage is applied to the data signal line SRj connected to the source terminal of the TFT 33 of the R pixel circuit 31r. Similarly, the light transmittance is also set for the G pixel circuit 31g and the B pixel circuit 31b. Then, by setting these light transmittances, an image is displayed on the sensor built-in liquid crystal panel 301.

  Next, the photosensor circuit 32 is arranged for each pixel. One pixel may be arranged in the vicinity of each of the R pixel circuit 31r, the G pixel circuit 31g, and the B pixel circuit 31b.

  In order for the optical sensor circuit 32 to output a voltage according to the light intensity, first, the sensor readout line RWi connected to one electrode of the capacitor 35 and the anode terminal of the photodiode 6 are connected. A predetermined voltage is applied to the sensor reset line RSi. In this state, when light is incident on the photodiode 6, a current corresponding to the amount of incident light flows through the photodiode 6. And according to the said electric current, the voltage of the connection point (henceforth connection node V) of the other electrode of the capacitor | condenser 35 and the cathode terminal of the photodiode 6 falls. When the power supply voltage VDD is applied to the voltage application line SDj connected to the drain terminal of the sensor preamplifier 37, the voltage at the connection node V is amplified and output from the source terminal of the sensor preamplifier 37 to the sensing data output line SPj. Based on the output voltage, the amount of light received by the optical sensor circuit 32 can be calculated.

  Next, the liquid crystal panel drive circuit 304, the optical sensor drive circuit 305, and the sensor output amplifier 44, which are peripheral circuits of the sensor built-in liquid crystal panel 301, will be described.

  The liquid crystal panel drive circuit 304 is a circuit for driving the pixel circuit 31, and includes a scanning signal line drive circuit 3041 and a data signal line drive circuit 3042.

  The scanning signal line driving circuit 3041 sequentially selects one scanning signal line from the scanning signal lines G1 to Gm for each line time based on the timing control signal TC1 received from the liquid crystal panel control unit 601. A high level voltage is applied to the selected scanning signal line, and a low level voltage is applied to the other scanning signal lines.

  Based on the display data D (DR, DG, and DB) received from the liquid crystal panel controller 601, the data signal line driver circuit 3042 applies a predetermined voltage corresponding to the display data for one row for each line time. Are applied to the data signal lines SR1 to SRn, SG1 to SGn, and SB1 to SBn (line sequential method). Note that the data signal line driver circuit 3042 may be driven by a dot sequential method.

  The optical sensor driving circuit 305 is a circuit for driving the optical sensor circuit 32. Based on the timing control signal TC2 received from the sensor control unit 602, the optical sensor driving circuit 305 selects a predetermined sensor readout signal line from the sensor readout signal lines RW1 to RWm for each line time. A read voltage is applied, and a voltage other than a predetermined read voltage is applied to the other sensor read signal lines. Similarly, based on the timing control signal TC2, a predetermined reset voltage is applied to the sensor reset signal line selected from the sensor reset signal lines RS1 to RSm for each line time, and the others. A voltage other than a predetermined reset voltage is applied to the sensor reset signal line.

  The sensing data output signal lines SP1 to SPn are grouped into p groups (p is an integer of 1 to n), and the sensing data output signal lines belonging to each group are connected via a switch 47 that is sequentially turned on in time division. And connected to the sensor output amplifier 44. The sensor output amplifier 44 amplifies the voltage from the group of sensing data output signal lines connected by the switch 47 and outputs the amplified voltage to the signal conversion circuit 306 as sensor output signals SS (SS1 to SSp).

(Configuration of sub-side main control unit 830)
Next, the configuration of the sub-side main control unit 830 included in the sensor-equipped sub-display device 230 (hereinafter abbreviated as the sub-display device 230) will be described with reference to FIG. FIG. 1 is a functional block diagram showing a configuration of the sub-side main control unit 830. As shown in the figure, the sub-side main control unit 830 includes a partial image processing unit (processing unit) 801, a search signal generation unit 802, a list generation unit 803, a transmission destination determination unit 804, a transmission image determination unit (image determination unit). ) 805 and a display image generation unit (display control means) 806.

  The partial image processing unit 801 processes the partial image displayed on the sensor built-in liquid crystal panel 301 in accordance with a user instruction. In particular, the partial image processing unit 801 writes a line along the locus of the fingertip of the user touching the sensor built-in liquid crystal panel 301 in the partial image. Details of processing in the partial image processing unit 801 will be described later.

  The search signal generation unit 802 searches for (detects) a partner terminal in a standby state that can communicate with the own apparatus by the short-range wireless communication unit 915. The Bluetooth (registered trademark) device address that is identification information of the partner terminal is used. A search signal for inquiring (hereinafter referred to as a BD address) is transmitted by broadcast from the short-range wireless communication unit 915 via the communication control unit 933.

  The partner terminal that has received the search signal transmits a response signal including the BD address of the partner terminal to the sub display device 230. When the sub display device 230 receives the response signal, the electric field strength detection unit 936 measures the electric field strength of the response signal, and outputs the measurement result to the list generation unit 803 in association with the response signal.

  The list generation unit 803 generates a BD address list 91 listing the BD addresses included in the received response signal, and transmits the BD address list 91 to the main display device 210, thereby indicating the BD address list 91. The main display device 210 is requested for the user name (target user name) corresponding to the BD address.

  At this time, the list generation unit 803 determines whether or not the electric field strength of the response signal measured by the electric field strength detection unit 936 is equal to or higher than a predetermined value. The communicable information is recorded in the BD address list 91 in association with the BD address included in the response signal. The list generation unit 803 records communication difficulty information indicating that communication is difficult if the electric field strength is smaller than a predetermined value in the BD address list 91 in association with the BD address included in the response signal. .

  Note that the BD address included in the response signal whose electric field strength is smaller than a predetermined value may not be recorded in the BD address list 91.

  As described above, search signal generation section 802 and list generation section 803 function as identification information acquisition means for detecting a partner terminal that can communicate and acquiring identification information of the partner terminal. The list generation unit 803 functions as a request unit that requests the main display device 210 for the name of the user corresponding to the identification information.

  The transmission destination determination unit 804 determines which transmission destination in the transmission destination list displayed on the sensor built-in liquid crystal panel 301 is selected by the user, and sets the BD address corresponding to the transmission destination selected by the user as the transmission image determination unit. To 805. Details of the processing in the transmission destination determination unit 804 will be described later.

  The transmission image determination unit 805 transmits an information request signal for requesting display size information to the counterpart terminal having the BD address transmitted from the transmission image determination unit 805 via the communication control unit 933. When receiving the display size information from the counterpart device, the transmission image determination unit 805 determines whether to transmit a composite image or a partial image to the counterpart device based on the display size information.

  Specifically, when the counterpart device has a display unit of a predetermined size or larger, an instruction to transmit a composite image is output to the main display device 210, and the display unit has a display unit smaller than the predetermined size. In this case, an instruction to transmit the partial image is output to the communication control unit 933.

  The predetermined size is a size serving as a reference for distinguishing between a large display unit included in a PC or the like and a small display unit included in a mobile phone or the like, and may be set as appropriate.

  The display image generation unit 806 generates a display image to be displayed on the sensor built-in liquid crystal panel 301 and transmits the display image to the display data processing unit 701 to display the display image on the sensor built-in liquid crystal panel 301.

(Configuration of main side main control unit 810)
Next, the configuration of the main side main control unit 810 will be described with reference to FIG. FIG. 18 is a functional block diagram showing the configuration of the main main control unit 810. As shown in the figure, the main main control unit 810 includes a partial image generation unit 811, a partial address book generation unit (partial address information generation unit) 812, an image determination unit 813, a composite image generation unit (combination unit) 814, A display image generation unit 815 and a mail transmission / reception unit 816 are provided.

  The partial image generation unit 811 generates a partial image 93 that is a part of the entire image in accordance with a user instruction input via the operation unit 919. For example, the partial image generation unit 811 cuts out a range designated by the user by dragging the mouse from the entire image and sets it as a partial image. This partial image is transmitted to the sub display device 230.

  The partial address book generation unit 812 refers to the address book (address information) 95 stored in the storage unit 911, so that the counterpart terminal having the BD address shown in the BD address list 91 transmitted from the sub display device 230 A partial address book (partial address information) in which the user name (target user name) is associated with the BD address and the communicable / difficult information is generated.

  The address book 95 is a table in which the user name of one or more partner terminals, the BD address of the partner terminal, and the e-mail address of the partner terminal are associated with each other. The user can edit the address book 95 via the operation unit 919. In addition, user information such as the user's address and telephone number may be recorded in the address book 95.

  Since the sub display device 230 does not include an operation unit and it is difficult to edit the address book, the address book 95 is stored in the storage unit 911 included in the main display device 210 including the operation unit 919.

  FIG. 19 is a diagram conceptually illustrating an example of a partial address book. In the example shown in the figure, the user name “A-kun”, the BD address of the terminal possessed by “A-kun”, and the communicable information are associated with each other, the user name “B-kun”, and “ The BD address of the terminal possessed by “Mr. B” is associated with the communication difficulty information.

  The image determination unit 813 determines whether or not partial image identification information indicating a partial image is attached to an image received from the counterpart terminal or an e-mail attached with the image.

  This partial image identification information is given to the partial image or an e-mail attached with the partial image by the mail transmitting / receiving unit 816 when the main display device 210 transmits the partial image to the counterpart terminal. Further, the partial image identification information may be incorporated in the file name of the partial image, may be incorporated in the partial image file such as a location where the creator information of the partial image is recorded, It may be incorporated in a header or the like, and the partial image may be transmitted to the counterpart terminal together with the partial image identification information. Further, the partial image identification information may be given to the partial image by the communication control unit 933, for example, when the sub display device 230 transmits the partial image to the counterpart device. Further, the partial image identification information may be given according to a user instruction, and the timing and method for giving the partial image identification information are not particularly limited.

  The image determination unit 813 transmits the partial image to the sub display device 230 when the partial image identification information is added to the image or e-mail received from the counterpart device (that is, when the image is a partial image). . On the other hand, when the partial image identification information is not given, the image determination unit 813 outputs the image to the display image generation unit 815 to display the image on the liquid crystal panel 311.

  In response to an instruction to generate a composite image from the sub display device 230, the composite image generation unit 814 combines the partial image transmitted from the sub display device 230 and the entire image corresponding to the partial image to generate the composite image. Generate.

  The display image generation unit 815 generates display data to be displayed on the liquid crystal panel 311 and outputs the display data to the display data processing unit 710.

  The mail transmission / reception unit 816 functions as mail software for exchanging electronic mail with the partner apparatus.

(Details of processing in partial image processing unit 801)
When the partial image transmitted from the partial image generation unit 811 of the main display device 210 is displayed on the sensor built-in liquid crystal panel 301, the partial image processing unit 801 of the sub display device 230 sets “coordinate” in the “data type” field. The command in the sub display device designated and “all” or “event” in the “data acquisition timing” field is transmitted to the data processing unit 730. The sensor data processing unit 703 transmits coordinate data indicating the position of the image of the user's fingertip in the captured image captured by the sensor built-in liquid crystal panel 301 to the partial image processing unit 801 at predetermined time intervals. Then, the partial image processing unit 801 writes a line along the locus of the user's fingertip indicated by the coordinate data transmitted from the sensor data processing unit 703 in the partial image. The thickness and color of the line to be entered at this time may be set according to the user's specification.

  In addition, characters, preset stamps (various graphics, character images, etc.), outer frames, etc. can be added to the partial images. Character input may be performed via the operation unit 919 of the main display device 210. As for stamps, outer frames, etc., a list of usable ones may be displayed and the user may select them.

  The partial image processing unit 801 stores the processed partial image (reference numeral 93 in FIG. 1) in the storage unit 931.

(Details of processing in transmission destination determination unit 804)
When receiving the partial address book 94 from the partial address book generating unit 812 of the main display device 210, the transmission destination determining unit 804 of the sub display device 230 stores the partial address book 94 in the storage unit 931 and also displays a display image generating unit. Output to 806.

  The display image generation unit 806 generates a display image indicating a list of communication partner terminals (transmission destination list) based on the received partial address book 94, and outputs the display image to the display data processing unit 701. The list is displayed on the sensor built-in liquid crystal panel 301. It is preferable that the transmission destination list also indicates whether or not the short-range wireless communication unit 915 can communicate.

  When the destination address determination unit 804 outputs the partial address book 94 to the display image generation unit 806, the sub-display device that specifies “coordinate” in the “data type” field and “event” in the “data acquisition timing” field The inner command is transmitted to the data processing unit 730. When the coordinate data is received from the sensor data processing unit 703, the position indicated by the coordinate data is compared with the display position of the transmission destination list in the sensor built-in liquid crystal panel 301 (the position of the display area indicating each transmission destination). It is determined which destination the user has selected.

  The transmission destination determination unit 804 transmits the BD address corresponding to the transmission destination selected by the user to the transmission image determination unit 805.

(Flow of image transmission processing)
Next, an example of the flow of image transmission processing in the two-screen display device 200 will be described with reference to FIG. FIG. 20 is a diagram illustrating an example of the flow of image transmission processing in the two-screen display device 200. The communication between the main display device 210 and the sub display device 230 is performed via the communication control unit 913, the wired communication unit 914, the wired communication unit 934, and the communication control unit 933. In order to simplify the description, description of these members may be omitted. In addition, it is assumed that the entire image is stored in the storage unit 911 in advance.

  First, the display image generation unit 815 of the main display device 210 displays the entire image on the liquid crystal panel 311 in accordance with a user instruction (S1).

  The user designates a partial area of the entire image by dragging using a mouse or the like, and inputs an instruction to generate a selected partial image (for example, select a process such as “Transfer to sub screen” from a pop-up menu or the like. The partial image generation unit 811 generates a partial image including the region, and transmits the generated partial image together with the display command to the display image generation unit 806 of the sub display device 230 (S2).

  Upon receiving the display command and the partial image, the display image generation unit 806 generates a display image including the partial image and displays it on the sensor built-in liquid crystal panel 301 (S3).

  When the partial image is displayed on the sensor built-in liquid crystal panel 301, the partial image processing unit 801 designates “coordinate” in the “data type” field and designates “all” or “event” in the “data acquisition timing” field. The sub-display device command is transmitted to the data processing unit 730. Then, the partial image processing unit 801 writes a line along the locus of the fingertip of the user indicated by the coordinate data transmitted from the sensor data processing unit 703 in the partial image (S4). The partial image processing unit 801 stores the processed partial image in the storage unit 931.

  Thereafter, when an instruction to transmit a partial image is acquired from the user, the search signal generation unit 802 searches for a search signal for inquiring about the BD address of a partner terminal in a standby state that can be communicated with the own device by the short-range wireless communication unit 915. It transmits from the distance radio | wireless communication part 915 (S5).

  The counterpart terminal (referred to as counterpart terminals A and B) that has received the search signal transmits a response signal including the BD address of the counterpart terminal to the sub display device 230 (S6).

  When receiving the response signal, the electric field strength detection unit 936 measures the electric field strength of the response signal and outputs the measurement result to the list generation unit 803 in association with the response signal.

  The list generation unit 803 determines whether or not the electric field strength of the response signal measured by the electric field strength detection unit 936 is equal to or higher than a predetermined value, and if it is equal to or higher than the predetermined value, communicable information indicating that communication is possible. Is recorded in the BD address list 91 in association with the BD address included in the response signal. On the other hand, if the electric field strength is smaller than a predetermined value, the list generation unit 803 records communication difficulty information indicating that communication is difficult in the BD address list 91 in association with the BD address included in the response signal. .

  Then, the list generation unit 803 transmits the generated BD address list 91 to the partial address book generation unit 812 of the main display device 210 together with a processing command instructing generation of the partial address book (S7). At this time, the BD address list 91 transmitted to the main display device 210 does not necessarily include communicable / difficult information.

  When the partial address book generation unit 812 receives the processing command and the BD address list 91, the partial address book generation unit 812 has the BD address indicated in the received BD address list 91 by referring to the address book 95 stored in the storage unit 911. A partial address book in which the user name (target user name) of the counterpart terminal is associated with the BD address and communicable / difficult information is generated. The partial address book generation unit 812 outputs the generated partial address book to the transmission destination determination unit 804 of the sub display device 230 (S8). In other words. The partial address book generation unit 812 transmits the target user name requested from the sub display device 230 to the sub display device 230 by accessing the storage unit 911.

  When receiving the partial address book 94 from the partial address book generation unit 812, the transmission destination determination unit 804 stores the partial address book 94 in the storage unit 931 and outputs it to the display image generation unit 806.

  The display image generation unit 806 outputs the transmission destination list display image indicated by the partial address book 94 to the display data processing unit 701, thereby displaying the transmission destination list on the sensor built-in liquid crystal panel 301 (S9).

  After that, the transmission destination determination unit 804 transmits a command in the sub display device in which “coordinate” is specified in the “data type” field and “event” is specified in the “data acquisition timing” field to the data processing unit 730. The sensor data processing unit 703 transmits coordinate data indicating the position of the image of the user's fingertip in the captured image captured by the sensor built-in liquid crystal panel 301 to the transmission destination determination unit 804. When receiving the coordinate data from the sensor data processing unit 703, the transmission destination determining unit 804 compares the position indicated by the coordinate data with the display position of the transmission destination list in the sensor built-in liquid crystal panel 301, so that the user can It is determined which destination (user name) has been selected (S10).

  The transmission destination determination unit 804 transmits the BD address corresponding to the transmission destination selected by the user to the transmission image determination unit 805 by referring to the partial address book 94 stored in the storage unit 931.

  Upon receiving the BD address, the transmission image determination unit 805 transmits an information request signal for requesting display size information to the partner terminal having the BD address (S11). Here, it is assumed that the user has selected partner terminal A as the transmission destination. In this case, the information request signal is transmitted to the counterpart terminal A.

  When the counterpart terminal A receives the information request signal, it transmits display size information indicating the size of the display unit it has to the sub display device 230 (S12). Here, it is assumed that the counterpart terminal A is a mobile terminal and includes a small display unit.

  When receiving the display size information from the counterpart terminal A, the transmission image determination unit 805 determines whether the display size indicated by the display size information is equal to or larger than a predetermined size. When the counterpart terminal A is a mobile terminal, the display size indicated by the display size information is smaller than the predetermined size, so the transmission image determination unit 805 outputs an instruction to transmit the partial image to the communication control unit 933 (S13).

  The communication control unit 933 transmits the partial image 93 stored in the storage unit 931 to the counterpart terminal A via the short-range wireless communication unit 935 (S14). Since the BD address of the counterpart terminal A is shown in the partial address book 94, the communication control unit 933 may obtain the BD address of the counterpart terminal A by referring to the partial address book 94. At this time, the communication control unit 933 gives partial image identification information indicating that the image to be transmitted is a partial image to the partial image, and transmits the partial image to which the partial image identification information is given to the counterpart terminal A. Also good.

  An information request signal for requesting display size information may be included in the search signal, but obtaining the display size information only from the counterpart terminal that transmits the partial image requires less total data to be transmitted and received. preferable.

(Another example of the flow of image transmission processing)
Next, another example of the flow of image transmission processing in the two-screen display device 200 will be described with reference to FIG. FIG. 21 is a diagram illustrating another example of the flow of the image transmission process in the two-screen display device 200. Here, a process when the counterpart terminal A is a PC and includes a display unit having a display size equal to or larger than a predetermined size will be described.

  Since steps S21 to S32 shown in FIG. 21 are the same as steps S1 to S12, description thereof is omitted.

  When the counterpart terminal A transmits the display size information to the sub display device 230, the transmission image determination unit 805 determines whether or not the display size indicated by the display size information is equal to or larger than a predetermined size (S33). When the counterpart terminal A is a PC, the display size indicated by the display size information is equal to or larger than a predetermined size, so the transmission image determination unit 805 sends a processing command for transmitting the composite image to the counterpart terminal A on the main display device 210. The image is output to the composite image generation unit 814 (S34). At this time, the transmission image determination unit 805 transmits the BD address of the counterpart terminal A and the partial image 93 stored in the storage unit 931 to the composite image generation unit 814 together with the processing command.

  Upon receiving the processing command, the BD address, and the partial image 93, the composite image generation unit 814 combines the received partial image and the entire image corresponding to the partial image to generate a composite image (S35). More specifically, the composite image generation unit 814 generates a new whole image by incorporating the processed partial image into a region corresponding to the partial image in the whole image.

  Then, the composite image generation unit 814 outputs the generated composite image to the mail transmission / reception unit 816 together with the BD address of the transmission destination. The mail transmission / reception unit 816 transmits the composite image to the counterpart terminal A having the BD address via the wired / wireless network communication unit 916 (S36). At this time, the mail transmitting / receiving unit 816 may acquire the electronic mail address of the partner terminal A by referring to the address book 95. In the address book 95, a BD address and an e-mail address are associated with each other terminal.

  In the above description, the two-screen display device 200 transmits an image obtained by synthesizing the entire image and the partial image. However, when the image is actually transmitted, the two-screen display device 200 responds to the request of the destination partner terminal or the user. The combined image may be transmitted as described above, or only the partial image 93 may be transmitted without performing the above-described combining process, or the entire image 95 and the partial image 93 before processing may be transmitted. Both may be sent.

(Flow of image reception processing)
Next, the flow of processing when the main display device 210 receives an image to which partial image identification information is assigned will be described with reference to FIG. FIG. 22 is a diagram illustrating an example of the flow of image reception processing in the two-screen display device 200.

  When the mail transmission / reception unit 816 of the main display device 210 receives an electronic mail with an image attached thereto from the counterpart terminal A (S41), the mail transmission / reception unit 816 outputs the image or the electronic mail to the image determination unit 813. The image determination unit 813 determines whether or not partial image identification information is given to the image or e-mail (S42). When partial image identification information is given to the image or the e-mail, the image determination unit 813 transmits the partial image together with the display command to the sub display device 230 (S43).

  When the partial image is received together with the display command, the display image generation unit 806 displays the partial image on the sensor built-in liquid crystal panel 301 (S44).

  On the other hand, when partial image identification information is not given to the image or the e-mail, the image determination unit 813 instructs the display image generation unit 815 to display the partial image on the display unit 310.

  As described above, the two-screen display device 200 includes the main display device 210 and the sub display device 230 that can operate independently of the main display device 210. The main display device 210 is a liquid crystal that displays the entire image. The sub display device 230 includes a panel 311 (first display unit), and includes a sensor built-in liquid crystal panel 301 (second display unit) that displays a partial image of the entire image displayed on the liquid crystal panel 311. The sub display device 230 transmits the partial image together with the partial image identification information for identifying the partial image to the counterpart device capable of communicating with itself, and the main display device 210 receives the image together with the partial image identification information. In this case, the image is transmitted to the sub display device 230. The sub display device 230 displays the image transmitted from the main display device 210 on the sensor built-in liquid crystal panel 301.

  By such processing, when the main display device 210 receives an image to which partial image identification information is added, the image is transferred to the sub display device 230 without being displayed by the main display device 210. Therefore, the processing relating to the image can be omitted in the main display device 210, and the processing efficiency can be improved.

(Effect of the two-screen display device 200)
As described above, in the two-screen display device 200, the sub display device 230 that does not have an address book inquires of the main display device 210 about the user name of the partner terminal that can communicate with the own device, and the obtained user name is The display allows the user to select a partner terminal desired by the user from communicable partner terminals. Then, the sub display device 230 transmits data (for example, a partial image) to the partner terminal selected by the user.

  Therefore, even if the sub display device 230 does not have an address book, the data can be transmitted to the counterpart terminal to which the data is to be transmitted by cooperating with the main display device 210.

  Further, by generating a partial image from the entire image and transmitting the partial image to the counterpart terminal, even if the counterpart terminal is a portable terminal having a small display unit, the image can be viewed comfortably. In addition, since the partial image has a data capacity smaller than that of the entire image, the processing amount required for transmitting and receiving the image can be reduced.

(Example of change)
The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope shown in the claims, and embodiments obtained by appropriately combining technical means disclosed in the embodiments. Is also included in the technical scope of the present invention.

  For example, when the sub display device 230 cannot communicate with a partner device designated by the user by short-range wireless communication (second data communication), the main display device 210 performs wired / wireless network communication (wide area communication, first data communication). ) May communicate with the other terminal.

  In this case, the sub display device 230 transmits the processed partial image and the BD address of the counterpart terminal to which the partial image is transmitted to the main display device 210. The main display device 210 may acquire an e-mail address corresponding to the BD address by referring to the address book 95 and transmit a partial image to the e-mail address. At this time, the mail transmission / reception unit 816 may add partial image identification information indicating that the attached image is a partial image to the email to be transmitted.

  The data transmitted by the main display device 210 and the sub display device 230 is not limited to image data, and may be other data such as character data.

  In addition, each block of the above-described two-screen display device 200, in particular, the sub-side main control unit 830 and the main-side main control unit 810 may be configured by hardware logic, or by software using a CPU as follows. It may be realized.

  That is, the two-screen display device 200 includes a CPU (central processing unit) that executes instructions of a control program that realizes each function, a ROM (read only memory) that stores the program, and a RAM (random access memory) that expands the program. ), A storage device (recording medium) such as a memory for storing the program and various data. An object of the present invention is a recording medium in which a program code (execution format program, intermediate code program, source program) of a control program of the two-screen display device 200 which is software for realizing the above-described functions is recorded so as to be readable by a computer. Can also be achieved by reading the program code recorded on the recording medium and executing it by the computer (or CPU or MPU).

  Examples of the recording medium include a tape system such as a magnetic tape and a cassette tape, a magnetic disk such as a floppy (registered trademark) disk / hard disk, and an optical disk such as a CD-ROM / MO / MD / DVD / CD-R. Card system such as IC card, IC card (including memory card) / optical card, or semiconductor memory system such as mask ROM / EPROM / EEPROM / flash ROM.

  The two-screen display device 200 may be configured to be connectable to a communication network, and the program code may be supplied via the communication network. The communication network is not particularly limited. For example, the Internet, intranet, extranet, LAN, ISDN, VAN, CATV communication network, virtual private network, telephone line network, mobile communication network, satellite communication. A net or the like is available. Also, the transmission medium constituting the communication network is not particularly limited. For example, even in the case of wired such as IEEE 1394, USB, power line carrier, cable TV line, telephone line, ADSL line, etc., infrared rays such as IrDA and remote control, Bluetooth ( (Registered trademark), 802.11 wireless, HDR (high data rate), mobile phone network, satellite line, terrestrial digital network, and the like can also be used. The present invention can also be realized in the form of a computer data signal embedded in a carrier wave in which the program code is embodied by electronic transmission.

  Since a communication mode in which a plurality of independent communication units are linked can be realized, the present invention can be applied to a terminal device (for example, a personal computer, a mobile phone, a PDA (personal digital assistance), etc.) having a plurality of independent communication units.

It is a block diagram which shows the structure of the sub side main control part with which the sub display apparatus with a sensor with which the two-screen display apparatus which concerns on one embodiment of this invention is provided is provided. 2A and 2B are schematic views showing a specific example of a two-screen display device according to an embodiment of the present invention. It is a figure which shows typically the cross section of the liquid crystal panel with a sensor with which the sub-display apparatus with a sensor with which the two-screen display apparatus which concerns on one Embodiment of this invention is provided is equipped. FIG. 4A is a schematic diagram showing a state in which a position touched by the user is detected by detecting a reflected image with the sensor built-in liquid crystal panel. FIG. 4B is a schematic diagram showing how the position touched by the user is detected by detecting a shadow image with the sensor-equipped liquid crystal panel. It is a block diagram which shows the principal part structure of the main display apparatus with which the 2 screen display apparatus which concerns on one Embodiment of this invention is equipped, and the sub display apparatus with a sensor. It is a block diagram which shows the more detailed structure of the display / light sensor part with which the said sub display apparatus with a sensor is provided, a display / light sensor circuit control part, and a data processing part. It is a block diagram which shows the structural example of the display part with which the said main display apparatus is provided, and a display circuit control part. It is a figure which shows typically an example of the frame structure of the command in a sub display apparatus. It is a figure for demonstrating an example of the value which can be designated to each field contained in the command in a sub display apparatus, and the outline | summary. It is a schematic diagram which shows an example of the frame structure of the command between apparatuses. It is a figure which shows the information which can be designated to each field about each of a scan command, a display command, and a process command. It is a figure for demonstrating an example of the value which can be designated to the 1st-5th field of a scan command, and the outline | summary. It is a figure for demonstrating an example of the value which can be designated to the 1st and 2nd field of a display command, and the outline | summary. It is a figure for demonstrating an example of the value which can be designated to the 1st-3rd field of a process command, and the outline | summary. FIG. 15A is obtained as a result of scanning the entire sensor-equipped liquid crystal panel when the object is not placed on the sensor-equipped liquid crystal panel in the two-screen display device according to one embodiment of the present invention. Image data. FIG. 15B shows image data obtained as a result of scanning the entire sensor-equipped liquid crystal panel when the user touches the sensor-equipped liquid crystal panel with a finger in the data display / sensor device. It is a block diagram which shows the principal part of the display / light sensor part with which the two-screen display apparatus which concerns on one Embodiment of this invention is provided, especially the structure of the liquid crystal panel with a built-in sensor, and the structure of a peripheral circuit. It is a figure for demonstrating the outline | summary of the 2 screen display apparatus which concerns on one Embodiment of this invention. It is a block diagram which shows the structure of the main side main-control part with which the said data display / sensor apparatus is provided. It is a figure which shows an example of a partial address book notionally. It is a figure which shows an example of the flow of the image transmission process in the said 2 screen display apparatus. It is a figure which shows another example of the flow of the image transmission process in the said 2 screen display apparatus. It is a figure which shows an example of the flow of the image reception process in the said 2 screen display apparatus.

Explanation of symbols

93 Partial image 94 Partial address book (partial address information)
95 Address book (address information)
200 Two-screen display device (communication device)
210 Main display device (first communication unit)
230 Sub display device with sensor (second communication unit)
310 Display section (first display section)
330 display / light sensor unit (second display unit)
801 Partial image processing unit (processing means)
805 Transmission image determination unit (image determination means)
814 Composite image generation unit (compositing means)
911 storage unit

Claims (11)

A communication device comprising a first communication unit and a second communication unit operable independently of the first communication unit,
The first communication unit includes a storage unit that stores address information in which identification information of a counterpart device that is a communication destination device is associated with a name of a user of the counterpart device,
When the second communication unit detects a partner device communicable with itself and acquires identification information of the partner device, the second communication unit assigns a target user name that is a user name corresponding to the identification information to the first communication unit. To request and
The first communication unit transmits information including the target user name requested from the second communication unit to the second communication unit by referring to the address information stored in the storage unit. Communication device. The second communication unit transmits the acquired identification information to the first communication unit as the request,
The first communication unit refers to the address information stored in the storage unit, so that the target user name corresponding to the identification information transmitted from the second communication unit is associated with the identification information. 2. The communication apparatus according to claim 1, wherein address information is generated and transmitted to the second communication unit. The first communication unit includes a first display unit that displays an entire image,
The second communication unit includes a second display unit that displays a partial image of the entire image displayed on the first display unit, and transmits the partial image displayed on the second display unit to the counterpart device. The communication device according to claim 1, wherein: The first or second communication unit transmits the partial image together with the partial image identification information for identifying the partial image to the counterpart device,
When the first communication unit receives an image together with the partial image identification information from the counterpart device, the first communication unit transmits the image to the second communication unit,
The communication device according to claim 3, wherein the second communication unit displays an image transmitted from the first communication unit on the second display unit.   5. The communication device according to claim 3, wherein the second communication unit further includes a processing unit that processes the partial image displayed on the second display unit.   The first communication unit further includes a combining unit that combines the partial image processed by the processing unit and the whole image, and transmits the combined image combined by the combining unit to the counterpart apparatus. The communication device according to claim 5. The second communication unit further includes image determining means for determining whether to transmit the partial image or the composite image to the counterpart device based on the display size of the display unit of the counterpart device.
The communication apparatus according to claim 6, wherein the image determined by the image determination unit is transmitted to the counterpart apparatus. The first communication unit performs first data communication,
The second communication unit performs second data communication in a manner different from the first data communication,
The first communication unit communicates with the counterpart device through the first data communication when the second communication unit cannot communicate with the counterpart device through the second data communication. The communication device according to any one of the above.   9. The communication apparatus according to claim 8, wherein the first data communication is wide-area communication, and the second data communication is short-range wireless communication.   The communication device according to claim 1, wherein the second communication unit is detachably provided. A control method for controlling a communication device comprising a first communication unit and a second communication unit operable independently of the first communication unit,
The first communication unit includes a storage unit that stores address information in which identification information of a counterpart device that is a communication destination device is associated with a name of a user of the counterpart device,
When the second communication unit detects a partner device communicable with itself and acquires identification information of the partner device, the target user name that is the name of the user corresponding to the identification information is set as the first communication unit. The required process required by
Target user name transmission in which the first communication unit transmits information including the target user name requested from the second communication unit to the second communication unit by referring to the address information stored in the storage unit A control method comprising the steps of:

Images