WO2018159579A1 - Display/operation terminal and display/operation separation system using same - Google Patents

Display/operation terminal and display/operation separation system using same Download PDF

Info

Publication number
WO2018159579A1
WO2018159579A1 PCT/JP2018/007129 JP2018007129W WO2018159579A1 WO 2018159579 A1 WO2018159579 A1 WO 2018159579A1 JP 2018007129 W JP2018007129 W JP 2018007129W WO 2018159579 A1 WO2018159579 A1 WO 2018159579A1
Authority
WO
WIPO (PCT)
Prior art keywords
display
terminal
unit
operation
electronic device
Prior art date
Application number
PCT/JP2018/007129
Other languages
French (fr)
Japanese (ja)
Inventor
晃 高嶋
史朗 菅原
薫 酒井
千登勢 中西
江藤 正幸
由紀子 大湊
Original Assignee
株式会社teamS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority to JP2017-035891 priority Critical
Priority to JP2017035891 priority
Application filed by 株式会社teamS filed Critical 株式会社teamS
Publication of WO2018159579A1 publication Critical patent/WO2018159579A1/en

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09GARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04MTELEPHONIC COMMUNICATION
    • H04M1/00Substation equipment, e.g. for use by subscribers; Analogous equipment at exchanges

Abstract

In order to improve the ease of use of display and input with an electronic apparatus, an embodiment of the present invention provides a display/operation terminal provided with a terminal receiving unit 130, a terminal display unit 110, a touch operation input unit 120, and a terminal transmission unit 140. The terminal receiving unit receives, from another electronic apparatus 200, a display signal 132 indicating display content, and the terminal display unit presents, to the user and on a display area 110a, a still image and/or video, or a combination thereof, forming the display content indicated by the display signal. The touch operation input unit receives an input from a touch operation by the user, and generates a touch operation signal, and the terminal transmission unit transmits an operation input signal 142 containing the touch operation signal to the electronic apparatus. The user can check the display content or perform a touch operation input as if using the electronic apparatus, while using the display/operation terminal. The present invention also provides a display/operation separation system including the display/operation terminal and the electronic apparatus.

Description

Display operation terminal and display operation separation system using the same

The present invention relates to a display operation terminal and a display operation separation system using the same. More specifically, the present invention relates to a display operation terminal that improves convenience in display and input of an electronic device, and a display operation separation system using the display operation terminal.

In recent years, against the backdrop of high performance and portability of IT equipment, computer equipment that can provide various information contents (contents) to users and further support user actions (in this application, “electronic equipment” ") Is widely used. In particular, if you use a wideband mobile communication network such as LTE (Long Term Evolution) or 4G (4th Generation), or an electronic device with such communication capability in an environment where a public or private wireless LAN is provided In addition, an Internet connection almost free of restrictions on place and time can be realized, and high convenience can be provided to the user. The high-quality interface (Human Computer Interaction) of each electronic device supports the convenience as well as the Internet. High-definition and beautiful display, touch input operation that enables input and operation corresponding to such display, and control software that closely links them are carefully tuned to give the user a natural feeling of use. Intuitive and sophisticated usability is realized so that the user forgets to operate the computing device. As a result, a wide range of users who have been difficult to spread in traditional character-based and keyboard-based computing environments are also benefiting from the development of IT equipment.

At present, electronic devices that provide the above-described computing environment are typically portable PCs such as smartphones, laptop personal computers (PCs), and tablet terminals. However, the use of these electronic devices by users involves various restrictions. A typical example is restrictions based on hardware specifications such as the weight of electronic devices, the power source duration of batteries, and the size. A smartphone has a voice call function as a mobile station, and therefore has a strong restriction on a size suitable for being always carried. Due to this upper size limit, display devices on smartphones are still not always sufficient despite destroying the keyboard and occupying the largest surface of the plate-like housing. Elderly people are difficult to read small letters, and the content that can be enjoyed on a small screen is limited. In addition, the power that can secure the operation time under the strong restrictions is a trade-off with the operation time of voice call and its standby, and only a part thereof is distributed to display and computing. In exchange for high computing capacity, portable PCs are large in size and have a battery that can consume power with a practical duration, and those weighing less than 1 kg are commercially available, but they are heavy to hold in hand for a long time. The tablet terminal may be considered to be a large-sized one obtained by removing the keyboard from the PC, or from the smartphone excluding the voice call function. For example, the tablet terminal is thin with a display device having a diagonal length of about 328 mm = 12.9 inches. However, users are still strongly aware of the hardware weight and power supply constraints. Any tablet device that provides a sufficiently large display is too heavy to hold for a long time.

As a slightly different approach, there is also a system that is established by shifting the balance between the trade-off elements under the above constraints in exchange for the limitation to the exclusiveness of the application. For example, electronic books, which have been distributed as publications in the past, are now being downloaded and sold as electronic data via the Internet. Display devices employed in terminals used in electronic book systems (“electronic book terminals”) are mainly electronic paper and liquid crystal display devices. Electronic paper ensures the fineness of character-based offset printing as long as it is limited to monochrome and still image display. Even if it is a monochrome still image display, if display with a certain quality is possible, restrictions such as securing a power source and charging, weight, and size related to batteries are less likely to be conscious. At the same time, the e-book system reduces the weight of publications, the time and effort of management, and the problem of distribution inventory, which greatly benefits users and publishers. However, in reality, the display on the current electronic book terminal that employs electronic pavers is such that the display of text-based displays in publications can be viewed without stress. For today's general consumers who are often exposed to rich content, such as color video displays, a screen that is smaller than a traditional publication, although it is an electronic device, can sometimes only be tolerated. However, if a liquid crystal display device is employed, the same problems as those of the PC and tablet terminal described above are encountered.

Some attempts have been made to separate some functions in order to remove the limitations of current electronic devices. For example, a watch-type terminal (smart watch) premised on cooperation with a smartphone and a glasses-type wearable display device are conceived and some of them are mass-produced. However, at the present time, it has not reached widespread use. “Ideal always-on-carrying” by wearing on the body is not accepted by users without resistance. In addition, in the current situation where similar sizes and styles are adopted to the old wristwatch, it can be used as a calling device in cooperation with a smartphone, and heartbeat logs and GPS logs can be easily taken from the body However, it does not solve the problems of conventional electronic devices.

An example of a screen transfer control device for a smartphone display is disclosed in Patent Document 1 (Utility Model Registration No. 3172006).

Utility Model Registration No. 3172006

This invention makes it a subject to solve at least any one of the various subject mentioned above. The present invention provides a display operation terminal that can utilize a display function and a touch input function that are the same as or similar to those mounted on an electronic device, and a display operation separation system using the display operation terminal. , It makes the user feel as much as possible and draws out the capabilities of electronic devices without any hesitation. As a result, the present invention makes it possible to exert the original appeal even if the content has been halved by the fact that the electronic device has only a small screen. This will contribute to further expansion of the user base. In addition, the present invention feels difficult to use in conventional small electronic devices (for example, smartphones) for physical reasons, or difficult to use in heavy electronic devices (for example, conventional tablet terminals). It also contributes to the expansion of the user base to the people who come.

The present inventor has realized that if an appropriately designed display and operation device can be realized, the ability of the electronic device can be utilized to provide high convenience to the user. For example, for users who own smartphones, if they can use smartphones like tablet devices, and if they can realize a light user experience that cannot be achieved with tablet devices, they will be provided with high convenience. The If you are a smartphone user, there is little necessity to buy more tablet terminals with overlapping functions, but you can further enhance the convenience of existing smartphones by adding a display operation device that can further expand the convenience of smartphones If you can, you will find it useful. The present inventor then displays and touch-inputs a dedicated terminal device for presenting information to the user and accepting input from the user, which is equivalent to an earphone or headset for a device for enjoying music. As a result, the present invention has been completed.

That is, in an aspect of the present invention, a terminal receiving unit that receives a display signal indicating display content from at least one electronic device, and the display content indicated by the display signal is at least one of a still image and a moving image, or those A terminal display unit having a display area to be presented to the user by combination, a touch operation input unit that receives an input by a touch operation from the user and generates a touch operation signal, and an operation input signal including the touch operation signal A terminal transmission unit that transmits to the electronic device; a communication control unit that controls establishment and release of connection with the electronic device by at least one of the terminal reception unit and the terminal transmission unit; and the terminal reception unit. , It can measure the intensity of receivable types of radio waves, and send a detection signal indicating the radio wave intensity received by the terminal receiver A wave intensity detector, and the communication controller controls at least one of the terminal receiver and the terminal transmitter based on the detection signal from the radio wave intensity detector, and the electronic device A display operation terminal for establishing the connection is provided.

The display operation terminal of the above aspect can be manufactured in a light weight with a wide display area in principle. This is because capabilities other than the minimum functions necessary for display and input and communication for them can be realized by utilizing the functions of the electronic device. From another viewpoint, for example, a display unit or touch input unit of a size that cannot be used in the past is used for the display operation terminal of the above aspect in comparison with a smartphone or the like that has a practical upper limit. Can do. For this reason, it is possible to provide the user with an easy-to-use operating environment having a larger size for utilizing various functions of the electronic device. In a specific embodiment, since a terminal having minimum necessary functions such as display and touch input is provided, realistic restrictions and obstacles when aiming to reduce the size while reducing the size are removed. The display operation terminal can be thinned to a thickness of about several millimeters, and can be light and flexible, providing an extremely light feeling of use. Ultimately, it would also be possible to realize the appearance of resin boards that are used as “pencil boards” placed on the back of paper when writing on paper with a writing instrument.

In another aspect of the present invention, there is provided the display operation terminal according to the above aspect, an electronic device including a device transmission unit that transmits the display signal, and a device reception unit that receives the operation input signal, Display contents for the electronic device are transmitted to the display operation terminal, and a touch operation input for the display operation terminal is transmitted to the electronic device, thereby displaying the display contents of the electronic device and a touch operation for the display contents. There is also provided a display operation separation system that is executed in the display operation terminal.

The hardware of the electronic device included in the display operation separation system of the above aspect can be assumed to be a smartphone equipped with an appropriate communication module, a portable PC such as a laptop PC, and a tablet terminal. In order for these electronic devices to function as the electronic devices of the display operation separation system according to the above aspect, a function unit for appropriately transmitting display contents and receiving a touch operation input from the display operation terminal is a display operation terminal. Is required. For example, in order to make a portable PC such as a smartphone, a laptop PC, and a tablet terminal into an electronic device included in the display operation separation system, in addition to being equipped with an appropriate module (for example, a communication module) as hardware. Software for making these modules cooperate with the display operation terminal is installed in the electronic device. As a result, these electronic devices can be used in cooperation with the display operation terminal of the above aspect. This enhances the convenience for the user, and also affects the request from the user for the electronic device itself. Taking a smartphone as an example, the display function and input function required for the smartphone itself will not be as important as before, and for example, the user will begin to think that a minimum display size is sufficient. Even in that case, when viewed as a whole display operation separation system combining them, a lighter display operation terminal having an enlarged display unit is provided, which is much higher than conventional electronic devices. Convenience can be realized.

In the entire description of the present application, the electronic devices include, as representative examples, portable PCs such as smartphones and laptop PCs, and tablet terminals, but are not limited thereto. Furthermore, each element for defining the invention can be realized by arbitrarily combining software and hardware. For example, an element described in the description of the present application by the expression “to part” such as a display unit may be one or more hardware units to realize a target function, and if necessary, In some cases, it is realized by combining one or more units of software components and one or more units of data or information used to realize the characteristics of the hardware as operations or procedures. One unit of the software component at this time includes an application program, an execution library, an API (Application Programming Interface), a protocol stack, a device driver, and the like, and includes a combination of two or more thereof. Even elements defined by the same expression may be realized by different hardware of the same type, and different elements specified by different expressions may be realized by one hardware. In addition, an operation may be specified to define the characteristics of each element, and the order in the operation is not necessarily described in the execution order.

In any aspect of the present invention, a highly practical technique for maximizing convenience for the user while removing restrictions on electronic devices is provided.

It is a schematic diagram which shows the whole structure of the display operation separation system in embodiment of this invention. It is a schematic diagram which shows the structure in the case of mounting the function of an automatic start in the display operation terminal using radio | wireless communication in embodiment of this invention. It is a schematic diagram which shows the structure in the case of mounting a some communication protocol in the display operation terminal in embodiment of this invention, and performing leveling according to the trial order designated beforehand, and performing mirroring. It is a schematic diagram which shows the structure of the display operation separation system which performs pairing operation | movement by wired communication in embodiment of this invention. It is a schematic diagram which shows the structure of the display operation terminal and electronic device which directly connect in embodiment of this invention. It is a schematic diagram which shows a mode that operation | movement of the display part (apparatus display part) of an electronic device is controlled in an electronic device for the mirroring in embodiment of this invention. It is a schematic diagram which shows the structure for utilizing a display operation terminal during a voice call, when the electronic device in embodiment of this invention provides a voice call function. It is a schematic diagram which shows the structure in the case of displaying a touch-type keyboard with a display operation terminal in the display operation separation system in embodiment of this invention. It is a schematic diagram which shows the structure of the display operation separation system used for unit content in embodiment of this invention. It is a schematic diagram which shows the typical example of operation | movement when the number of electronic devices is 1 unit and the number of display operation terminals is 2 or more in the embodiment of the present invention. In the embodiment of the present invention, it is a schematic diagram showing a configuration of a display operation separation system in which the number of electronic devices is two or more and the number of display operation terminals is one.

Hereinafter, an embodiment relating to a display operation separating system including a display operation terminal according to the present invention will be described with reference to the drawings. Unless otherwise stated in the description, common portions or elements are denoted by common reference symbols. In the drawings, each element of each embodiment is not necessarily shown in a scale ratio.

1. Overall Configuration A display operation separation system 10 is provided as a system that is the basis of the present embodiment. FIG. 1 is a schematic diagram showing the overall configuration of a display operation separating system 10 of the present embodiment. The display operation separation system 10 typically includes a display operation terminal 100 and an electronic device 200.

2. Configuration of Terminal The display operation terminal 100 includes a terminal reception unit 130, a display unit 110, a touch operation input unit 120, and a terminal transmission unit 140. The terminal receiving unit 130 receives a display signal 132 indicating display content from at least one electronic device 200. The display unit 110 has a display area 110a that presents the display content indicated by the display signal 132 to the user by at least one of a still image and a moving image, or a combination thereof. At least one of a still image and a moving image or a combination thereof includes arbitrary display information such as an image, video, text data, and computer graphic display that can be viewed in the display area 110a. The touch operation input unit 120 receives an input by a touch operation from a user and generates a touch operation signal. The terminal transmission unit 140 transmits an operation input signal 142 including a touch operation signal to the electronic device 200. A typical example of display contents displayed on the display unit 110 of the display operation terminal 100 is the same as the display contents to be displayed on the electronic device 200 and the display contents that should have been displayed. The function of presenting such display contents on a display unit of another device and multiplexing or transferring the display contents is called mirroring in the prior art, and the term is used in this application.

2-1. Display Unit The display unit 110 is an arbitrary display device such as an organic EL display (OLED), for example, and the detailed specifications of the display area 110a of the display unit 110 in the display operation terminal 100 are determined in consideration of various viewpoints. . For example, when the electronic device 200 is a smartphone having the same hardware specifications as that of a conventional device, the size of the electronic device 200 is generally in a range that does not interfere with carrying at all times. Inevitably, the display area 210a of the display unit 210 becomes smaller, and for example, a diagonal of about 152 mm = 6 inches is an upper limit guideline. When combined with such an electronic device 200, if the display operation terminal 100 has a sufficient size, a display area 110a having a larger display capacity (total number of pixels) than that when the electronic device 200 is used alone is provided. Since the display unit 110 having the display unit 110 can be mounted, it is possible to provide a user with a wide area display that is not limited by the size of the display unit 210. Thus, the size of the display area 110a is important in that it is an incentive for the user to add the display operation terminal 100 in addition to the electronic device 200. If the specific size of the display operation terminal 100 is, for example, a rectangle of 257 mm × 182 mm (B5 size in Japan, which is close to the size of a book in Japan), it brings the user ease of use in the device used by the hand. Further, when the display operation terminal 100 is implemented and manufactured, the viewpoint of manufacturing efficiency of the display unit employed as a component is also taken into consideration. For example, one of the sizes having an area equivalent to 257 mm × 182 mm and capable of efficiently producing a display device is a rectangle of 270 mm × 152 mm (diagonal 310 mm = 12.2 inches). This size has a screen ratio of 16: 9, which is one of the best for viewing movie content. It is also possible to prioritize the display size over the ease of holding for the hand. For example, providing the display unit 110 of the same size as that of A4 or letter-size paper is also useful for applications that refer to paper-oriented materials. The viewpoint of the balance between the convenience and the manufacturing efficiency described here is appropriately determined according to the intended use including whether the user picks it up, the user group, or the allowable cost.

Note that the display unit 110 is not limited to an organic EL display, and a liquid crystal display device or any other display component can be employed. The display operation terminal 100 has few elements that consume power other than display, input, and communication related thereto. For this reason, the ratio of the electric power allocated for a display apparatus is comparatively high, and the restrictions in selection of the display part 110 are weak at that point. Further, if a display device with low power consumption is selected, the weight allocated to the power source such as a battery can be reduced, and the battery duration can be extended, so that convenience is enhanced. In addition, in this application, when the element of a function similar to each other like the display unit 110 of the display operation terminal 100 and the display unit 210 of the electronic device 200 is described by a name focusing on the difference in function, the elements are distinguished from each other. In order to do this, the expressions “terminal” indicating the display operation terminal 100 and “device” indicating the electronic device 200 may be clearly referred to as the terminal display unit 110 and the device display unit 210, respectively. The same applies to other elements.

2-2. Touch Operation Input Unit The touch operation input unit 120 can be any touch type input device that allows a user to specify a position limited to at least some extent in order to reflect his / her intention and input with a fingertip. Typically, an arbitrary touch input method such as a capacitance method or a resistance film method is employed. The object for specifying the position includes not only the fingertip of the user but also a finger pad, a pen tip subjected to electrical conduction, a pen that operates by electromagnetic induction, and the like, and one or more positions may be specified simultaneously. The touch position detection range of the touch operation input unit 120 typically matches the display area 110a of the display unit 110 or in order to detect an input in a peripheral part outside the outer frame of the display area 110a. The size is slightly larger than the display area 110a. The external appearance of the touch operation input unit 120 and the display unit 110 combined in this way is like an integrated device of a display device and a position input device, which is also called a touch panel. For example, ipad (registered trademark) product group of US Apple Inc. Components similar to those of such tablet terminals are lined up. The touch operation input unit 120 that coincides with the display area 110a is made of a material that is transparent to light so that the presence of the touch operation input unit 120 cannot be visually recognized by the user, or is integrated with the display unit 110. It is preferable to do. A touch operation signal is output from the touch operation input unit 120.

2-3. Communication In the display operation separation system 10, the display operation terminal 100 performs bidirectional communication with the electronic device 200. Typically, the display operation terminal 100 and the electronic device 200 are connected to each other by wireless communication. For this purpose, the electronic device 200 includes a device transmission unit 240 and a device reception unit 230. The terminal receiver 130 communicates with the device transmitter 240, and the terminal transmitter 140 communicates with the device receiver 230. The display content (display content) displayed on the display unit 110 when the user is using the display operation terminal 100 is obtained from the electronic device 200. For this reason, in a situation where the display content from the electronic device 200 is displayed on the display operation terminal 100 and some operation is performed on the display operation terminal 100, the electronic device 200 transmits to the display operation terminal 100 in the bidirectional communication. The terminal reception unit 130 and the device transmission unit 240 are responsible for transferring the display signal 132 indicating the display content, and the terminal transmission unit 140 and the device transmission unit 142 are configured to transfer the operation input signal 142 that conveys the operation input from the display operation terminal 100 to the electronic device 200. The device receiving unit 230 takes charge.

The communication connection protocol (communication protocol) that can be adopted for these communications is, for example, Miracast (or WiDi), LTE Direct, Bluetooth (registered trademark) (hereinafter referred to as “BT”), BLE (Bluetooth (registered trademark) Low Energy) and WiFi Direct (both are registered trademarks) can be employed. Typically, P2P (Peer to Peer) communication that does not use a server in any communication in the present embodiment is performed. The example illustrated here is also for performing P2P communication.

2-4. Two-way communication In the display operation separation system 10 of the present embodiment, communication for the display signal 132 between the terminal receiver 130 and the device transmitter 240 and an operation input signal between the terminal transmitter 140 and the device receiver 230 The protocol may be the same or different in both the communication for 142. In general, the display content of the display content included in the communication for the display signal 132 includes an image and a video, and the amount of data is large. On the other hand, the communication for the operation input signal 142 is input data by the touch operation input unit 120, and the data amount is small. In order to reflect such a difference in data amount, an appropriate protocol may be selected and combined separately for each of the communication for the display signal 132 and the communication for the operation input signal 142. As a result, even if each protocol adopted includes provisions for the mutual authentication (pairing) procedure of the devices, the authentication procedure for pairing in the present embodiment It is not essential to carry out as specified. For example, when a protocol for communication for the display signal 132 from the electronic device 200 to the display operation terminal 100 is adopted, it is assumed that bidirectional communication is defined in the protocol and mutual authentication is possible. Even in such a case, the display operation separation system 10 according to the present embodiment uses a different protocol used for communication for the operation input signal 142 from the display operation terminal 100 to the electronic device 200 instead of mutual authentication. Mutual authentication is also possible.

2-5. Further Improvements The display operation separation system 10 according to the present embodiment can be used for various applications and purposes of use. For this reason, as an option, an additional structure and the characteristic can be provided to the structure mentioned above.

One example is a configuration close to what is called a thin client, which is adopted from the viewpoint of data security. That is, the display operation terminal 100 after the connection with the electronic device 200 is released does not hold information (data) that can reproduce the display content. When the display operation terminal 100 is manufactured by a normal mounting method, the display operation terminal 100 is manufactured as a computing device that operates a microprocessor (not shown) using software (program) and an operating system (OS) for functioning itself. . In such a case, it is highly likely that the display operation terminal 100 also includes a nonvolatile recording medium in accordance with the mounting form. In consideration of the risk of intermittent wireless communication for the display signal 132 described above, a configuration in which communication contents are temporarily stored in such a nonvolatile recording medium can be considered. However, if the display operation terminal 100 reproduces information that should be concealed at a timing not intended by the user by itself, there is a security risk. For this reason, it is useful to erase the display contents from the above-described nonvolatile storage medium.

In the display operation separation system 10 of the present embodiment, it is preferable that the display operation terminal 100 further includes an additional finger operation input unit such as a button. The finger operation input unit typically receives an operation input by a finger from the user and is disposed at a position outside the display area 110a. It is useful that such a signal by the finger operation input unit is included in a part of the operation input signal sent by the terminal transmission unit 140 and transmitted to the electronic device. In particular, when the electronic device 200 to be combined is one in which some kind of finger operation input unit is mounted, it is advantageous that the finger operation input unit in the display operation terminal 100 also performs the same function. Further, it is possible to display a button element in a part of the display area 110a and to have a similar function. In this case, when the user performs an operation on the button element, a touch operation signal including information indicating that the operation is performed on the button element is generated and transmitted as a part of the operation input signal.

It is also preferable to equip the display operation terminal 100 with an additional input unit other than the finger operation input unit. For example, a sensor for detecting the entire direction and its change can be incorporated in the display operation terminal 100, and the inclination of the entire display operation terminal 100 can be included in a part of the operation input signal. If the whole is tilted and programmed to mean "go forward", the user displays the display content (such as an electronic book) displayed on a page basis and holds the display operation terminal 100 with one hand. The page turning operation can be easily performed while browsing. Furthermore, when the entire display operation terminal 100 is made to be at least partially flexible, it is also preferable that the display operation terminal 100 is equipped with a sensor that detects that the user bends. The bending of the operation terminal 100 by the user can be used for page turning, for example. Preferably, the display operation terminal 100 can be provided with a sensor for detecting tilt or acceleration. It is also useful in terms of further improving usability to detect that the display operation terminal 100 tilts the user and perform page turning.

When realizing the ultimate thin shape in order to maximize the convenience of the display operation terminal 100, there will be a battery problem as in many electronic devices. The display operation terminal 100 employed in the display operation separation system 10 of this embodiment is also typically operated by a battery. In addition, it is useful to cover part or all of the power used by an external power source or an external battery that is wirelessly connected. Wireless power connection has been put into practical use by a non-contact power transmission technology such as Qi (US Wireless Power Consortium). Further, this power connection can be used for charging the internal battery of the display operation terminal 100. The external battery described above is installed in the operation terminal 100 and used for the display operation terminal 100 and the electronic device 200 in close proximity to each other, and in addition to the power of the display operation terminal 100, all of the electric power of the electronic device 200 or It may be something that covers a part. In addition, a battery installed in the electronic device 200 can be used as an external battery for the display operation terminal 100. Furthermore, if the display operation terminal 100 uses wireless power connection, the connectors can be reduced or eliminated from the display operation terminal 100, so that it is easy to design to improve convenience such as drip-proofing and waterproofing. Therefore, there are fewer obstacles in how to use alcohol sterilization when brought into a sterile room.

In the display operation terminal 100 of this embodiment, it is preferable that the terminal display unit 110 exhibits flexibility. The flexible display unit 110 can be realized by, for example, an organic EL display device or a liquid crystal display device manufactured using a plastic film substrate. The display unit 110 having flexibility is generally more resistant to impact than a display unit using a glass substrate, for example. Therefore, the impact protection function of the display unit 110 to be provided to the housing of the display operation terminal 100 can be simplified, and the display operation terminal 100 can be reduced in weight. Furthermore, it is also useful that the casing of the display operation terminal 100 is flexible so that, for example, a mounted circuit board, a battery, and the like can cope with it. If, for example, a flexible substrate or a flexible lithium polymer battery is used in addition to the housing or display unit 110 of the display operation terminal 100 having flexibility, the display operation terminal 100 as a whole has a certain degree of flexibility and elasticity. Can be granted. Since the display operation terminal 100 having flexibility and elasticity can be used very easily, it is possible to provide the user with maximized convenience.

3. Configuration of Electronic Device Next, an electronic device 200 used in combination with the display operation terminal 100 in the display operation separation system 10 will be described. If what can become the electronic device 200 is illustrated, a smart phone, portable PC, and a tablet terminal will be typical. These are given a certain degree of portability by themselves, and are expected to be directly operated by the user, and have a display unit (device display unit). Other than these, what can be the electronic device 200 is a desktop PC, a server computer, or the like. It can be said that the requirement for the electronic device 200 is a computing device, which itself provides a display function to a user, accepts input from the user, and implements a necessary communication function. Hereinafter, although description will be given mainly assuming a smartphone as the electronic device 200, the electronic device 200 of the present embodiment is not limited to this.

When using an existing smartphone for the electronic device 200, the necessary hardware requirements are not special. In addition to the requirements described above, there is a requirement that wireless communication with a necessary communication protocol is possible if wireless communication is performed. The electronic device 200 employed in the display / operation separation system 10 has a surface with fewer restrictions than a smartphone that has been used so far. For example, the electronic device 200 may be extremely small. This is because if the display and input can be left to the display operation terminal 100, it is not necessary to increase the size of the display unit in terms of convenience. For this reason, in the case of adopting an existing smartphone for the electronic device 200, if the display operation terminal 100 is of a sufficient size, the request for increasing the size of the electronic device 200 for the purpose of usability is eased. There is practical significance. However, this embodiment does not particularly limit the size of either the display operation terminal 100 or the electronic device 200.

Note that some of the functions implemented in existing smartphones and tablet terminals are protected by manufacturers in order to limit the implementation of extended functions by others (for example, installed by Apple Inc. in the United States). Product family). Such protection is not an essential obstacle to the implementation of this embodiment. This is because the protection is unrelated to the manufacturer, and the protection can be canceled by changing the policy. For this reason, it is also possible to employ a device in which such protection is mounted for the electronic device 200 of the present embodiment.

4). System operation (overview)
Next, the configuration and operation of the entire display / operation separation system 10 according to the present embodiment will be described with additional operations. As shown in FIG. 1, the display operation separation system 10 is typically used in a one-to-one connection in which the display operation terminal 100 and the electronic device 200 are combined one by one.

4-1.1 One-to-One Connection When used in a one-to-one connection, the display operation terminal 100 is typically used in a state where a combination with the electronic device 200 is assumed to some extent. When wireless connection is performed when the display operation terminal 100 and the electronic device 200 are connected to each other, processing for mutual authentication (pairing processing) is required.

In the display / operation separation system 10, it is preferable to implement an automatic start function that minimizes user intervention in the pairing process. FIG. 2 is a schematic diagram showing a configuration when such a function is implemented in the display operation terminal 100 using this wireless communication. In addition to the above-described configuration, the display operation terminal 100 includes a communication control unit 162, a pairing information recording unit 164, and a radio wave intensity detection unit 166 for the automatic start function. The communication control unit 162 controls establishment and release of connection with the electronic device 200 by the terminal receiving unit 130 and the terminal transmitting unit 140. The pairing information recording unit 164 stores pairing information 164a for registering the electronic device 200 as one of the connection authenticated devices. The pairing information 164a is implemented, for example, in a file format managed on an operating system for causing the display operation terminal 100 to function. The radio wave intensity detection unit 166 is connected to the terminal receiving unit 130 and can determine the intensity of the type of radio waves that can be received, and sends a detection signal 166a that can indicate the received radio wave intensity by the terminal receiving unit 130. The radio wave intensity detection unit 166 detects the radio wave intensity indicating whether or not there is a problem in maintaining the connection after the electronic device 200 and the display operation terminal 100 establish the communication connection. For this detection operation, the display operation terminal 100 may employ a technique of, for example, issuing a signal for searching for the electronic device 200 through the terminal transmission unit 140 and detecting a radio wave emitted by the electronic device 200 as a reception confirmation corresponding thereto. it can.

Typically, the display operation terminal 100 and the electronic device 200 have already been paired in advance, and the electronic device 200 has already been registered as a connection authenticated device in the pairing information 164a of the display operation terminal 100. . Registered in the pairing information 164a is general pairing information for wireless communication. For example, Miracast, BT, NFC, etc. Usually, only in those communication, the mutual authentication information is first set for pairing. It is of any communication protocol as required. In such a case, the communication control unit 162 determines whether the received radio wave intensity is stronger than a predetermined intensity reference value based on the detection signal 166a from the radio wave intensity detection unit 166. When sufficient received radio wave intensity is obtained, the communication control unit 162 controls the terminal receiving unit 130 and the terminal transmitting unit 140 to establish a connection with the electronic device 200. When the electronic device 200 that has completed the pairing process and has been registered at least once by such an operation becomes communicable to the display operation terminal 100, the display operation terminal 100 connects to the electronic device 200 with a few steps. It can be established and can provide high convenience to the user.

More specifically, an example in which communication connection establishment is performed in three stages can be considered. As the first first stage, the display operation terminal 100 detects that the electronic device 200 is very close by NFC. At that time, the display content cannot be received by the terminal receiver 130 yet. For communication by NFC, pairing information for NFC stored in pairing information 164a is used. Since the devices need to be close to each other in order to establish the NFC connection, the detection of the electronic device 200 by the display operation terminal 100 itself becomes the operation of the radio wave intensity detection unit 166. Next, as a second stage, the Miracast connection is established in response to the NFC connection establishment. For this purpose, the pairing information for Miracast stored in the pairing information 164a is used. As a result, the display operation terminal 100 can receive the display signal 132 indicating the display content through the terminal receiving unit 130. Further, as a third stage, communication at the BT is established. Also in this case, the pairing information for BT stored in the pairing information 164a is used. Thereby, the display operation terminal 100 can transmit the operation input signal 142 including the touch operation signal to the electronic device 200 by the terminal transmission unit 140.

The automatic start function shown in FIG. 2 is also executed when the power is turned on, for example, when the display operation terminal 100 is turned off. If the pairing information recording unit 164 is mounted by a nonvolatile memory or the like, the information is retained even when the power is turned off.

In the display operation separation system 10, it is preferable to use a plurality of communication protocols for mirroring. In communication protocols, there are often trade-offs in the relationship between transmission speed, display quality determined by the display band, etc., reachable distance, and power consumption. For this reason, for example, when the communication state is good, it is advantageous to adopt a communication protocol that prioritizes display quality, and when it becomes worse, it is advantageous to adopt a communication protocol that prioritizes reachability over display quality and facilitates communication establishment. In such a case, it is highly practical to give the order of connection attempts to a plurality of communication protocols. In this embodiment, a communication channel that transmits display contents and performs mirroring is particularly problematic. Therefore, it is preferable to mainly receive data when viewed from the display operation terminal 100. FIG. 3 is a schematic diagram showing a configuration in the case where a plurality of communication protocols are mounted on the display operation terminal 100 and mirroring is performed in levels according to trial orders designated in advance. In addition to the above-described configuration, the display operation terminal 100 having such a preferable configuration includes a reception control unit 172, a pairing information recording unit 174 for each protocol, a trial sequence setting unit for a mirroring function divided into levels by a plurality of protocols. 176.

The reception control unit 172 operates so as to control establishment and release of a reception connection with the electronic device 200 by the terminal reception unit 130. The protocol-specific pairing information recording unit 174 stores protocol-specific pairing information 174a for registering the electronic device 200 as one of the connection-authenticated devices in association with each of a plurality of communication protocols. The trial sequence setting unit 176 stores trial sequence information 176a for setting trial ranks and trial times for trying to establish a connection of each communication protocol for a plurality of communication protocols for receiving the display signal. Here, the terminal receiving unit 130 can receive a display signal from the electronic device 200 using a plurality of communication protocols. Furthermore, the reception control unit 172 executes trial processing for establishing a reception connection from a device registered as connection-authenticated in the protocol-specific pairing information 174a while switching the communication protocol. At that time, the order of a plurality of communication protocols and the waiting time of each communication protocol corresponding to the trial rank and trial time of the trial sequence setting unit 176 are used. In a typical case, the trial order is arranged in the order of the display quality prioritized first and the communication establishment easy thereafter.

Further, when the display operation terminal 100 and the electronic device 200 are capable of wired communication in addition to wireless communication, the wired communication is set to a higher priority than the wireless communication in the same level of mirroring. Is advantageous. In that case, either or both of the terminal receiver 130 and the terminal transmitter 140 further communicate with the electronic device 200 by wired communication. The pairing information by protocol 174a registers an electronic device as one of connection-authenticated devices for wired communication. The trial sequence information 176a stored in the trial sequence setting unit 176 sets the trial order higher than the wireless communication for the wired communication. In wired communication, unlike wireless communication, it is not always necessary to set a trial time, so the trial time does not necessarily have to be set.

In addition, in the display / operation separation system 10, it is also preferable to further utilize wired communication for wireless communication. The procedure for pairing in the wireless communication protocol is a relatively troublesome process for the user. In the wired connection, the user can indicate the intention of connection between the display operation terminal 100 and the electronic device 200 with the connection action. It can be said that registering the display operation terminal 100 and the electronic device 200 as a combination that can be paired even by wireless communication with the connection of the wired communication saves the user's trouble and brings about high convenience.

FIG. 4 is a schematic diagram showing the configuration of the display operation separating system 10 that performs the pairing operation by wired communication. White double arrows in the figure indicate a state where connection by wired communication is established. That is, in the display operation separation system 10 that performs the pairing operation by wired communication, the terminal reception unit 130 and the device transmission unit 240 communicate with each other by wired communication, or the terminal transmission unit 140 and the device reception unit 230 have It is assumed that they communicate with each other by wired communication. The display operation terminal 100 includes a terminal communication control unit 162 that controls establishment and release of the connection with the electronic device 200 by either the terminal reception unit 130 or the terminal transmission unit 140. In addition, the terminal pairing information recording unit 164 stores pairing information 164a for registering the electronic device 200 as one of the connection authenticated devices. Then, the terminal communication control unit 162 connects the electronic device 200 that is not recorded as a connection-authenticated device in the terminal pairing information recording unit 164 to the display operation terminal 100 through the terminal transmission unit 140 or the terminal reception unit 130 by wired communication. If it is, the terminal transmission unit 140 or the terminal reception unit 130 is controlled to execute a procedure for mutually authenticating the display operation terminal 100 and the electronic device 200. Then, after the connection with the electronic device 200 is established, the electronic device 200 is registered in the terminal pairing information 164a as a connection authenticated device.

By such an operation, the display operation terminal 100 recognizes the electronic device 200 as a paired device after being wired to the electronic device 200 once, and performs the procedure omitted with the electronic device 200 in the procedure omitted during wireless communication. Connection can be established.

The functions and processing for wired communication described above can be implemented on the electronic device 200 side in addition to the display operation terminal 100 or in place of the display operation terminal 100. In this case, as shown in FIG. 4, the electronic device 200 includes a device communication control unit 262 and a device pairing information recording unit 264. The device communication control unit 262 controls the establishment and release of the connection with the display operation terminal 100 by both the device reception unit 230 and the device transmission unit 240. The device pairing information recording unit 264 stores pairing information 264a for registering the display operation terminal 100 as one of connection-authenticated devices. When the display operation terminal 100 that is not recorded as a connection-authenticated device in the device pairing information recording unit 264 is connected to the electronic device via the device transmission unit 240 or the device reception unit 230 by wired communication, the device communication control unit 262 The device control unit 240 controls the device transmission unit 240 or the device reception unit 230 to execute a procedure for mutually authenticating the electronic device 200 and the display operation terminal 100, and the display operation terminal 100 is connected after the connection with the display operation terminal 100 is established. The device pairing information 264a is registered as an authenticated device.

The operation by software in these operations is as follows. A program (initialization application program) for performing initialization settings with the display operation terminal 100 is installed in the electronic device 200 in advance. Similarly, an application program for initialization with the electronic device 200 is also installed in the display operation terminal 100. And the electronic device 200 and the display operation terminal 100 are connected by wire, such as USB (Universal Serial Bus), for example. As a result of this connection establishment, the initialization application program is started on each of the display operation terminal 100 and the electronic device 200. Both initialization application programs communicate with each other to complete the initialization setting for pairing.

The most typical scene in which the display operation separation system 10 is used is a situation in which the electronic device 200 is a smartphone owned by a user, and the user uses the display operation terminal 100 himself. The user uses the display operation terminal 100 in expectation of high convenience obtained by enlarging the display unit of the smartphone. If the electronic device 200 is assumed to be used by a specific user, there is a high possibility that the display operation terminal 100 in such a usage pattern belongs to that user. In such a case, the display operation terminal 100 and the electronic device 200 are highly likely to be used by a single user at the same location, and as a result, the display operation terminal 100 and the electronic device 200 are wirelessly connected. In addition to the above, you may want to connect directly without using cables. FIG. 5 is a schematic diagram illustrating the structure of the display operation terminal 100 and the electronic device 200 that perform such direct connection. A typical shape of the display operation terminal 100 has a shape that forms a flat plate when the display operation terminal 100 is used, and the terminal display unit 110 of the display operation terminal 100 extends along one surface 100a of both surfaces forming the thickness of the flat plate. Has been placed. In such a case, if the terminal connection terminal group 150 is provided on the other surface 100b of the both sides of the display operation terminal 100, direct connection (wired connection) using the terminal connection terminal group 150 becomes possible. . In the electronic device 200, a device connection terminal group 250 that can be connected to the terminal connection terminal group 150 is provided on one of the connection surfaces 200c. The electronic device 200 is attached to the other surface 100b of the display operation terminal 100 in a state where the terminal connection terminal group 150 and the device connection terminal group 250 are connected to each other. When directly connected in this way, the advantage of wired connection, that is, congestion that is unique to wireless connection does not become a problem even when a large number of users simultaneously use the display operation terminals 100 in a narrow space, and a high transfer rate. Also realized. In addition, it is possible to perform the process (described above) of executing initialization setting for pairing for wireless connection using wired connection by direct connection between the terminal connection terminal group 150 and the device connection terminal group 250. .

4-2. Screen In the display operation separation system 10 of the present embodiment, it is preferable to devise a screen display during the mirroring operation. This is because various problems and waste that may be caused by mirroring the display operation terminal 100 can be reduced. Such problems and wastes typically occur even though the display content of the electronic device 200 is originally scheduled to be displayed on the device display unit 210 of the electronic device 200 itself. This is because it is displayed on the display operation terminal 100 as it is. One of the measures for dealing with this is mainly from the viewpoint of power consumption, and the other is for using the display operation terminal 100 when the electronic device 200 has a voice call function. Is.

FIG. 6 is a schematic diagram illustrating a state in which the operation of the device display unit 210 of the electronic device 200 is controlled by the electronic device 200 during mirroring. The electronic device 200 includes a device display unit 210, and the device display unit 210 can present display contents to the user. The electronic device 200 includes a remote input reception unit 282 and an operation control unit 284 in addition to the configuration described above. The remote input reception unit 282 receives an operation input signal transmitted from the terminal transmission unit 140 of the display operation terminal 100 and received by the device reception unit 230. The operation control unit 284 controls the transition of the device display unit 210 to the power saving operation or the stop operation, or the return from any operation based on the signal from the remote input reception unit 282. The display operation terminal 100 superimposes a setting prompt 112 that prompts a determination on whether the display operation of the device display unit 210 of the electronic device 200 should be a power saving operation or a stop operation on the display content as an additional display content on the terminal display unit 110. Display. Then, an operation input signal including an input of a setting result indicating that a power saving operation or a stop operation received by the touch operation input unit 120 from the user should be transmitted to the setting prompt 112 through the terminal transmission unit 140 is displayed. The operation terminal 100 operates. When the remote input reception unit 282 receives an operation input signal through the device reception unit 230, the operation control unit 284 of the electronic device 200 causes the device display unit 210 to perform a power saving operation or a stop operation based on the operation input signal including the setting result. Operates to make transition.

FIG. 7 is a schematic diagram showing a configuration for utilizing the display operation terminal 100 during a voice call when the electronic device 200 is a smartphone or the like and provides a voice call function. The electronic device 200 includes a device display unit 210, a voice call processing unit 202, a function management control unit 204, and a remote input reception unit 282 in addition to the configuration described above. The display content of the device display unit 210 can be presented to the user of the electronic device 200. The function management control unit 204 is operated during a busy period in which a user provides a voice call function for making a voice call. This function management control unit 204 is used to selectively start, pause, or stop the function processing unit of the electronic device itself including the voice call processing unit. The function management control unit 204 also executes control of display contents of the device display unit 210. In order to use the electronic device 200 for a voice call of a single smartphone, a dialer screen (for a voice call) is usually displayed on the device display unit 210 before or while the user places the electronic device 200 directly on the mouth and ear. Screen) and perform operations for voice communication. For example, a caller selection screen for call operation is displayed, and transition from call button to call operation is accepted. When called from the other party, information on the other party in the address book data is presented or simply notified. Display the number. For example, when the proximity sensor (not shown) detects that the user is approaching the user's face, the screen is darkened. When a user makes a call using an earphone set with a microphone, the dialer screen is continuously displayed as it is. In any case, when the electronic device 200 is used alone, a display suitable for only a voice call is usually made. In contrast, the display operation separation system 10 shown in FIG. 7 includes a remote input reception unit 282 that receives an operation input signal transmitted from the terminal transmission unit 140 to the electronic device 200 and received by the device reception unit 230. In this case, in a state where the voice call processing unit 202 is in operation, the display content of the device display unit 210 is typically just before displaying the voice call screen presented when the voice call processing unit 202 is activated. It is controlled by the function management control unit 204 so that the screen is returned to a screen that does not include the voice call screen or the home screen that is the default screen for operation. Since this display content is also mirrored on the display operation terminal 100, the display content is presented on the terminal display unit 110. The operation input signal output from the touch operation input unit 120 when the user of the display operation terminal 100 inputs to the touch operation input unit 120 is transmitted to the remote input reception unit 282 via the terminal transmission unit 140 and the device reception unit 230. Accepted. Since this operation is a screen that does not include the voice call screen before the start of the call or is a user input to the home screen, the user does not need to see the voice call screen, and the size of the display operation terminal 100 is reduced. You can enjoy the convenience of the voice call during a voice call. Since the user of the electronic device 200 in this application is typically the same person as the user of the display operation terminal 100, the person can use the content of the call stored in the electronic device 200 during the call, for example. Material data of related information and unrelated information can be viewed on the large display unit 110 of the display operation terminal 100.

4-3. Input In the display operation separation system 10 according to the present embodiment, even if the electronic device 200 is small, the user can enjoy the convenience of taking advantage of the size of the display operation terminal 100. One example thereof is a touch keyboard that takes advantage of the sizes of the display unit 110 and the touch operation input unit 120 of the display operation terminal 100. In order to perform a keyboard operation such as inputting an accurate numerical value, a touch-type keyboard that displays a keyboard key arrangement by software on a display screen and makes an input by a touch operation as a key input is useful.

FIG. 8 is a schematic diagram showing a configuration when a touch-type keyboard is displayed on the display operation terminal 100 in the display operation separation system 10 of the present embodiment. In addition to the above-described configuration, the electronic device 200 includes a device display unit 210, a cursor control unit 212, an input control unit 222, and a remote input reception unit 282. The device display unit 210 presents display contents to the user. The cursor control unit 212 displays an input cursor indicating the input position in the input enabled column 210b included in the display content of the device display unit 210. The input control unit 222 controls input display contents at the input position in the device display unit 210. The remote input reception unit 282 receives an operation input signal transmitted from the terminal transmission unit 140 of the display operation terminal 100 and received by the device reception unit 230. The display operation terminal 100 includes a touch keyboard control unit 122. The touch keyboard control unit 122 generates an input key signal including a key code from the touch operation signal from the touch operation input unit 120. Furthermore, the touch keyboard control unit 122 has a touch operation signal indicating a position of a key on which a touch input operation corresponding to the key input is performed among keys included in the keyboard image presented to the user in the terminal display unit 110. In addition, an input key signal indicating at least the key code of the key is output. The terminal transmission unit 140 transmits the input key signal from the touch keyboard control unit 122 or the input key data corresponding to the input key signal included in the operation input signal. The device receiving unit 230 receives an operation input signal from the terminal transmission unit 140, and the remote input reception unit 282 receives an operation input signal including an input key signal or corresponding input key data. When the key indicated by the key code is pressed based on the input key signal from the remote input accepting unit 282 or the key code indicated by the input key data corresponding to the input key signal, the cursor control unit 212 and the input control unit 222 The display change that should occur is reflected in the display content in the input enable field 210b corresponding to the input cursor. This change in display typically includes display of a key input by characters, deletion of operation display contents by backspace key, enter key, and ESC key, confirmation of input, affirmative input, cancellation, and as a result. Includes screen transitions. In addition to the input of alphabets and symbols such as the so-called QWERTY key, the operation for generating the key code here is a numerical input with only the numerical keys displayed, such as a so-called portable key input or flick input. This includes detecting the number of touches corresponding to key presses and finger actions after key touches.

In particular, it is preferable that the display operation terminal 100 is further devised when the input enable field 210b in the electronic device 200 does not assume input by a touch keyboard. Specifically, if the display operation terminal 100 performs a process of enlarging the display using the cursor as a clue, the convenience can be improved. The display operation terminal 100 having such a configuration includes a display control unit 114 and a cursor search unit 116 in addition to the configuration described above. The display control unit 114 controls the display position and the enlargement ratio when the display content is presented to the user on the terminal display unit 110. The cursor search unit 116 searches for an input cursor included in the display content and outputs a cursor position signal indicating the position of the input cursor. The display control unit 114 controls the relative display position so that the position in the display content of the input cursor indicated by the cursor position signal from the cursor search unit 116 does not overlap the keyboard image 110b. For example, the entire screen is shifted so that the input cursor is positioned 110c just above the keyboard image 110b on the display area 110a. This shift can be easily performed by shifting the read address of a frame memory (not shown) that temporarily stores image data. Further, as necessary, the display control unit 114 enlarges and displays the display content in the vicinity including the input cursor. By using such a function for assisting input by the touch-type keyboard, even if the display content on the electronic device 200 is premised on the display by the device display unit 210, the size of the display operation terminal 100 is utilized. It is possible to input and is convenient.

4-4. Content The display operation separation system 10 of the present embodiment can further enhance the convenience of the function of transmitting display content from the electronic device 200 to the display operation terminal 100. In the description so far, the configuration in which the display content of the electronic device 200 is basically mirrored on the display operation terminal 100 as it is has been described. However, in an actual usage situation, there may be a problem in the power supply duration of the electronic device 200, or a situation in which the wireless communication itself is affected or restricted. In such a case, the display device for browsing or viewing the electronic device 200 before the timing of use of the display operation terminal 100, in units of content (referred to as unit content) of a size that the user can use together. It is preferable to transfer only that unit. This unit content is typically one computer file. FIG. 9 is a schematic diagram showing the configuration of the display / operation separation system 10 used for unit content. For such a device, in the display operation separation system 10, it is preferable that the electronic device 200 further includes a device content recording unit 292, and the display operation terminal 100 further includes a terminal content recording unit 192. Both the device content recording unit 292 and the terminal content recording unit 192 can store the entire unit content serving as a unit of browsing or viewing. The electronic device 200 transmits the entire unit content stored in the device content recording unit 292 to the display operation terminal 100 that has established the connection through the device transmission unit 240. The display operation terminal 100 receives the entire transmitted unit content through the terminal receiving unit 130 and stores it in the terminal content recording unit 192. The wireless communication is stopped by releasing the connection from one or both of the electronic device 200 and the display operation terminal 100 after the transmission / reception of the entire unit content is completed. As a result, the display operation terminal 100 can display the content related to the display of the unit content stored in the terminal content recording unit 192 on the terminal display unit 110 after the connection with the electronic device 200 is released. Note that the cancellation of the wireless connection described above means that the connection is at least temporarily canceled, and there is a possibility that the connection is established again thereafter.

The display operation separation system 10 of the present embodiment can be further devised when the content to be handled is mainly a still image. Examples of content mainly composed of still images include, for example, a browsing document file having a plurality of pages, an electronic book file, a web page, a Twitter (registered trademark) screen, a text message application program screen, a mail message screen, Can be mentioned. Also in this case, if the image content can be comfortably browsed while dealing with the problem of the power supply duration of the electronic device 200 and the failure or limitation of the wireless communication itself, the convenience is enhanced. FIG. 9 also shows the configuration of the display operation separation system 10 used for image content. In such a configuration, the electronic device includes the device image content recording unit 294, and the display operation terminal includes the terminal image content recording unit 194. The device image content recording unit 294 and the terminal image content recording unit 194 can store at least one image of image content including a plurality of images.

The electronic device 200 transmits one or more images among the plurality of images of the image content stored in the device image content recording unit 294 to the display operation terminal 100 that has established the connection through the device transmission unit 240. The display operation terminal 100 receives one or more images transmitted until the connection is released from among the plurality of images of the transmitted image content, and stores them in the terminal image content recording unit 194. The electronic device 200 and the display operation terminal 100 cancel the mutual connection from one or both sides after the transmission / reception of one or more images is completed. The display operation terminal 100 can display the latest image stored in the terminal content recording unit 192 on the terminal display unit 110 after the connection with the electronic device 200 is released.

4-5. Various Connection Modes The display operation separation system 10 according to this embodiment uses the above-described number of electronic devices and display operations when the electronic device 200 owned by an individual user is used through the display operation terminal 100 owned by the same individual user. Typically, the number of terminals is one. However, the present embodiment is not necessarily limited to the combination of each one electronic device and display operation terminal, and high convenience can be provided by various connection forms. One of them is one electronic device and two or more display operation terminals, and the other is two or more electronic devices and one display operation terminal. It is intended to be a stand. Note that the description related to the display operation separation system 10 throughout the present application is part of the description that simultaneously describes the display operation separation systems 12 and 14 (FIGS. 10 and 11) described below unless otherwise specified. .

FIG. 10 is a schematic diagram illustrating a typical example of the operation when the number of electronic devices is one and the number of display operation terminals is two or more in the present embodiment. As shown in FIG. 10, this operation is explained by a display / operation separation system 12 having one electronic device and two display / operation terminals. Both of these two display operation terminals (first display operation terminal 100A and second display operation terminal 100B) can have the same features as the display operation terminal 100 described so far. The device transmission unit 240 of the electronic device 200 generates a common display signal indicating display contents common to both the first display operation terminal 100A and the second display operation terminal 100B, a display signal for the first display operation terminal 100A, and a second display signal. It is included in each of the display signals for the display operation terminal 100B and transmitted. The device receiving unit 230 of the electronic device 200 receives operation input signals from the first display operation terminal 100A and the second display operation terminal 100B in a distinguished manner from each other. By adopting such a configuration, the display contents for the electronic device 200 can be confirmed on both the first display operation terminal 100A and the second display operation terminal 100B, and an input operation to the electronic device 200 is performed from both sides. be able to.

At this time, it is also useful to set which of the first display operation terminal 100A and the second display operation terminal 100B is the input to the electronic device 200. That is, the electronic device 200 includes a device input receiving unit 286, a remote input receiving unit 282, and a device display control unit 288 in addition to the above-described configuration. The device input receiving unit 286 receives an operation input from the user. The remote input receiving unit 282 receives an operation input signal received from the first and second display operation terminals 100A and 100B via the device receiving unit 230. Then, the device display control unit 288 controls additional display contents to be presented on the device display unit based on signals from the device input receiving unit 286 and the remote input receiving unit 282. The device receiving unit 230 can perform the receiving operation by distinguishing between the first and second display operation terminals 100A and 100B. The device display control unit 288 superimposes a selection prompt 288a on the display content as additional display content for selecting which of the first display operation terminal 100A and the second display operation terminal 100B should accept the operation input signal. The device input receiving unit 286 is configured to receive an input of a selection result from a user of the electronic device. The device input receiving unit 286 receives a signal by touch input from the touch operation input unit 220, for example. Thereafter, the remote input receiving unit 282 operates to selectively receive an operation input signal from either the first display operation terminal 100A or the second display operation terminal 100B according to the selection result.

Next, a description will be given of another connection configuration in which the number of electronic devices is two or more and the number of display operation terminals is one. FIG. 11 is a schematic diagram showing a configuration of the display operation separation system 14 in which the number of electronic devices is two or more and the number of display operation terminals is one. Both the first electronic device 200A and the second electronic device 200B have the characteristics of the electronic device 200 described above. The terminal receiving unit 130 of the display operation terminal 100 receives the first display content that is the display content of the first electronic device 200A and the second display content that is the display content of the second electronic device 200B in a distinguished manner from each other. The terminal display unit 110 displays the first display content and the second display content. The touch operation input unit 120 can determine from the touch position whether the input is the first display content or the second display content, and generates a touch operation signal corresponding to the input. The terminal transmission unit 140 displays the operation input signal including the touch operation signal on the terminal display unit and the first operation input signal corresponding to the first display content displayed on the terminal display unit 110. The second operation input signal corresponding to the second display content is transmitted to each of the first electronic device 200A and the second electronic device 200B so as to be distinguishable from each other.

5). Application 5-1. Expandability The display operation separation systems 10, 12, and 14 described in the present embodiment are extremely versatile and can be easily linked with existing computing devices. As a result, the display operation separation system 10 can be applied to a wide variety of uses. In order to assume the application, the configuration proposed in the display operation separation system 10 can be improved and evolved in the future. Specifically, the present inventors consider that the display / operation separation system 10 will continue to evolve in almost three stages. The first is the advancement of the hardware itself of the display operation terminal 100, and the second is the advancement of hardware materials utilizing the concept of the display operation terminal 100. The third is the advancement of software that realizes the concept of the display / operation separation system 10, 12, 14.

The advancement of the first hardware itself is in line with the direction of development of generally oriented hardware such as thin and light weight, large screen, and power saving. These are generally oriented even in conventional computing devices within realistic constraints. However, in the display / operation separation system 10 of the present embodiment, when the power consumption of the display / operation terminal 100 is viewed, it is negligible except for the display and touch input functions and the functions for communicating them. Therefore, the configuration including the battery for making it thin or light is much easier to deal with than in the past. That is, it is a great advantage of the display operation separation system 10 that the objects such as reduction in thickness and weight are easily achieved even at the same technical level.

Secondly, while utilizing the concept of the display operation terminal 100, for example, it is possible to incorporate the evolution of materials including soft materials and waterproof functions. In the display operation terminal 100 configured by display, touch input, and their communication and limited elements, there are few technical items that are restricted or required in the evolution of materials. As a result, the display operation terminal 100 that can quickly receive the benefits of the evolution of the material can easily enhance the attractiveness of the product at the time of concrete implementation.

The evolution of the third software adds an unprecedented application to the scene where the display operation separation system 10 is used. For example, although the display operation terminal 100 has been described as having a single display unit 110, it is also useful to use a plurality of screens such as two screens. Furthermore, in the entertainment field such as games provided for smartphones, the user experience has a decisive difference due to the difference in screen size.

5-2. Application example: Learning tools / for elderly people / for hospitals By combining the first to third evolutions described above, it is possible to pioneer new applications that have not been targeted in the past and that meet the needs of users. For example, a learning tool for children could be developed if it is included at home as a terminal that parents and children operate together or even at school. For example, in the case of a small display device, it has been necessary to reduce the display content in the case where a character is displayed in a large size, whereas in the case of using this embodiment, an intuitive operation can be performed while being possible. A large screen can be adopted, and the reduction of display contents can be minimized. This feature will be useful not only for children but also for the elderly. In addition, people who may use the IT device or feel a barrier to a heavy tablet or PC can easily access the information content by using the device using this embodiment. In addition, if the surface of the housing or the display unit 110 is made of a material that can be sterilized with alcohol, a patient who is isolated for medical reasons in an environment (such as a sterile room) where portable PCs cannot be used also uses information content. Will be able to.

5-3. Application example: visual guide Furthermore, in a system in which at least one of the display operation terminal 100 and the electronic device 200 is plural, such as the display operation separation systems 12 and 14 (FIGS. 10 and 11), a new one that takes advantage of the advantages Various applications are also in the horizon. One of them is visual guides at art museums. At present, voice guides are used in many museums. This is to provide individual viewers with explanation audio of each exhibit through earphones. If commentary content for similar purposes can be provided not only as a still image but also as a visual image / video including moving images, it is possible to provide more impressive information to supplement knowledge about the exhibits in front of you. it can. Examples of information unique to display using images and videos include a microscopic image of a fine structure that can be enlarged and visually recognized, and an image that shows an internal structure that cannot be confirmed in the display state. Furthermore, it can provide a wide variety of experiences such as conducting interactive questionnaires and running learning programs. In that case, a viewer who goes around in front of a large number of exhibits exhibits content at their own timing according to their own interests. For this purpose, the electronic device 200 preferably performs an operation such as streaming distribution that distributes display contents to the individual first display operation terminal 100A, the second display operation terminal 100B, and the like at an appropriate timing. In that case, it will be possible to use a larger number of display operation terminals by applying the technique shown in FIG. 9 such as temporarily storing the unit content in the first display operation terminal 100A.

5-4. Application Example: Conference Terminal Another application in which the present embodiment is effectively implemented is a conference terminal. Conventionally, materials are distributed by printed matter for meetings in various scenes of business. However, at present, most materials are created electronically, and it is wasteful to distribute them in print for a one-time meeting. On the other hand, electronic materials created on the other hand are not distributed as they are by electronic media because secret management is difficult and the risk of information leakage often becomes a problem. If the display operation separation system 12 having one electronic device and two or more display operation terminals according to the present embodiment described with reference to FIG. 10 is employed, the advantage of using electronic materials as they are, It is possible to achieve both easy secret management.

In this case, for example, the electronic device 200 is a relatively high-functional computer device, and the document file of the electronic device 200 can be browsed simultaneously by two or more display operation terminals 100A and 100B. Each user can browse different pages of one file at the display operation terminals 100A and 100B at the same time, as long as the electronic device 200 employs a server operating system that assumes multi-users. Further, similarly, even when the electronic device 200 and two or more display operation terminals 100A and 100B are employed, the display of the display operation terminals 100A and 100B can be made completely the same. In such a case, it is possible to share information by inputting the target file being browsed between the display operation terminals 100A and 100B, and each user is seated in a use imitating a white board that can be written on each other. Can be used.

Furthermore, particularly preferably from the viewpoint of information management, the risk of information leakage can be minimized by the configuration of a thin client that does not hold display content data in a reproducible form. In addition, the conference terminal could be used to assist, for example, in taking a vote.

5-5. Application example: table-type mirroring terminal The versatility of this embodiment is extremely high, and the use of the display operation separation systems 10, 12, and 14 is not limited to the above-described handheld devices. For example, due to the recent generalization of large-sized and high-definition display devices, a sufficiently high-quality display can be realized even if the display of a plurality of electronic devices is simultaneously displayed on one display device. For example, it is realistic to display the respective smartphones brought by a plurality of users on one display device. For example, for a user who is seated facing each side in a table type and shares a table top, a table-integrated terminal in which a display device is arranged on a part or all of the table top ( Also called “table type mirroring terminal”. In that case, the screen of his or another person can be moved to the position where he / she sits, or the screen can be freely flipped 180 degrees in order to show the contents of each other. In this way, if the display position, orientation, order, and combination of images can be set by the user's operation on the table-type mirroring terminal, the user can not only share the display but also select the screen of his / her smartphone. You can refer to it from the seating position or show it to others, making discussions easier. Now, after participating in a party or event with multiple people, reviewing and publicizing experiences such as viewing videos and photos taken by each person on the video screen, sending videos and photo files through smartphone operation, It is possible at the gathered place. In addition, when the user presents image data, text data, and the like stored in his / her smartphone to other users, his / her experience can be performed during normal smartphone operation. By running the same game application on each smartphone, it becomes possible for multiple users to play a game together. The thin client configuration is also useful for the use of a table type mirroring terminal. For example, if there is no function for storing data, no data will remain after the use is completed and the user's smartphone is disconnected. As a result, it is easy to ensure security through the table-type mirroring terminal. As a result, the table-type mirroring terminal can be installed in a public place and temporarily used by non-owners.

In such a table-type mirroring terminal, for example, the same number of mirror image data transmission paths as the number of people who use the table at the same time are mounted on the display operation terminal of this embodiment. This image data transmission path is paired one-to-one with each smartphone. As a typical example, when using WiFi Direct and BT, a plurality of WiFi and BT transceivers are mounted on the table. In addition, when performing transmission by wired connection, a connection port of a USB terminal is mounted. A screen is mounted on the upper surface of the table top of the table-type mirroring terminal, and a plurality of people can use it in a posture that looks down on the same screen at the same time. An arbitrary number of mirroring screens are displayed in a designated area on the table-type mirroring terminal. The mirroring transmission line can be realized by using Miracast (or WiFi Direct) wirelessly. This connection may be wired. For example, it can be realized by a USB cable connection. In addition, the touch input operations of the table-type mirroring terminal are distinguished from each other and transmitted to the respective smartphones by BT connection, cable connection, or the like between each smartphone and the table-type mirroring terminal. In addition, when the table-type mirroring terminal is equipped with a speaker, it is also useful to synthesize and output all the sound from the connected smartphone from the connected smartphone.

In the communication control unit according to the present invention, it is possible to control to perform bidirectional communication only with a specific communication carrier or a specific electronic device, for example, control of a display operation terminal and display of an image from the electronic device. Furthermore, it is also possible to set the communication carrier or electronic device other than the specific one so that only one direction, for example, an image from the electronic device can be displayed. *

6). Variations The present embodiment can be modified in various ways.
The electronic device 200 or the display operation terminal 100 according to the present embodiment can be implemented even if configured to be configured by combining functional blocks. A functional block is a functional block that allows users to function independently by preparing parts that should be constituent elements of an electronic device or display / operation terminal in advance so that they have some degree of freedom in combination with each other. This refers to a method that allows users to configure a single terminal by selecting them. The user can realize a necessary function by selecting a function block and connecting the function blocks to each other. Therefore, the function of the electronic device or the display operation terminal can be realized at a cost burden of only the minimum necessary function.

If a specific functional block other than the minimum component block used in the electronic device or display / operation terminal is broken or needs to be replaced in order to improve its performance, it can be replaced by replacing only that functional block. There is no need to replace the electronic device or the entire display operation terminal. Each functional block for the electronic device or the display / operation terminal is mounted with a CPU for controlling and controls functional components. The functional blocks are mounted with a common serial transmission terminal, and any functional block can be connected as a mechanical, electrical, or information transmission path. Each functional block has a function of notifying which functional block it is through a serial transmission terminal, and at the same time has a function of knowing what functional block is connected to itself. The CPU of each functional block recognizes what functional block is connected to the serial transmission terminal, determines an overall function to be realized by the combination, and software corresponding to the function operates.

Main functional blocks for an electronic device or a display operation terminal, that is, a display unit, a battery unit, a WiFi and BT wireless unit, a camera unit, and a television receiving unit are realized as follows.

The display unit is mainly composed of a panel, panel driver, and one or more serial signal terminals. Video information is received from a serial terminal such as a USB, the information is decoded by a panel driver, and control processing for displaying the image information on the panel is performed. Power is supplied from the outside through the serial terminal. As a result, an externally input video signal is displayed on the panel.

The battery unit is composed of one or more serial signal terminals mainly composed of battery cells and a power management IC. When a power supply device or a charging power source is connected to a serial terminal such as a USB, the battery cell is charged with the power supplied from the outside by sensing it. In addition, when a device or a functional block that requires power supply is connected to the serial terminal, the power charged in the battery cell is supplied. The power management IC performs these controls.

The application CPU section is mainly composed of a CPU for operating an application program, a storage, one or more serial signal terminals, a motion sensor, a home button, and a power button. The operation of the application CPU unit is as follows. First, what is connected to a serial terminal such as a USB is recognized, and appropriate input and output operations are determined by the CPU in accordance with the connected functional block. When the display unit is connected, the input video information is output. When a wireless unit, TV receiver, or mobile phone or personal computer is connected, data communication is performed, and when a video signal is received, the video signal is detected and output to the connected display unit To do. In the CPU unit, application software is stored in a nonvolatile storage area, and the application is executed by a user operation. Power is supplied from the outside through the serial terminal.

The WiFi and BT radio units are mainly composed of WiFi signal processing, BT signal processing and RF radio units, and one or more serial terminals. When power is supplied from the serial terminal, the WiFi and BT wireless units start to operate, perform data communication, and output a video signal to the connected display unit.

The cellular radio unit is mainly composed of a cellular signal processing unit, an RF radio unit, an antenna unit, and one or more serial terminals. When the application CPU unit is connected, data to be transmitted / received is exchanged with the application CPU unit, and control itself is operated by a command transmitted from the application CPU unit.

The camera unit is mainly composed of a camera body, buttons, and an image processor, and takes an image by pressing a button or an external command. When the application CPU unit is connected, image data is output to the application CPU unit. When only the display unit is connected without the application CPU unit being connected, image data is output to the display unit.

The TV receiving unit mainly includes a TV receiving tuner, a TV antenna, a signal processor, and one or more serial terminals. When power is supplied from the outside, the operation starts, and the TV radio wave received by the antenna is demodulated to receive a video signal. The received video signal is output to the application signal processing unit when the application CPU unit is connected, and is output to the display unit when the display unit is connected without being connected.

There are the following styles for operating these function blocks in combination.
(1) Display unit alone When the display unit itself is operated, a display screen is mirrored and displayed on the display unit by connecting to a personal computer, a mobile phone, and a power source with a serial communication cable such as USB.
(2) Single battery unit The operation of the single battery unit is nothing but to use it as a power source such as charging an electronic device by connecting with a serial communication cable such as USB.
(3) Display unit + WiFi, BT wireless unit + battery unit:
One example of operating these is to mirror the display screen of the electronic device main body by P2P wireless communication such as Miracast paired from the electronic device. From the electronic device, the display screen of the electronic device can be mirrored and displayed by P2P wireless communication such as paired Miracast.
(4) Display unit + application CPU unit + WiFi, BT wireless unit + battery unit To operate this combination, P2P wireless unit communication such as Miracast paired from the electronic device is performed, and the display screen of the electronic device is mirrored and displayed. can do. In addition, the same function as that of a conventional tablet can be realized by operating application software in the application CPU unit and performing a network connection in the WiFi wireless unit. At the time of use, a USB connection for supplying power is not always necessary, so that it can be carried and used outdoors.
(5) Display unit + application CPU unit + cellular radio unit + battery unit This combination can be used in the same manner as a conventional electronic device (such as a mobile phone).
(6) Display unit + camera unit + battery unit This combination enables the same use as a conventional camera photographing machine.
(7) Display unit + TV receiver This combination realizes the same function as a conventional TV receiver.

When a specific function block is connected and a certain combination (terminal type A) is realized, a sequence for transitioning to another combination (terminal type B), such as connecting a new function block, is as follows. The procedure is as follows.
Step 1: Each functional block detects that a new functional block has been added while the terminal type A is realized. This can be realized by periodically inquiring about the connected functional block via the serial terminal. It is determined that each functional block is independently changed to the terminal type B.
Step 2: When the application CPU unit is connected, this functional block detects what all the connected functional blocks are, and determines that the terminal type B is changed.
Step 3: Software for realizing the terminal type B is activated in each functional block.

The embodiment of the present invention has been specifically described above. Each of the above-described embodiments and configuration examples are described for explaining the invention, and the scope of the invention of the present application should be determined based on the description of the claims. In addition, modifications that exist within the scope of the present invention including other combinations of the embodiments are also included in the scope of the claims.

The display operation terminal of the present invention and the display operation separation system using the same can be used for any computing device.

10, 12, 14 Display operation separation system 100 Display operation terminal 100a One surface 100b The other surface 110 Display unit, terminal display unit 110a Display area 110b Keyboard image 112 Setting prompt 114 Display control unit 116 Cursor search unit 120 Touch operation input unit 122 touch keyboard control unit 130 terminal reception unit 132 display signal 140 terminal transmission unit 142 operation input signal 150 terminal connection terminal group 162 communication control unit, terminal communication control unit 164 pairing information recording unit, terminal pairing information recording unit 164a pairing Information, terminal pairing information 166 radio wave intensity detection unit 166a detection signal 172 reception control unit 174 pairing information recording unit by protocol 174a pairing information by protocol 176 trial sequence setting unit 176a trial Kens information 192 Terminal content recording unit 194 Terminal image content recording unit 200 Electronic device 200A First electronic device 200B Second electronic device 200c Connection surface 202 Voice call processing unit 204 Function management control unit 210 Display unit, device display unit 210a Display area 210b Inputable field 212 Cursor control unit 220 Touch operation input unit 222 Input control unit 230 Device reception unit 240 Device transmission unit 250 Device connection terminal group 262 Device communication control unit 264 Device pairing information recording unit 264a Pairing information, device pairing information 282 Remote input reception unit 284 Operation control unit 286 Device input reception unit 288 Device display control unit 288a Selection prompt 292 Device content recording unit 294 Device image content recording unit

Claims (26)

  1. A terminal receiver that receives a display signal indicating display content from at least one electronic device;
    A terminal display unit having a display area for presenting the display content indicated by the display signal to the user by at least one of a still image and a moving image or a combination thereof;
    A touch operation input unit that receives an input by a touch operation from the user and generates a touch operation signal;
    A terminal transmission unit that transmits an operation input signal including the touch operation signal to the electronic device;
    A communication control unit that controls establishment and release of connection with the electronic device by at least one of the terminal reception unit and the terminal transmission unit;
    A radio wave intensity detecting unit connected to the terminal receiving unit, capable of measuring the intensity of receivable types of radio waves, and sending a detection signal indicating the received radio wave intensity by the terminal receiving unit;
    The communication control unit controls at least one of the terminal reception unit and the terminal transmission unit based on the detection signal from the radio wave intensity detection unit to establish a connection with the electronic device. Display operation Terminal.
  2. It further includes a pairing information recording unit for storing pairing information for registering the electronic device as one of connection-authenticated devices,
    The communication control unit targets a connection-authenticated device registered in the pairing information when the received radio wave intensity is higher than a predetermined intensity reference value based on the detection signal from the radio wave intensity detection unit. The terminal is configured to control at least one of the terminal receiver and the terminal transmitter to establish a connection with the electronic device.
    The display operation terminal according to claim 1.
  3. A reception control unit that controls establishment and release of a reception connection with the electronic device by the terminal reception unit;
    Protocol-specific pairing information recording unit for storing protocol-specific pairing information for registering the electronic device as one of connection-authenticated devices in correspondence with each of a plurality of communication protocols;
    A trial sequence setting unit for storing trial sequence information for setting trial order and trial time for trying to establish connection of each communication protocol for a plurality of communication protocols for receiving the display signal; and
    The terminal reception unit can receive the display signal from the electronic device by the plurality of communication protocols;
    The reception control unit corresponds to the trial sequence and the trial time of the trial sequence setting unit for a trial process to establish a reception connection from the electronic device registered as connection authenticated in the pairing information by protocol. In the order of the plurality of communication protocols to be performed and the waiting time of each communication protocol, it is executed while switching the communication protocol,
    The display operation terminal according to claim 1.
  4. Either or both of the terminal reception unit and the terminal transmission unit further communicate with the electronic device by wired communication,
    The pairing information by protocol registers the electronic device as one of connection-authenticated devices for the wired communication,
    The trial sequence information stored in the trial sequence setting unit defines the trial order so that the trial order for the wired communication is higher than the wireless communication.
    The display operation terminal according to claim 3.
  5. The terminal transmission unit can transmit identification information for a communication protocol for establishing a reception connection with the electronic device using a communication protocol different from the communication protocol.
    The display operation terminal according to claim 3 or 4.
  6. A finger operation input unit disposed at a position outside the display area for receiving an operation input by a finger from the user;
    The terminal transmission unit includes a signal from the finger operation input unit as a part of the operation input signal and transmits the signal to the electronic device.
    The display operation terminal according to claim 1.
  7. The display operation terminal according to claim 1, wherein information that can reproduce the display content is not retained after the connection with the electronic device is released.
  8. The display operation terminal according to claim 1, wherein part or all of the power used is covered by an external power source or an external battery that is wirelessly connected to power.
  9. The display operation terminal according to claim 1, wherein the terminal display unit exhibits flexibility.
  10. A display operation terminal according to any one of claims 1 to 9,
    An electronic device including a device transmission unit that transmits the display signal and a device reception unit that receives the operation input signal;
    The display content for the electronic device is transmitted to the display operation terminal, and the touch operation input for the display operation terminal is transmitted to the electronic device, thereby displaying the display content of the electronic device and touching it. A display operation separation system in which an operation is executed in the display operation terminal.
  11. The terminal reception unit and the device transmission unit communicate with each other by wired communication, or the terminal transmission unit and the device reception unit communicate with each other by wired communication,
    The display operation terminal is
    A terminal communication control unit that controls establishment and release of connection with the electronic device by both the terminal reception unit and the terminal transmission unit;
    A terminal pairing information recording unit that stores pairing information for registering the electronic device as one of the connection-authenticated devices, and
    The terminal communication control unit is configured such that the electronic device that is not recorded as a connection authenticated device in the terminal pairing information recording unit is connected to the display operation terminal through the terminal transmission unit or the terminal reception unit through the wired communication. In the case where the terminal transmitting unit and the terminal receiving unit are controlled to execute a procedure for mutually authenticating the display operation terminal and the electronic device, and after the connection with the electronic device is established, the electronic device The device is registered in the terminal pairing information as a connection-authenticated device.
    The display operation separation system according to claim 10.
  12. The terminal reception unit and the device transmission unit communicate with each other by wired communication, or the terminal transmission unit and the device reception unit communicate with each other by wired communication,
    The electronic device is
    A device communication control unit that controls establishment and release of the connection with the display operation terminal by both the device reception unit and the device transmission unit;
    A device pairing information recording unit for storing pairing information for registering the display control terminal as one of the connection-authenticated devices, and
    In the device communication control unit, the display operation terminal that is not recorded as a connection authenticated device in the device pairing information recording unit is connected to the electronic device through the device transmission unit or the device reception unit through the wired communication. The electronic device and the display operation terminal are mutually authenticated by controlling at least one of the device transmission unit and the device reception unit when the connection with the display operation terminal is established. The display operation separation system according to claim 10, wherein the display operation terminal is registered in the device pairing information as a connection authenticated device.
  13. The electronic device is
    A device display unit capable of presenting the display content to a user;
    A remote input reception unit that receives an operation input signal transmitted from the terminal transmission unit and received by the device reception unit;
    An operation control unit that controls a shift to a power saving operation or a stop operation of the device display unit based on a signal from the remote input reception unit or a return from any operation, and
    The display operation terminal causes the terminal display unit to display a setting prompt on the display operation of the device display unit of the electronic device as an additional display content on the terminal display unit as an additional display content, The operation input signal including an input of a setting result indicating that the touch operation input unit should accept the power saving operation or the stop operation received from the user in response to the setting prompt is transmitted through the terminal transmission unit,
    When the remote input receiving unit receives the operation input signal through the device receiving unit, the operation control unit shifts the device display unit to a power saving operation or a stop operation based on the operation input signal including the setting result. Is something that works to let you
    The display operation separation system according to claim 10.
  14. The electronic device is
    A device display unit capable of presenting the display content to a user of the electronic device;
    A voice call processing unit that provides a voice call function for the user to make a voice call, and can continue a busy operation even if the function processing unit of the electronic device itself is operating;
    A function management control unit capable of selectively starting, pausing, or stopping the function processing unit of the electronic device itself including the voice call processing unit;
    A remote input reception unit that receives an operation input signal transmitted from the terminal transmission unit and received by the device reception unit, and
    The function management control unit is in a busy state in which the voice call processing unit is operated.
    The display content of the device display unit is returned to a screen that does not include the voice call screen immediately before displaying the voice call screen presented when the voice call processing unit is activated, or the default screen of the operation is displayed. Control it to be one of the home screens,
    Accepting an input of the touch operation input unit from a user of the display operation terminal presented with the display content after control by the terminal display unit, accepting an operation input signal output by the touch operation input unit as the remote input That is accepted by the department,
    The display operation separation system according to claim 10.
  15. The electronic device is
    A device display unit capable of presenting the display content to a user;
    A cursor control unit for displaying an input cursor indicating an input position in an input enabled column included in the display content of the device display unit;
    An input control unit for controlling input display contents to the input position in the device display unit;
    A remote input reception unit that receives an operation input signal transmitted from the display transmission unit and received by the device reception unit, and
    The display operation terminal includes a touch keyboard control unit that generates an input key signal including a key code from the touch operation signal from the touch operation input unit,
    When the touch operation signal indicates a position with respect to a key on which a touch input operation corresponding to the key input is performed among keys included in a keyboard image presented to the user in the terminal display unit And at least outputting the input key signal indicating the key code of the key,
    The terminal transmission unit transmits the operation key signal including the input key signal or the corresponding input key data from the touch keyboard control unit,
    The device reception unit receives the operation input signal from the terminal transmission unit,
    The remote input receiving unit is configured to receive the operation input signal including the input key signal or the corresponding input key data.
    When the key indicated by the key code is pressed based on the key code indicated by the input key signal from the remote input receiving unit or the corresponding input key data, the cursor control unit and the input control unit A change in display to occur is reflected in the display content at the input position corresponding to the input cursor,
    The display operation separation system according to claim 10.
  16. The display operation terminal is
    A display control unit that controls a display position and an enlargement ratio when presenting display contents to the user at the terminal display unit;
    A cursor search unit that searches for an input cursor included in the display content and outputs a cursor position signal indicating the position of the input cursor;
    The display control unit controls a display position so that a position in the display content of the input cursor indicated by the cursor position signal from the cursor search unit does not overlap the keyboard image, and includes the input cursor The display contents in the vicinity are enlarged and displayed.
    The display operation separation system according to claim 15.
  17. The electronic device further includes a device content recording unit capable of storing the entire unit content as a unit of browsing or viewing,
    The display operation terminal further includes a terminal content recording unit capable of storing the entire unit content as a unit of browsing or viewing;
    The electronic device transmits the entire unit content stored in the device content recording unit to the display operation terminal that has established a connection through the device transmission unit,
    The display operation terminal receives the whole transmitted unit content through the terminal reception unit and stores it in the terminal content recording unit,
    The electronic device and the display / operation terminal release the mutual connection from either or both sides after the transmission / reception of the entire unit content is completed,
    The display operation terminal is capable of displaying, on the terminal display unit, contents related to the display of the unit content stored in the terminal content recording unit after the connection with the electronic device is released.
    The display operation separation system according to claim 10.
  18. The electronic device further includes a device image content recording unit capable of storing at least one image of image content including a plurality of images;
    The display operation terminal further includes a terminal image content recording unit capable of storing at least one image of image content including a plurality of images;
    The electronic device transmits one or more images among a plurality of images of a certain image content stored in the device image content recording unit to the display operation terminal having established a connection through the device transmission unit. And
    The display operation terminal receives the one or more images transmitted until the connection is released among the plurality of images of the transmitted image content through the terminal reception unit and stores them in the terminal image content recording unit Is what
    The electronic device and the display / operation terminal release the mutual connection from either or both sides after the transmission / reception of the one or more images is completed.
    The display operation terminal is capable of displaying the last transmitted image stored in the terminal content recording unit on the terminal display unit after the connection with the electronic device is released.
    The display operation separation system according to claim 10.
  19. Both have a first display operation terminal and a second display operation terminal, which are the display operation terminals,
    The device transmission unit of the electronic device generates a common display signal indicating display content common to both the first display operation terminal and the second display operation terminal, a display signal for the first display operation terminal, and the first display operation terminal. 2 It is included in each display signal for the display operation terminal and transmitted.
    The device receiving unit of the electronic device receives the operation input signals from the first display operation terminal and the second display operation terminal separately from each other.
    The display operation separation system according to claim 10.
  20. The electronic device is
    A device display unit capable of presenting the display content to a user;
    A device input receiving unit that receives an operation input from the user;
    A remote input reception unit that receives an operation input signal received from the first display operation terminal and the second display operation terminal via the device reception unit;
    A device display control unit that controls the additional display content presented in the device display unit based on signals from the device input reception unit and the remote input reception unit,
    The device display control unit superimposes a selection prompt on the display content as the additional display content for selecting which of the first display operation terminal and the second display operation terminal should accept the operation input signal. Is displayed on the device display unit,
    The device input receiving unit is configured to receive an input of a selection result from the user of the electronic device,
    The remote input reception unit is configured to selectively receive an operation input signal from either the first display operation terminal or the second display operation terminal according to the selection result. Display operation separation system.
  21. Both of the electronic devices include a first electronic device and a second electronic device,
    The terminal receiving unit of the display operation terminal receives the first display content that is the display content of the first electronic device and the second display content that is the display content of the second electronic device separately from each other. And
    The terminal display unit displays the first display content and the second display content,
    The touch operation input unit generates the touch operation signal that can be determined from a touch position as to which of the first display content and the second display content is input,
    The terminal transmission unit transmits the operation input signal including the touch operation signal to the first operation input signal corresponding to the first display content displayed on the terminal display unit and the terminal display unit. The second operation input signal corresponding to the displayed second display content can be distinguished from each other and transmitted to each of the first electronic device and the second electronic device. Yes,
    The display operation separation system according to claim 10.
  22. The display operation terminal has a flat plate shape in use,
    The terminal display unit of the display operation terminal is arranged along one of both surfaces forming the thickness of the flat plate,
    The display operation terminal is provided with a terminal connection terminal group on the other surface of the both surfaces,
    The electronic device is provided on a connection surface, which is a device connection terminal group connectable with the terminal connection terminal group,
    The electronic device is attached to the other surface of the display operation terminal in a state where the terminal connection terminal group and the device connection terminal group are connected to each other.
    The display operation separation system according to claim 10.
  23. The device transmitter and the terminal receiver communicate by wireless communication.
    The display operation separation system according to claim 10.
  24. The device reception unit and the terminal transmission unit communicate by wireless communication.
    The display operation separation system according to claim 10.
  25. The display operation terminal does not hold information that can reproduce the display content after releasing the connection with the electronic device.
    The display operation separation system according to claim 10.
  26. The terminal transmission unit can transmit identification information for a communication protocol for establishing a reception connection with the electronic device using a communication protocol different from the communication protocol.
    The display operation separation system according to claim 10.
PCT/JP2018/007129 2017-02-28 2018-02-27 Display/operation terminal and display/operation separation system using same WO2018159579A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2017-035891 2017-02-28
JP2017035891 2017-02-28

Publications (1)

Publication Number Publication Date
WO2018159579A1 true WO2018159579A1 (en) 2018-09-07

Family

ID=63370696

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2018/007129 WO2018159579A1 (en) 2017-02-28 2018-02-27 Display/operation terminal and display/operation separation system using same

Country Status (1)

Country Link
WO (1) WO2018159579A1 (en)

Citations (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002198887A (en) * 2000-12-25 2002-07-12 Shin Caterpillar Mitsubishi Ltd Communication control method for machinery and device therefor
JP2002202941A (en) * 2000-08-25 2002-07-19 4C Soft Inc Multimedia electronic learning system and learning method
JP2003244343A (en) * 2002-02-21 2003-08-29 Toyota Motor Corp Display device, portable terminal and information display system
JP2006512794A (en) * 2002-07-12 2006-04-13 ソニー・エリクソン・モバイルコミュニケーションズ, エービー Wireless communication terminal with detachable active housing and associated connector
JP2010530110A (en) * 2007-06-29 2010-09-02 ノキア コーポレイション System, method, apparatus and computer program product for downloading content for offline browsing
JP2011176591A (en) * 2010-02-24 2011-09-08 Nec Corp Radio communication system using mobile terminal
JP2012019413A (en) * 2010-07-08 2012-01-26 Toshiba Corp Display system, terminal device, display device, and program
US20120242687A1 (en) * 2011-03-25 2012-09-27 Choi Sungha Image processing apparatus and image processing method
US20130113717A1 (en) * 2011-11-09 2013-05-09 Peter Anthony VAN EERD Touch-sensitive display method and apparatus
JP2013536613A (en) * 2010-07-10 2013-09-19 アイシーイー コンピューター、インコーポレイテッド Intelligent platform
JP2013219420A (en) * 2012-04-04 2013-10-24 Denso Corp Communication device, program, and communication establishing method
JP2013228562A (en) * 2012-04-25 2013-11-07 Sharp Corp Display device, transmission device, display system, control method of display device, control method of transmission device, control program, and recording medium
JP2014525065A (en) * 2011-05-31 2014-09-25 アップル インコーポレイテッド Device, method and graphical user interface for document manipulation
JP2014236351A (en) * 2013-05-31 2014-12-15 株式会社アイ・オー・データ機器 Communication connection system and program therefor
JP2015133577A (en) * 2014-01-10 2015-07-23 株式会社Nttドコモ communication device, communication method and program
JP2015191487A (en) * 2014-03-28 2015-11-02 株式会社日立ソリューションズ presentation server and mobile terminal
JP2016046797A (en) * 2014-08-25 2016-04-04 現代自動車株式会社Hyundaimotor Company Bluetooth pairing method using wired connection
WO2016084485A1 (en) * 2014-11-27 2016-06-02 ソニー株式会社 Information processing device and information processing method
JP2016219416A (en) * 2015-05-18 2016-12-22 株式会社半導体エネルギー研究所 Power storage device and electronic apparatus

Patent Citations (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002202941A (en) * 2000-08-25 2002-07-19 4C Soft Inc Multimedia electronic learning system and learning method
JP2002198887A (en) * 2000-12-25 2002-07-12 Shin Caterpillar Mitsubishi Ltd Communication control method for machinery and device therefor
JP2003244343A (en) * 2002-02-21 2003-08-29 Toyota Motor Corp Display device, portable terminal and information display system
JP2006512794A (en) * 2002-07-12 2006-04-13 ソニー・エリクソン・モバイルコミュニケーションズ, エービー Wireless communication terminal with detachable active housing and associated connector
JP2010530110A (en) * 2007-06-29 2010-09-02 ノキア コーポレイション System, method, apparatus and computer program product for downloading content for offline browsing
JP2011176591A (en) * 2010-02-24 2011-09-08 Nec Corp Radio communication system using mobile terminal
JP2012019413A (en) * 2010-07-08 2012-01-26 Toshiba Corp Display system, terminal device, display device, and program
JP2013536613A (en) * 2010-07-10 2013-09-19 アイシーイー コンピューター、インコーポレイテッド Intelligent platform
US20120242687A1 (en) * 2011-03-25 2012-09-27 Choi Sungha Image processing apparatus and image processing method
JP2014525065A (en) * 2011-05-31 2014-09-25 アップル インコーポレイテッド Device, method and graphical user interface for document manipulation
US20130113717A1 (en) * 2011-11-09 2013-05-09 Peter Anthony VAN EERD Touch-sensitive display method and apparatus
JP2013219420A (en) * 2012-04-04 2013-10-24 Denso Corp Communication device, program, and communication establishing method
JP2013228562A (en) * 2012-04-25 2013-11-07 Sharp Corp Display device, transmission device, display system, control method of display device, control method of transmission device, control program, and recording medium
JP2014236351A (en) * 2013-05-31 2014-12-15 株式会社アイ・オー・データ機器 Communication connection system and program therefor
JP2015133577A (en) * 2014-01-10 2015-07-23 株式会社Nttドコモ communication device, communication method and program
JP2015191487A (en) * 2014-03-28 2015-11-02 株式会社日立ソリューションズ presentation server and mobile terminal
JP2016046797A (en) * 2014-08-25 2016-04-04 現代自動車株式会社Hyundaimotor Company Bluetooth pairing method using wired connection
WO2016084485A1 (en) * 2014-11-27 2016-06-02 ソニー株式会社 Information processing device and information processing method
JP2016219416A (en) * 2015-05-18 2016-12-22 株式会社半導体エネルギー研究所 Power storage device and electronic apparatus

Similar Documents

Publication Publication Date Title
KR101517082B1 (en) Mobile terminal using flexible display and operation method thereof
US8866698B2 (en) Multi-display handheld device and supporting system
US9977539B2 (en) Mobile terminal and method for controlling the same
CN202548700U (en) Accessory equipment and flexible accessory equipment
US8543166B2 (en) Mobile terminal equipped with flexible display and controlling method thereof
US8958747B2 (en) Mobile terminal, mobile terminal system, and method for controlling operation of the same
US20120282914A1 (en) Smart phone companion loop
KR101832958B1 (en) Mobile terminal and 3d image displaying method thereof
US20180307268A1 (en) Finger-wearable computing devices
KR101991133B1 (en) Head mounted display and the method for controlling the same
KR20110136587A (en) Mobile terminal and operation method thereof
KR101685363B1 (en) Mobile terminal and operation method thereof
JP2010503052A (en) Configurable personal audio / video device for use in networked application sharing systems
KR101659032B1 (en) Mobile terminal and control method thereof
US20100081505A1 (en) Accessory for playing games with a portable electronic device
KR100826872B1 (en) Wearable computer system and method controlling information/service in wearable computer system
US9654589B2 (en) Configurable personal audiovisual device for use in application-sharing system
KR20100023603A (en) Mobile terminal and operation control method thereof
KR20160000793A (en) Mobile terminal and method for controlling the same
KR101758164B1 (en) Mobile twrminal and 3d multi-angle view controlling method thereof
US20130109371A1 (en) Computing device operable to work in conjunction with a companion electronic device
JP2016059030A (en) Portable electronic device
CN101924823A (en) Mobile terminal and method of controlling operation of the mobile terminal
CN104298450A (en) Mobile terminal and control method of same mobile terminal
US9807694B2 (en) Mobile terminal and control method thereof

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18761014

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase in:

Ref country code: DE