DE102013004781A1 - Device for accessing data from mobile phones and program for carrying out the access - Google Patents

Abstract

The invention relates to a docking station with the help of which conventional peripheral devices of computers can be connected to a mobile phone, in particular a smartphone, in such a way that it is possible to work on the mobile phone using a mouse, keyboard and screen. In addition, the coupling of the smartphone and the transfer of commands from the docking station to the smartphone are controlled via a program according to the invention. The program according to the invention is also responsible for transferring the data, for example screen contents, in the opposite direction from the smartphone to the docking station and displaying it there on the monitor.

Description

The invention relates to a device for accessing data from mobile phones, in particular smartphones, and a program for performing the access. Thus, it should be possible to use conventional computer peripherals for the use of programs on mobile phones, without the access time to these programs is too large.

State of the art

According to the Federal Statistical Office, one in three companies today provide their employees with devices with a mobile Internet connection, such as a smartphone. This makes it possible to work with colleagues everywhere on the same project, especially in global work processes where you can work with colleagues from the US, China or other countries. During the process, the smartphone is regularly synchronized with the PC or a similar device by synchronization. But to be able to use the mobile phone even more efficiently, you have to integrate it better into the workflow. Problems such as printing or writing text on smartphones hinder this process. As a solution, the idea of a "docking station" was developed, which allows the smartphone with mouse, keyboard, screen, etc. to be linked.

That's how it describes DE 10 2006 011 538 A1 a docking station in the form of a laptop, where the smartphone is to represent the heart of the device. About certain interfaces such. B. USB, devices own plug, WiFi or Bluetooth data are transmitted from the smartphone to the docking station, which according to the invention comprises the screen and other peripherals. The desired in this document operation of the docking station via the touchpad of the smartphone is used essentially to run only the playback and possibly backup via the docking station, essential functions of the smartphone can not be controlled by the docking station. Another disadvantage is that the docking station must be designed as a rule device-dependent, so that you need a new docking station when changing the smartphone. In addition, the data transfer rate via USB or other plugs but also via wireless communication links is limited by the limitation of the data volume.

To improve the wireless transmission of data provides the font DE 10 2011 105 874 A1 an electronic device with which audio, video and / or video data can be transmitted from a mobile player to external playback devices. For this purpose, an external electronic device is connected to the mobile playback device, which is used for transmission to the playback device. Again, a device-dependent design is again required because a direct physical coupling of the devices must be made. In addition, the player must have a video output, so that video data can be transmitted. This greatly limits the number of usable devices.

In the Revelation DE 11 2010 003 350 T5 For example, the extension of communication between a single PC and a single communication device via Remote Desktop Protocol to communications between a communication device and a variety of other computing devices is described. One difficulty here is the slow data transfer performance of the protocol described above, which makes liquid handling difficult. Therefore, according to the invention, a PC is also always used, which processes the data from the communication device, so that the communication device is only treated as a storage medium.

All of the prior art disclosures have in common that the mobile phone is only coupled to a computer, but the processing of the data on the mobile phone is not taken over by the mobile phone but by the computer.

Presentation of the invention

In the age of the modern office environment, in which employees sometimes sit on other workplaces on a daily basis and also regularly have home office days, the described coupling is perceived as a hindrance. Either a variety of mobile phone-adapted docking stations would be needed or a fully equipped and therefore expensive PC at each workstation.

The object of the invention is therefore to provide a docking station, which serves merely as a communication element between the mobile phone and peripherals, while the actual computer power is provided by the mobile phone. The aim is also to provide a docking station, which automatically initiates a connection of the smartphone with the docking station when the owner arrives at the workplace, so that one can immediately start to work.

Since the above mentioned difficulties of data transfer between mobile phone and docking station when using the CPU of the Mobile phones as the host lead to delays in processing, it is also an object of the invention to provide a program that allows unrestricted working at high speed.

To achieve the first object, the connection of an ungrooved smartphone with a docking station according to the invention can be carried out in a first embodiment. This can already record the mouse and keyboard input and send it back to the mobile phone and also transfer the screen content of the mobile phone to the docking station, which displays this via a screen. To do this, the mobile phone is connected to the docking station via a USB cable.

In another embodiment, the improvement of the screen transmission is performed by a video compression, whereby the docking station is only slightly burdened with computing power. But for this embodiment, a rooted mobile phone is required, d. H. the user must have superuser rights (administrator rights), which allow unrestricted access to all functions. In this way, a connection via WLAN can be made possible, whereby the further goal is achieved. For Wi-Fi connections, the mobile phone must also be rooted.

Since the docking station according to the invention only serves to display the contents of the mobile phone on the screen and to forward user input via mouse and keyboard to the mobile phone, a program is required for fluid information shift between the mobile phone and docking station, with which the Android mobile phone can be operated. In principle, this application has the task to display the current screen content of the smartphone on the monitor and to accept the user input and forward it to the mobile device.

The easiest way to communicate between the mobile phone and the docking station is to use the USB interface. So z. For example, in the Software Development Kit for Android (Android SDK for short), a collection of libraries and applications needed to program for the Android operating system, it already includes a utility to share data between your PC and your mobile device. This so-called Android Debug Bridge (ADB) is normally operated from the command line and can send various commands to the smartphone. For example, it is possible to install, open, close or copy files from the PC to the phone. It starts for this on the computer a server, which runs in the background and with a Daemon, a background program, on which Smartphone exchanges data. Typically, the ADB is used during the development phase of an app for troubleshooting. For the user of the device, this connection is not visible and has no further effects. However, this requires that the option "USB debugging" is activated on the phone in the menu, otherwise the ADB can not be started. This is to prevent malicious applications from secretly remotely controlling the phone without the user's knowledge.

Using this connection it is possible to save the data from the program, eg. B. a mouse click of the user to send from the computer via the USB cable to the phone. On the phone, these data packets, that is, the input of the user, now be evaluated and trigger the right action. For this purpose, a program is used, which is part of the Android operating system, the so-called "Monkey". This program is actually meant to stress newly developed apps. This can be numerous, very fast consecutive user input such. For example, simulate screen touches to see if they are properly processed in each situation. Through a special interface for application programming, also known as Application Programming Interface (API), it is possible to address this "Monkey" from the computer via USB connection. So you can transfer from the PC an instruction such as "touch the screen at the coordinates x = 100, y = 100" to the Monkey program on the smartphone. This receives the instruction and then simulates a finger touch of the user at the specified location. The operating system itself does not notice any difference between the actual touch inputs on the display and the faked instructions and performs the appropriate action. In addition to screen touches, it is also possible with the "Monkey" to simulate keyboard input on the mobile phone. Again, the device behaves exactly as if you were writing directly on the on-screen keyboard.

The second important part of the application is transferring the screen of the smartphone to the screen on the docking station. For this purpose, the already described API and the program called "Monkey" are also used. This provides the ability to read the image memory of the Android device, so create a screenshot. The image can then be sent via the USB connection to the computer and further processed there.

This was a way to implement all the desired features of the application found. The program now follows a simple scheme. After the start, the ADB first establishes a data connection to the connected smartphone. If successful, the program will cause the "Monkey" to take several screenshots per second and transfer them to the computer. There they are displayed in a program window, so that you can see the screen content of his smartphone on the monitor in front of him. The user now has the option of controlling his mobile phone from the PC with mouse clicks on the window and keystrokes. Both the simple click and other touchscreen inputs are possible. The changes that cause the inputs of the user on the phone, are immediately visible on the PC screen, as by the constant transmission of screenshots to the computer, a nearly liquid image reproduction is achieved. In this way, the impression arises that the visible content in the program window is not the operating system of a separate device, but a conventional PC program.

Only functions that are already integrated into the operating system on every Android device are used on the smartphone for the program. This has several positive side effects: on the one hand, the user does not need to install any additional software on the phone before he can use the application. All you have to do is select the function "Enable USB Debugging" in the menu, as already mentioned. In addition, using existing system modules ensures that the application works with any connected Android device, no matter which manufacturer it comes from or which operating system version it uses. Thus, the program is more or less a simple universal solution for operating the smartphone from the PC.

However, there are some problems with this method of transmitting user input to the mobile phone, and especially when reading the screen contents. So is the frame rate, so the number of images sent per second from the mobile device to the computer, depending on the screen resolution of the phone. The higher the resolution of the display, the more data has to be transmitted per second to reach a frame rate of about 30 fps (frames per second). This refresh rate is just perceived by the human eye as a fluid movement. Currently, screen resolutions of 800 x 480 pixels, or more expensive devices, even 1280 x 720 pixels, are widespread. Cheap smartphones sometimes have a resolution of 480 x 320 pixels. The problem now is that the system tool "Monkey", which is used for image transfer, does not compress the screenshots before sending them. For this reason, the speed of the USB 2.0 connection that is currently used by all smartphones is often insufficient for a reasonably fast image transfer. This can be shown by a simple calculation: an uncompressed image of the display has a file size of 480 x 800 x 32 = 12.288 MBit at a resolution of 800 x 480 pixels and a color depth of 32 bits. Of these images, at least thirty must be transmitted per second, which corresponds to a data rate of 12.288 Mbit x 30 = 368.64 Mbps. USB 2.0 theoretically allows a data rate of 480 MBit / s under optimal conditions. In practice, this speed is due to many factors, such. B. the signal quality and built-in PCs and mobile phones USB controller, but never reached. The result is a halting playback of the screen content of the smartphone on the PC monitor.

To solve this problem, it is necessary to compress the image data before transferring through the USB cable. This could be done in two main ways: first, it would be possible to extend the "Monkey" with a compression function. However, this would mean a deep intrusion into the operating system, forcing the user to install a modified operating system version before he can use the program. However, such a hurdle is not reasonable for the user, so this path is eliminated. The second option is to write your own app that reads the screen contents, compresses them and sends them to the computer. The problem here is that theoretically no user-installed application on the smartphone is allowed to access the framebuffer, which is the image memory of the display.

As a docking station, a device had to be found, which takes up little space and still provides a relatively high computing power. A single-board computer, which occupies a very small space and thus far undercuts a conventional computer with a four-hundredth of the volume of space required, is very inexpensive to purchase. He has a very low power consumption, but still has sufficient power. Furthermore, the small computer should be equipped with 512 MB of RAM and several have at least two USB slots. As the operating system, different distributions of Linux and some lesser known operating systems are suitable. The docking station has no internal main memory, which is why an SD card in a required SD slot is used as memory.

The docking station has the software Java SDK and a UDEV file, with the help of which Device also recognizes and accepts all mobile phones when connected.

As already explained, the publicly available software that uses the "Monkey" for image transmission can not guarantee a smooth image display with all connected smartphones. For this reason, a software is provided according to the invention, which compresses the screen content before transmission to the PC or the docking station. This reduces the amount of data sent so that the speed of USB 2.0 is sufficient for a fluid transfer. In addition, this program should also ensure that the mobile phone can optionally be connected via WLAN with the docking station, as it is desired task. The application should essentially read the screen content, compress it and send it via WLAN or USB to the docking station.

The first hurdle is a security feature of the Android operating system. It is generally not possible for applications installed by the user to access certain areas of the mobile phone. These "locked" areas also include the frame buffer of the device, ie the memory area in which the screen content is stored. This is to prevent an app with virus functionality from reaching the smartphone and spying on the user. Unfortunately, this protective measure prevents the execution of many benign programs. So that the app can still access the image memory, you have to bypass this security provision. This can be achieved by starting an application with extended permissions, so it runs as a so-called "root user" or "super-user". Given this, even a user-installed app can access all areas of the phone, as well as the operating system itself. How to obtain this root privilege differs widely from device to device, but all have in common that you have the guarantee of Manufacturer loses on the smartphone when you root it. Depending on the model, this process can be simple or very complicated to impossible. In any case, the intervention is at your own risk and in the worst case, it may happen that the device does not start after a failed "root attempt". It is therefore sensible to have an option set up by the manufacturer.

To code a video efficiently, there are countless different methods available that have all the advantages and disadvantages. For the transmission of the display content, a h.264 encoded video stream is preferred. H.264 is a codec that is used very often and compresses the image material very much. Disadvantage is that generating a h.264 video compared to the coding of a video with other methods requires relatively much computing power. The Android operating system provides an h.264 encoder, so a program to encode a video in this process, only from version 3.0 or higher ready. So that users who still use a smartphone with an older operating system version can also use the app, a corresponding codec, for example, from the so-called "FFmpeg project" is recommended. This is a collection of free programs and libraries that allow you to capture, convert, send, and package video and audio files in various file formats. FFmpeg is developed on a Linux operating system, but there are variants for many other platforms, eg. Windows. Thus, there is a way to encode the screen content of the smartphone. Also for the "streaming", so the sending of the video, FFmpeg provides a function. To send the video stream over Wi-Fi, the smartphone and the docking station must be logged in to the same Wi-Fi network. Such a connection can either be established by both the docking station and the mobile phone both logs on to a wireless router, or operates the mobile phone as a mobile Wi-Fi hotspot. This means that the smartphone spans its own Wi-Fi network, in which other devices, such. As the docking station can dial. There are conceivable scenarios for both variants: for example, if you want to use an existing router as an access point to the Internet, which is usually the case at work or at home, then it is recommended to use the former option. But if you want to use the mobile Internet of the smartphone, it is more advantageous to establish a direct radio connection.

If there is a network connection, the screen content can be sent as live video to the docking station. The program then presents the video in a program window, which the user can then use to operate the smartphone as described in Section 2.1.

In an advantageous embodiment of the invention, the docking station goes into standby mode as soon as one leaves the WLAN area with the mobile phone.

Embodiment of the invention

The invention will be explained in more detail with reference to two drawings. 1 shows a scheme of the connection of devices using the device according to the invention for accessing data from mobile phones. In 2 is the program flow shown in the form of a flow chart.

The mobile phone 1 is in 1 with the help of wireless technology with the device 2 to access data from mobile phones 1 coupled. At the facility 2 is a screen 3 arranged on which the screen display of the mobile phone 1 is reproduced. In addition, other peripherals, such. As mouse and keyboard, connected.

In 2 a flow chart is shown, from which can be removed, which program steps in the communication between mobile phone 1 and furnishings 2 to access data from mobile phones 1 respectively.

As is the case with the previous device described in detail 2 By way of example, modifications may be made in a conventional manner to a wide extent by those skilled in the art without departing from the scope of the invention. In particular, the concrete embodiments of the device can 2 in a different form than that described here. Likewise, the software can be designed in a different form, if this is necessary depending on the device. Further, the use of the indefinite article "on" does not preclude that the features in question may also be duplicated.

LIST OF REFERENCE NUMBERS

1

mobile phone

2

Device for accessing data from mobile phones

3

screen

4

peripherals

Legend Fig. 2:

• a) Smartphone connected to the PC / Dock (for example via Android Debug Bridge)?
• b) Selection of the desired connection type
• c) USB or WLAN selected?
• d) Connecting via USB
• e) Connecting via WLAN
• f) Connection successful?
• g) Display of the program window
• h) Connection successful?
• i) error message
• j) repeat connection attempt?
• k) as long as the program is not terminated / connection to the smartphone is not interrupted
• l) Transfer of the current screen contents of the smartphone to PC / Dock
• m) Display of the current screen content of the smartphone
• n) File transfer from the PC / Dock to the smartphone or from the smartphone to the PC / Dock
• o) User input is done?
• p) drag and drop action?
• q) transmission of user input to the smartphone

QUOTES INCLUDE IN THE DESCRIPTION

This list of the documents listed by the applicant has been generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• DE 102006011538 A1 [0003]
• DE 102011105874 A1 [0004]
• DE 112010003350 T5 [0005]

Claims (7)

Device for accessing data from mobile phones comprising a computer, an SD card slot, at least two USB ports and a digital image output, said device being coupleable to a mobile phone, characterized in that the device comprises a program which generates commands from inputs , with which the mobile phone is controllable, and image output of the mobile phone on a screen, which is connected via the digital image output represents. Device for accessing data from mobile phones according to claim 1, characterized in that the coupling takes place via a USB cable. Device for accessing data from mobile phones according to claim 1, characterized in that the coupling takes place by means of WLAN. Device for accessing data from mobile telephones according to one of the preceding claims, characterized in that the device for accessing data from mobile telephones is a single-board computer. A program for accessing data from mobile telephones on a device for accessing data from mobile telephones, characterized in that the program is adapted to bring about a coupling between the device for accessing data from mobile telephones and a mobile telephone. A program for performing access to data from mobile phones on a device for accessing data from mobile telephones according to claim 5, characterized in that the coupling of the device for accessing data from mobile phones takes place automatically as soon as the mobile phone is within reach of the device for accessing Data from mobile phones arrives. Program package for carrying out the access to data of mobile phones, characterized in that a program from the program package is computer-installed, and this routine converts the coupling of the means for accessing data of mobile phones, and each command input to the device into a recognizable by the mobile phone command, and transferred from the mobile phone image data into a display format for the screen, and that another program is installed on the mobile phone side, this program all the image data of the mobile phone compression and transmitted to the device for accessing data from mobile phones.

Images