WO2016079355A1 - Computing method, system and product for interacting with a laser pointer - Google Patents

Abstract

The invention relates to a computerized method for interacting with a laser pointer (300) by means of a portable electronic device (100) over a projection area (401) defined by a projector (400) connected to a computer (2), in such a way that the interactivity of the user with a presentation of slides is improved, permitting a number of actions to be carried out using the pointer, via a mobile application and a computing system, such as, inter alia, drawing on the presentation or turning the page.

Description

 METHOD, SYSTEM AND COMPUTER PRODUCT TO INTERACT WITH A

 LASER POINTER

DESCRIPTION

The object of the present invention is a method, a system and a computer product applicable to smart mobile phones for interaction with a laser pointer in such a way that the interactivity of the user is improved with a slide show allowing to perform, through a mobile application and a computer system, perform numerous actions with the pointer itself, such as drawing on the presentation, turning the page, or any other.

State of the art

Technology has evolved rapidly, for example, the mobile phone industry. This has evolved from devices whose sole purpose was voice communication, to countless models of smartphones, with many services.

However, the presentations have not evolved almost anything in the last ten years, not to say they have not changed at all. A projector, a computer and a laser pointer are common elements that are used with minimal interaction. This has caused a problem, the lack of attention of the listeners. In addition, in many cases, the presenter is very limited and has little control over his presentation. This generates an inefficient presentation style where the user has to return to the computer to move to the slides. Again, this causes a lack of attention of the listeners.

The lack of attention generates an even greater problem, which is inefficient learning. Due to the attention deficit of listeners, everything indicated in the presentation tends to be forgotten. Subsequently, if at any time the listener wants to review or reread the presentation, nothing said by the presenter is remembered. In most cases it is about deciphering the slides and more time is wasted trying to remember what was said.

Therefore, a system that improves the interaction between the user and his slide show through a standard laser pointer is recommended.

Description of the invention

It is an object of the invention to improve slide shows, both interactivity and visual appeal. The main objective is to increase the effectiveness of this presentation for both the presenter and the listener.

The solution to the technical problem is to place a mobile device in front of where the projector that is projecting the images of the presentation is placed and while the presenter exposes his project, the user will be able to manage his presentation through a laser pointer, being able to draw different figures and erase, pass or go back in the presentation. For example, if you want to highlight an important formula, a circle or rectangle will be painted around that formula with the laser pointer, automatically appearing on the screen.

More specifically, the solution to the indicated technical problem is described in the independent claims that accompany the following specification, as well as its various particular embodiments, described in each of the dependent claims.

Throughout the description and the claims the word "understand" and its variants are not intended to exclude other technical characteristics, additives, components or steps. For those skilled in the art, other objects, advantages and features of the invention will be derived partly from the description and partly from the practice of the invention. The following examples and drawings are provided by way of illustration, and are not intended to restrict the present invention. In addition, the present invention covers all possible combinations of particular and preferred embodiments indicated herein.

Brief description of the figures

A series of drawings that help to better understand the invention and that expressly relate to an embodiment of said invention which is presented as a non-limiting example thereof is described very briefly below.

FIG 1.- Shows a diagram of the portable electronic device

 (100) that implements the present invention.

 FIG 2.- Shows the diagram of an architecture of a computer (2) illustrating the architecture of a server used in the different practical embodiments of the invention.

 FIG 3.- Shows the diagram of the architecture of the system object of the present invention.

 FIG 4.- Block diagram of the recognition operations implemented in the mobile device (100).

Statement of a detailed embodiment of the invention

The present invention is implemented in a portable electronic device 100 which can be any selected among computers, tablets and mobile phones, although a preferred architecture for a mobile device is shown in Figure 1. In general, any programmable communications device can be configured as a device for the present invention

Figure 1 illustrates a portable electronic device, in accordance with some embodiments of the invention. The portable electronic device 100 of the invention includes a memory 102, a memory controller 104, one or more processing units (CPU) 106, a peripheral interface 108, an RF circuit system 112, a circuit circuit system 114 audio, a speaker 116, a microphone 118, an input / output (I / O) subsystem 120, a touch screen 126, other input or control devices 128 and an external port 148. These components communicate over one or more communication buses or signal lines 110. The device 100 may be any portable electronic device, including, but not limited to, a laptop, a tablet, a mobile phone, a media player, a personal digital assistant (PDA), or the like, including a combination of two or More of these items. It should be appreciated that the device 100 is only an example of a portable electronic device 100 and that the device 100 may have more or less components than those shown or a different configuration of components. The different components shown in Figure 1 can be implemented in hardware, software or in a combination of both, including one or more specific application integrated circuits and / or signal processing. Similarly, the screen 126 has been defined as touch, although the invention can also be implemented in devices with a standard screen.

Memory 102 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic disk storage devices, flash memory devices or other non-volatile solid-state memory devices. In some embodiments, memory 102 may further include storage located at remotely with respect to one or more processors 106, for example a storage connected to a network that is accessed through the RF circuit system 112 or the external port 148 and a communications network (not shown) such as the Internet, intranet (s), local area networks (LAN), extended local area networks (WLAN), storage area networks (SAN) and so on, or any of their appropriate combinations. Access to memory 102 by other components of device 100, such as CPU 106 and peripheral interface 108, can be controlled by memory controller 104.

The peripheral interface 108 connects the device's input and output peripherals to the CPU 106 and to the memory 102. One or more processors 106 execute different software programs and / or instruction sets stored in the memory 102 to perform different functions of the device. 100 and for data processing.

In some embodiments, peripheral interface 108, CPU 106 and memory controller 104 may be implemented on a single chip, such as chip 111. In some other embodiments, they may be implemented on several chips.

The RF (radio frequency) 112 system receives and sends electromagnetic waves. The RF circuit system 112 converts electrical signals into electromagnetic waves and vice versa and communicates with communications networks and other communication devices through electromagnetic waves. The RF circuit system 112 may include a circuit system widely known to perform these functions, including but not limited to, an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a CODEC chipset, a subscriber identity module (SIM) card, a memory, etc. The 112 system of RF circuits can communicate with networks, such as the Internet, also called World Wide Web (WWW), an Intranet and / or a wireless network, such as a cellular telephone network, a wireless local area network (LAN) and / or a metropolitan area network (MAN) and with other devices via wireless communication. Wireless communication can use any of a plurality of communications standards, protocols and technologies, including, but not limited to, the global system for mobile communications (GSM), the enhanced GSM data environment (EDGE), multiple access by Broadband code division (W-CDMA), multiple code division access (CDMA), time division multiple access (TDMA), Bluetooth, wireless access (Wi-Fi) (for example, IEEE 802.11a , IEEE 802.11b, IEEE 802. llg and / or IEEE 802.11η), voice over IP protocol (VoIP), Wi-MAX, a protocol for email, instant messaging and / or short message service (SMS) or any another suitable communication protocol, including communication protocols not yet developed on the date of presentation of this document.

The audio circuit system 114, the speaker 116 and the microphone 118 provide an audio interface between a user and the device 100. The audio circuit system 114 receives audio data from the peripheral interface 108, converts the data from audio in an electrical signal and transmits the electrical signal to the speaker 116. The speaker converts the electrical signal into audible sound waves for the human being. The audio circuit system 114 also receives electrical signals converted by the microphone 116 from sound waves. The audio circuit system 114 converts the electrical signal into audio data and transmits the audio data to the peripheral interface 108 for processing. Audio data can be retrieved and / or transmitted to memory 102 and / or RF circuit system 112 via peripheral interface 108. In some embodiments, audio circuit system 114 includes also a headphone connection (not shown). The headphone connection provides an interface between the system 114 of audio circuits and detachable audio input / output peripherals, such as a head-only headphone or a headphone both output (headphones for one or both ears) and input (microphone ).

The I / O subsystem 120 provides the interface between the input / output peripherals of the device 100, such as the touch screen 126 and other input / control devices 128, and the peripheral interface 108. The I / O subsystem 120 includes a touch screen controller 122 and one or more input controllers 124 for other input or control devices. The input controller (s) 124 receives / sends (n) electrical signals from / to other input or control devices 128. The other input / control devices 128 may include physical buttons (e.g. push buttons, rocker buttons, etc.), dials, slide switches, and / or geographic location means 201, such as GPS or equivalent.

The touch screen 126 in this practical embodiment provides both an output interface and an input interface between the device and a user. The touch screen controller 122 receives / sends electrical signals from / to the touch screen 126. Touch screen 126 shows the visual output to the user. The visual output can include text, graphics, video and any of its combinations. Part or all of the visual output may correspond to user interface objects, the additional details of which are described later.

The touch screen 126 also accepts user inputs based on the haptic or touch contact. The touch screen 126 forms a contact-sensitive surface that accepts user inputs. The touch screen 126 and the touch screen controller 122 (together with any of the associated modules and / or instruction sets of memory 102) detects the contact (and any movement or loss of contact) on the touch screen 126 and converts the detected contact into interaction with the user interface objects, such as one or more programmable keys that are displayed on the touch screen. In an exemplary embodiment, a contact point between the touch screen 126 and the user corresponds to one or more fingers of the user. The touch screen 126 may use LCD technology (liquid crystal display) or LPD (light emitting polymer screen) technology, although other display technologies may be used in other embodiments. The touch screen 126 and the touch screen controller 122 can detect the contact and any movement or lack thereof using any of a plurality of contact sensitivity technologies, including, but not limited to, capacitive, resistive, infrared and of surface acoustic waves, as well as other arrangements of proximity sensors or other elements for determining one or more points of contact with the touch screen 126.

The device 100 also includes a feeding system 130 for feeding the different components. The power system 130 may include a power management system, one or more power supplies (e.g. batteries, alternating current (AC)), a rechargeable system, a power failure detection circuit, a converter or inverter. energy, an indicator of the energy status (for example, a light emitting diode (LED)) and any other component associated with the generation, management and distribution of energy in portable devices.

In some embodiments, the software components include an operating system 132, a communication module 134 (or instruction set), a contact / motion module 138 (or instruction set), a module 140 (or instruction set) graphic, a module 144 (or instruction set) of the status of the user interface and one or more applications 146 (or instruction set).

The operating system 132 (e.g., Darwin, RTXC, LINUX, UNIX, OS X, WINDOWS, or an embedded operating system), includes different software components and / or controllers to control and manage general system tasks (e.g., memory management, control of storage devices, energy management, etc.) and facilitates communication between the different hardware and software components.

The communication module 134 facilitates communication with other devices through one or more external ports 148 and also includes different software components to manage the data received by the RF circuit system 112 and / or the external port 148. The external port 148 (for example, a universal serial bus (USB), FIREWIRE, etc.) is adapted to connect directly to other devices or indirectly through a network (for example, Internet, wireless LANs, etc.).

The contact / motion module 138 detects contact with the touch screen 126, together with the touch screen controller 122. The contact / motion module 138 includes different software components to perform different operations related to the detection of the contact with the touch screen 126, such as determining if the contact has occurred, determining if there is movement of the contact and tracking the movement through the touch screen, and determine if the contact has been interrupted (that is, if the contact has ceased). The determination of the movement of the contact point may include determining the speed (magnitude), the speed (magnitude and direction) and / or the acceleration (including magnitude and / or direction) of the contact point. In some embodiments, the touch / motion module 126 and the touch screen controller 122 also detect the contact on the touch pad.

The graphic module 140 includes different known software components for displaying and displaying graphics on the touch screen 126. Note that the term "graphics" includes any object that can be displayed to a user including, but not limited to, text, Web pages, icons (such as user interface objects that include programmable keys), digital images, videos, animations and the like

In some embodiments, the graphic module 140 includes an optical intensity module 142. The optical intensity module 142 controls the optical intensity of graphic objects, such as user interface objects, shown on the touch screen 126. The control of the optical intensity may include the increase or decrease of the optical intensity of a graphic object. In some embodiments, the increase or decrease may follow predetermined functions.

The user interface status module 144 controls the status of the user interface of the device 100. The user interface status module 144 may include a lock module 150 and an unlock module 152. The blocking module detects the satisfaction of any one or more conditions for transitioning the device 100 to a locked state of the user interface and for making the transition of the device 100 to the locked state. The unlocking module detects the satisfaction of any one or more conditions for transitioning the device to an unlocked state of the user interface and for transitioning the device 100 to the unlocked state.

The application or applications 130 may (n) include any application installed on device 100, including, but not limited to, a browser, address book, contact list, email, instant messaging, word processing, keyboard emulations, graphic objects, JAVA applications, encryption, digital rights management, voice recognition, voice replication, ability to determine position (such as that provided by the global positioning system (GPS)), a music player (which plays recorded music stored in one or more files , such as MP3 or AAC files), etc.

In some embodiments, the device 100 may include one or more optional optical sensors (not shown), such as CMOS or CCD 200 image sensors, for use in imaging applications.

However, the indicated hardware structure is one of the possible ones and it should be taken into account that the device 100 may include other image capture elements such as, camera, scanner, laser plotter or the combination of any of these types of devices , which can provide the mobile device with the representation of the real environment in video format, image sequence, vector format or any combination of the aforementioned formats.

Similarly, the device 100 may include geographic location devices based on GPS positioning satellite networks, geographical location aid devices based on GPS satellite networks and IP location of internet networks -AGPS-, devices geographical location based on the triangulation of radio signals provided by WIFI antennas and Bluetooth® devices (ISSP), the combination of any of these devices mentioned or any type of device that allows to provide the mobile device with numerical data of its geographical location. The device 100 can include any type of element capable of representing images in real time with a minimum of 24 FPS (Frames Per Second, images per second) such as, TFT, TFT-LED, TFT-OLED, TFT-Retina screens, the combination of any of the above, in addition to new generation Holo-TFT, transparent and Micro-Projector screens or any graphic representation device that can provide the mobile device 100 with a way of representing visual content to the user.

The device 100 includes a processor or set of processors that by themselves or in combination with graphic processors such as GPU (Graphics Processing Unit) or APU (Accelerated Processing Unit) can provide the mobile device 100 the ability to represent, in real execution time, vector graphics and form textured polygons with them, through vector representation libraries (sets of standardized graphic representation procedures for different platforms) such as, OpenGL, DirectX or any type of libraries intended To this task.

With reference now to Figure 2, a brief, general description of an appropriate computing environment (a personal computer) in which embodiments of the invention can be implemented is briefly described. Although the invention will be described within the general context of program modules that run on server computer systems and mobile devices 100, those skilled in the art will warn that the invention can also be implemented in combination with other types of computer systems and of program modules.

Generally speaking, program modules include routines, programs, components, data structures and other types of structures that perform specific tasks or implement specific abstract data types. The invention can also be carried out in distributed computing environments in which the tasks are carried out by means of remote processing devices that are linked through a communications network. In a distributed computing environment, the program modules can be located both in local devices and in remote memory storage devices.

Referring now to Figure 2, an illustrative computer architecture for a computer 2 used in the various embodiments of the invention will be described. The computer architecture, shown in Figure 2 illustrates a conventional desktop or portable computer and can also be used as a server, which includes a central processing unit 5 (CPU), a system memory 7, which includes a memory of random access 9 (RAM) and a read-only memory (ROM) 11, and a system bus 12 that couples the memory to the CPU 5. A basic input / output system that contains the basic routines that help transfer information between The elements located on the computer, such as during startup, are stored in ROM 11. Computer 2 also includes a mass storage device 14 for storing an operating system 16, application programs and other program modules, which will be described in greater detail in the following lines.

The mass storage device 14 is connected to the CPU 5 via a mass storage controller (not shown) connected to the bus 12. The mass storage device 14 and its associated computer-readable media provide non-volatile storage to the computer 2. Although the description of the computer-readable media contained herein refers to a mass storage device, such as a hard disk or a CD-ROM drive, it should be appreciated, by those skilled in the art, that computer-readable media can be any media that can be accessed by computer 2.

By way of example, and not limitation, computer readable media may comprise computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented in any method or technique for storing information, such as computer readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EPROM, EEPROM, flash memory or other solid-state memory techniques, CD-ROMs, digital versatile disks (DVDs) or other storage devices. optical storage, magnetic cassettes, magnetic tapes, magnetic disk storage devices or other magnetic storage devices or any other means that can be used to store the desired information and that can be accessed by computer 2.

According to various embodiments of the invention, the computer 2 can operate in a networked connection environment using logical connections with remote computers via a network 18, such as the Internet. The computer 2 can connect to the network 18 through an interface unit 20 with the network connected to the bus 12. It should be appreciated that the interface unit 20 with the network can also be used to connect with other types of networks and from distant computer systems. The computer 2 may also include an input / output controller 22 for receiving and processing an input from a plurality of other devices, including a keyboard, or an input tablet of data (not shown in figure 2). Similarly, an input / output controller 22 may provide an output to a display screen, a printer, or other type of output device.

As indicated briefly in the previous lines, a plurality of program modules and data files may be stored in the mass storage device 14 and in the RAM 9 of the computer 2, including an operating system 16 suitable for the control of the operation of the networked personal computer, such as the MICROSOFT CORPORATION® WINDOWS® operating system. The mass storage device 14 and the RAM 9 can also store one or more program modules. In particular, the mass storage device 14 and the RAM 9 can store a Web browser application program 10. As those skilled in the art know, the Web browser application program 10 is operative to request, receive, reproduce and provide interactivity with electronic documents, such as a Web page 24 that has been formatted using HTML. Likewise, the Web browser application program 10 can be operative to execute directives contained in the Web page 24, such as directives using the JAVASCRIPT language of SUN MICROSYSTEMS, INC. According to an embodiment of the invention, the Web browser application program 10 comprises THE INTERNET EXPLORER Web browser application program of MICROSFOT CORPORATION. It should be appreciated, however, that other Web browser application programs from other manufacturers can be used to materialize the various aspects of the present invention, such as the FIREFOX Web browser application of MOZILLA FOUNDATION.

In particular, the Web page 24 can include an HTML and directives which, when represented by the application 10 of the Web browser, provide a representation visual of a program or programs stored in the computer 2. Likewise, the directives included in the Web page 24 make it possible for a user of the computer to interact with the representation provided by the application 10 of the Web browser and modify the application 10.

As can be seen in Figure 3, the system object of the invention comprises a mobile device 100, a computer 2, a laser pointer 300 and a projector 400, such that the projector is connected and showing a slide show on a projection 401.

The user, that is, the presenter of the slides, will subsequently draw any object in the projection 401 with the laser pointer 300 and while the drawings are made, the mobile device 100 is recognizing the points that are being drawn.

Depending on the different configurations, the mobile device 100 sends the recognized points to a server, that is to a computer 2 that receives said points and processes the information to recognize the different shapes drawn, processing said information and:

 a) Processing said information and modifying the presentation; or

 b) Sending said information to the computers (2 ', 2' ') of the listeners of the presentation, where the forms drawn on the file that is being represented in projection 401 will be recorded.

This is the standard behavior of the application, and that is the perfect integration of all systems. The system operates in real time, therefore, connection speed and stability and processing speed play an important role. Each one is described in detail below. of the parts that make the general operation described in Figure 3 possible.

For normal operation of the system, the mobile device (100) comprises a program or programs (132 to 146) in which said programs (132 to 146) are stored in the memory 102 and configured to be executed by means of the processor (s) 106 and where the programs include instructions for:

Configure your communications and operation;

 Calibrate image capture;

 Capture the projection of a laser pointer (300) on a projection 401 which in turn projects images from a projector 400 connected to a computer 2;

 Recognize the images captured in step (c);

 Communicate to a computer 2 the images recognized in step (d).

Communications and operation configuration

The configuration of communications and operation allows the user to have several ways to use the application and choose the one that best suits their real situation. Thus, the user of the mobile device (100) normally the person presenting the slides, can choose whether to use WIFI or Bluetooth. By default, the system understands that a normal user cannot know the network conditions in advance, therefore it is recommended that whenever there is Wi-Fi available to work with the WIFI connection. When the client has trouble connecting to the Internet then the mobile device (100) will use the Bluetooth connection.

In any case, the user tells the mobile device (100) how to connect to the computer (2). The connection can also be, in real time, where the user decides if while painting with the laser pointer 300 the dots are automatically displayed at the same time as the shape is drawn in projection 401. If this option is checked, no option is possible. machine learning class. The user will feel like he is drawing with a pen on the projector 400.

The user has the option to select if he wants the mobile device to have machine learning based on fuzzy logic. With this option checked, the mobile device 100 tries to decipher what is being drawn without errors. That is, if you recognize a circular shape, what you do is correct the imperfection of the freehand drawing, drawing a perfect circle.

Calibration

The recognition of the points that the user draws in real time on the projection 401 is shown in Figure 4. Before reaching a definitive solution, various possibilities were explored, such as trying to use the color of the laser pointer 300 in a RGB scale However, color distortion makes it impossible to use this path. Thus, the best option is to take advantage of the brightness of the laser pointer 300 to recognize the desired points. This process is structured in two phases: calibration and recognition.

The basic data flow consists of: (i) calibrating the camera 200 of the mobile device 100, then remaining in recognition mode until the user finishes on the end button; and (ii) in the recognition phase the points are searched until they are detected and sent to computer 2. It has been indicated that the recognition is done through the recognition of the brightness of the laser pointer 300. The brightness is a relative expression of the intensity of the energy produced by a visible light source. It can be expressed as a total energy value (different for each of the curves in the diagram), or as the amplitude in the wavelength where the intensity is greater (identical for the three curves). In the RGB color model, the amplitudes of red, green and blue for a particular color can each range from 0 to 100 percent of full brightness. The range of decimal numbers from 0 to 255, or hexadecimal numbers from 00 to FF represents these levels.

To be able to recognize the brightness of a laser point, which is normally higher than any other value, the mobile device (100) must first analyze the scene and find out which points are the brightest. Subsequently, in order to scale the correctly located points, the mobile device 100 has to know which are the four corners of the projection 401.

All this is achieved in this calibration stage as explained below: i. The first calibration looks for the brightest point in the scene. This is called the brightness barrier.

 ii. In a second calibration, the four corners of the projection in the projection 401 are searched to be able to scale the points correctly.

The flowchart of the calibration stage is shown in Figure 5. As indicated, if the brightness barrier exceeds 255 then an error message will be thrown.

As indicated above, in the first calibration you Look for the brightest point in the scene and establish a barrier brightness for the future recognition of the points. The recognition process does not proceed until a valid barrier brightness value is found. The calibration process analyzes the brightness value in each pixel.

When the actual brightness point is higher than the saved value, the maximum brightness value is updated and the index position is saved. The received pixel matrix has each position stored in order from the original frame, first position (row 0 (x0), column 0 (y0)) until the last position (row width, column height). That is why the size of the matrix is the size of the camera (width * height). This value is necessary to normalize indexed position data. Only the second phase of the calibration can begin with the brightest spot correctly located and normalized.

The process ends when the end of the matrix is reached and the maximum brightness has been obtained. At this time the first phase of the calibration is over and is not repeated until the entire recognition process is over.

One of the problems that appear in the first phase of the calibration is when there is a very bright spot in the projection 401 that is even greater than the brightness of the laser pointer 300, which makes it impossible to recognize. Therefore, it is necessary for the calibration to continue even if there is more than one excessively bright pixel. In this case, the mobile device 100 basically looks for points where the brightness exceeds a value of 256 (the barrier for the laser pointer 300) and discards those points as a brightness barrier and, therefore, they are not stored in the matrix. This correction method comprises the following stages: i. Obtain the points that exceed the brightness barrier and store them in a second matrix.

 ii. Apply an identification algorithm to the second matrix to group the points together into small pieces iii. Create an avoidable small rectangle in the second phase of the calibration.

The identification algorithm (type DBSCA http: // in. Wikipedia. Org / wiki / DBSCi¾N) can identify groups or clusters in large spatial data sets by looking at the local density of the elements of a database, using only one parameter of entry. In addition, the user gets a suggestion about which parameter value would be appropriate. Therefore, a minimum knowledge of the domain is required. The algorithm can also determine what information should be classified as noise or extreme values. Despite this, its work process is fast and scales very well with the size of the database - almost linearly. The idea is to group the different points that exceed the brightness barrier and group them into rectangles. This rectangle will be used later in the second phase of the calibration and in the recognition phase.

The second phase of the calibration consists of the division of any scene into several rectangles of equal size, i.e. the same amount of pixels, where only the corners have a differentiated color, while the rest of the scene is white. This scene, logically, is provided by computer 2, which projects this image on projection 401. The average RGB value of the created rectangle is then computed. The algorithm subsequently obtains the RGB value of each rectangle and all average values are subtracted from each other using the absolute value function.

All differences are then stored in a matrix with their original position. This matrix is stored using the QuickSort algorithm (http: // 'in. wikipedia. org / wiki / Quicksort) since it is the fastest solution.

After the calibration is complete, the four located corners are initialized and saved for the recognition session. Since it is presumed that the mobile device (100) is in a stationary position, the calibration parameters will not change during the entire presentation as long as it remains within the projection rectangle that will coincide in size with the rectangle described in the second phase of Calibration

Point Recognition

The point recognition process begins after the end of the calibration phase. As stated before, when the second calibration process ends, a rectangle will appear with four corners. As of now, no point outside this will be recognized four corners. This is achieved during pixel analysis with the following method.

The parameters (xl, yl) (x2, y2) (x3, y3) (x4, y) correspond to the 4 corners. The method receives the position of the matrix and transforms it into (x, y) real values. If the point is inside, the function returns "true", if not "false".

As described above, if a point is too bright during the calibration process, that point is added to the blind spot matrix. Again, during the recognition process, those points will be ignored and nothing can be extracted at those points. This may sound very bad, but in reality the test shows that the blind spots are minimal, no more than three per scene, therefore, in almost two million pixels (1080 * 1440) that does not affect much.

Blind spots are ignored when they are outside the four corners of the projector screen 401, this saves the algorithm a lot of processing time.

If the recognized point is neither a blind spot nor an outside point, the point is accepted as valid and stored in an array of accepted points. There are two modes of operation: real-time recognition and recognition with fuzzy logic.

If the recognition is in real time, whenever the application recognizes a point, it is immediately sent to computer 2. In order to track the path, the previously recognized point is passed. With this procedure the application avoids inconsistency in the drawings, for example, if the point between the drawings was not recognized for whatever reason. The user preferences vector saves the setting option. In order to track each time the application recognizes a point, the previous one is saved and then the matrix of recognized points is deleted. After additions and deletions, points are sent.

If the fuzzy logic mode is selected, the application maintains point recognition until the user finishes drawing, in other words, until the laser pointer is no longer active and the application does not recognize any points. At that time, the application sends the matrix of points that have been recognized during the drawing period. To protect the algorithm from malfunction, at least three frames were analyzed in a way that has to be empty = no recognized point. This prevents against momentary malfunction or prevents the user while said user is painting. After sending the points, the dot matrix is deleted and the application waits for future drawings. The recognition algorithm remains active until the user presses the Finish button.

Data submission

Another vital part of the application is the connection between the mobile device 100 and the computer 2. TCP is a communication protocol, currently documented in IETF RFC 793. It is a transport layer protocol according to the OSI model. In addition, the necessary thing is added to provide a service that allows communication between two systems functions without errors, without losses and security. TCP was chosen for an obvious reason:

 The application has to follow a drawing order, therefore, the order of arrival of the packages is important.

It has to be reliable, packages cannot be lost.

Finally, it has to be easy to use.

The parameters were inserted manually. Two parameters have to be chosen to establish a successful socket connection, the port and the IP port is a construction of specific application software or specific process that serves as an endpoint of communications in the host operating system of a computer. The purpose of the ports is to uniquely identify the different applications or processes that run on a single computer and therefore that can share a single physical connection to a packet switching network such as the Internet. The port number had to be available, that is why port 8063 was chosen. An IP address is a numerical tag that identifies in a logical and hierarchical way, an interface (communication / connection element) of a device (usually a computer) within a network using (Internet Protocol) IP protocol, which corresponds to the OSI model network layer. In this case, the IP that was entered was the IP of computer 2.

The invention comprises a dedicated server for hosting a service or services for users (such as a computer 2). Hosting servers are the most used to host websites, but can also be used to host files, images, games and similar content. Hosting servers can be shared among many clients (shared hosting servers) or dedicated to a single client (dedicated servers), the latter of which is particularly common for larger Web sites, where the hosting needs of the owner of the Website require more control and / or bandwidth.

The basic idea is to use the same client / server connections as before, but now with a final static IP address. In this way the user never enters a new IP. The connection structure is as follows:

A listening server for mobile phone connections, this server is called A. This server is also responsible for the fuzzy logic intensive processing algorithm.

 Another server listens to all clients who want to receive all the information of the mobile device 100 and the modifications, this server would be called B.

In this new model, the mobile device 100 eliminates the differentiation between the client connected to the projector 400 and the one that is not. This solution can be extended with more servers when the system reaches a certain number of clients. Another positive aspect is that if WI-FI is available, the user can use roaming / 3g / 4g and will not have any problems while using the system. This solution meets all the requirements presented above. It is a portable solution, has minimal interaction with the user and is fast. It only presents problems when there is a huge overhead in the WIFI network. The scalability will depend on the number of ports and servers.

The computer (2)

The program 26 stored in the memory 14 of the computer 2 is responsible for handling all the information received by the mobile device 100. It is divided into three parts:

Connections and analysis.

 Fuzzy Logic and the recognition of forms.

 Generation of drawings on the screen.

In computer technology, an analysis program is a program, usually part of a compiler, that receives input in the form of sequential program code instructions, interactive online commands, dial tags, or some other defined interface and it breaks them into parts (for example, nouns (objects), verbs (methods), and their attributes or options) that can then be managed by other programs (for example, other components in a compiler).

In the invention, the analysis is performed at each stage, the mobile application, the processing and the drawing. It normally reconstructs the information in such a way that it facilitates the reception process by the computer 2 and also transforms the information into the format required by the program 26.

Actually, there are three phases of analysis: Phase 1 of analysis receives a matrix of points and is transformed into a JSONObject (JavaScript Object Notation) to make the process easier, it is transformed into a chain, so the program uses BufferReaders

 Phase 2 is also very important, since the information is prepared for the recognition stage. Read all the points in the form of a chain and transform them into a cloud of points in the form of a matrix.

 Convert the points of the different shapes into an ordered object.

This ordered object is the one that is analyzed by diffuse logic, locating the geometric shapes that have been drawn and incorporating them into the presentation. The recognition method is based on two basic ideas:

First, it uses entirely global geometric properties extracted from the input forms. Since we are primarily interested in the identification of geometric entities, the recognizer relies primarily on geometry information.

 Second, the algorithm uses fuzzy logic to associate degrees of certainty with recognized forms and manage ambiguities naturally.

This algorithm recognizes elementary geometric shapes such as the triangle, the rectangle and circles or lines as well as simple commands to operate, for example, a PowerPoint® presentation such as moving from one slide to another or escaping the presentation.

Once the algorithm has recognized that it is not a simple order to operate the projection, the process of classifying the points is initiated by various geometric operations. The algorithm uses geometric shapes like the Convex Hull (http: //en.wikipedia. Org / iki / Convex hull) created with a Graham algorithm

(http: // en. wikipedia ■ org / wiki / Graham sean), or any other well known geometric method for an expert

(http: / / en. wikipedi. org / wiki / Minimum bounding box algorithms).

Once the form is recognized, it will only be necessary to draw it on the screen.

In summary, the program 26 in the computer 2 comprises a user interface in which, basically, the connection is started and a kind of chess board is shown for the calibration of the mobile device 100 as described previously. Once calibrated, the invisible panel is opened where it is painted, which also has those arrows on the sides so that the user knows where he has to aim to pass slides, and a button to exit the application.

In this part of the application, the modes in which the user wants to work are received (he will select them on the mobile), which can be in real time (as the points are painted), or learning (he will cumulate points , and when the user stops painting, if some form has been recognized, it is painted). And both modes together, in which you paint in real time, but if when the user stops painting a figure is recognized, it is corrected (painting correctly).

In addition, the orders made with the laser pointer 300 and recognized by the camera 200 of the mobile device 100 are received, and carried out. These orders are at least:

Paint figures, or paint in real time. This painting is done on an "invisible" panel.

 Move from slides to the next and the previous one.

Erase everything painted. Specifically delete a figure. With the laser pointer 300, a zigzag shape will be made as crossing out the figure to be erased, this is recognized by the mobile and is passed to this part of program 26, which erases the figure on which that zigzag has been made. To do this, the midpoint of the zigzag drawing is calculated, and compared to the midpoints of all the figures that are painted at that time, the one that is closest will be the one that is erased.

 Union of the part of recognition of forms and painted in real time.

 Send to those who are connected to the application the same thing that is being projected, so that they can follow the presentation from their computer.

Automatic calibration

In addition to the manual calibration described above, in a second embodiment an automatic calibration is described. This requires the correct interaction of the mobile device 100 and the program 26 of the computer 2. The objective is a correct transformation of the points so that the orientation and the parameters of the camera do not affect the result. To achieve this purpose the steps are as follows: i. Mobile device 100 sends an image to computer 2, which is a photo of the current scene. The image is converted to jpg and encapsulated in Base64

ii. Computer 2 receives the image and analyzes it. The objective is to look for the image in the form of a chess board that has been shown by computer 2 and that basically recognizes the squares inside chess. Once recognized as we know the position of the points in the real world, since we draw the chess board and know its dimensions, we use a method called homography in such a way that Through this method we can transform all points without any error from 2D to 3D. From this moment all the points received will be transformed, (http: / / en. Wikipedia. Org / wiki / Hoinography (compute visio

iii. Once the final results are finished, they are sent to the mobile device 100 to lighten the processing and only search within the area where the aj edrez has been found.

Finally, through some parameters that are updated every time the point is recognized, the search space becomes smaller, reducing the computation time.

The square is updated making the process much faster. If no point is found, the rectangle becomes the area of incidence and so on continuously. If the rectangle leaves the area (edges) the parameters are adjusted so that it always stays inside.

In an additional embodiment to those indicated herein, the laser pointer (300) can be generalized to any electronic device that allows focusing a beam of light towards the projection zone (401). Thus, for example, we can think of a pen-like device that incorporates a small flashlight-like light at one end and that would allow to develop the same functionalities as those described for the laser pointer (300).

Claims

1 - Method implemented by computer to interact with a laser pointer (300) by means of a portable electronic device (100) on a projection area (401) defined by a projector (400) connected to a computer (2) and characterized by which comprises the stages of:

 a) Calibrate the image capture of a camera (200) of a portable electronic device (100) by recognizing points of a standard image;

 b) Capture a plurality of points of light emitted by the laser pointer (300) where

 to. Said points of light are within the projection area (401);

 b. In said projection area (401) there is a virtual reproduction of a computer file stored in a computer memory (2); C. Where said virtual reproduction is emitted by a projector (400); Y

 d. Where the capture is made by the camera (200) of the portable electronic device (100);

 c) Recognize the points of light captured in step (b) and form a matrix of points with their relative positions in the projection area (401);

 d) Send the dot matrix of stage (c) to the computer (2) where they are analyzed;

 e) Recognize the shapes derived from the dot matrix through fuzzy logic; Y

 f) Generate the shapes recognized in the projected computer file on the projection area (401) by means of a projector (400).

2 - A portable electronic device (100), comprising:

a touch screen (126); one or more processors (106);

 a camera (200);

 a memory (102); Y

 one or more programs (132 to 146), in which the program (s) (132 to 146) are stored in memory (102) and configured to be executed by means of the processor (s) (106); characterized in that the programs (132 to 146) include instructions for:

 Configure your communications and operation;

 Calibrate image capture by the camera (200);

 Capture the points of light emitted by a laser pointer (300) within a projection area (401) in which a virtual reproduction of a computer file stored on a computer (2) connected to a projector (400) appears;

 Recognize the points of light emitted by the laser pointer (300) and captured within the projection area (401); and Send to the computer (2) a matrix of points that includes their relative position in the projection area (401), said points being those recognized as points of light emitted by the laser pointer (300).

3 - A computer (2) comprising

 one or more processors (5);

 a memory (14); Y

 one or more programs (26) in which said programs (26) are stored in a memory (14) and configured to be executed by means of the processor (5), characterized in that the programs (26) include instructions for:

 Receiving from the device (100) of claim 2 a dot matrix that includes the relative position within a projection area (401) of the light points emitted by a laser pointer (300) within said projection area (401) ;

Analyze the matrix of points received; Recognize shapes derived from the dot matrix through fuzzy logic; Y

 Generate the shapes recognized in a computer file stored in memory (14) and projected onto a projection area (401) by means of a projector (400).

4 - System for interacting with a laser pointer (300) by means of a portable electronic device (100) on a projection (401) emitted by a projector (400) connected to a computer (2) comprising:

 a portable electronic device (100) according to claim 2; Y

 a computer (2) according to claim 3.

5 - A computer program product with instructions configured for execution by one or more processors which, when executed by a portable electronic device (100) according to claim 2 and a computer (2) according to claim 3 make that the system of claim 4 carry out the method according to claim 1.