CN114727142A

CN114727142A - Display device and collaborative drawing method

Info

Publication number: CN114727142A
Application number: CN202210330352.8A
Authority: CN
Inventors: 董率; 张振宝; 肖媛
Original assignee: Hisense Visual Technology Co Ltd
Current assignee: Hisense Visual Technology Co Ltd
Priority date: 2022-03-30
Filing date: 2022-03-30
Publication date: 2022-07-08
Anticipated expiration: 2042-03-30

Abstract

The application provides a display device and a collaborative drawing method, and when a user inputs a control instruction indicating that a historical drawing operation is cancelled to a first display device, an input device for inputting the historical drawing operation is detected in response to the control instruction; if the input device is a second display device which performs collaborative drawing with the first display device, sending a revocation request to the second display device to request the second display device to feed back first feedback information which is used for indicating that revocation is granted; and in response to receiving the first feedback information, refreshing the electronic drawing board interface, and sending a synchronization instruction to the server to instruct the server to send a synchronization request to each display device participating in collaborative drawing so as to refresh the electronic drawing board interface of each second display device. The problem that the user can cancel the content input by other display equipment at will when canceling operation is carried out is solved, and the user experience is facilitated.

Description

Display device and collaborative drawing method

Technical Field

The application relates to the technical field of smart television drawing boards, in particular to a display device and a collaborative drawing method.

Background

The display device refers to a terminal device capable of outputting a specific display picture, such as a smart television, a mobile terminal, a smart advertisement screen, a projector, and the like. Taking an intelligent television as an example, the intelligent television can be based on an Internet application technology, has an open operating system and a chip, has an open application platform, can realize a bidirectional man-machine interaction function, integrates multiple functions of audio and video, entertainment, education, data and the like, and is a television product for meeting diversified and personalized requirements of users. For example, a drawing board application software can be installed on the smart television for drawing by the user.

In the related art, a plurality of display devices may implement information sharing through network connection, so that a plurality of users may perform collaborative drawing through different display devices to display the same drawing content in each display device, and when one of the users edits a drawing, the display interfaces of other users may also display the content edited by the user.

However, in the above scenario, when each display device performs the undo operation, the content edited by the last drawing may be undone directly, without distinguishing which user edited the content, that is, the target user may undo the content edited by another user at will, and the interaction experience of another user is affected.

Disclosure of Invention

The application provides a display device and a collaborative drawing method, which are used for solving the problem that contents input by other display devices can be randomly cancelled when a user performs cancelling operation, and are beneficial to user experience.

In a first aspect, the present application provides a display device comprising:

a display configured to display an electronic palette interface, the electronic palette interface including a control area and a drawing area, the control area including at least one control for performing an input operation on the drawing area, the drawing area being used to present input content;

a communicator configured to establish a communication connection with a server so that the display device draws as a first display device in cooperation with at least one second display device connected to the server;

a controller configured to:

acquiring a control instruction which is input by a user and indicates to cancel historical drawing operation;

detecting an input device for inputting the historical drawing operation in response to the control instruction;

if the input device is the second display device, sending an undo request to the input device, wherein the undo request comprises a query instruction for undoing the historical drawing operation, and the undo request is used for requesting the second display device to feed back first feedback information indicating the undoing agreement after the second display device agrees to undoing the historical drawing operation;

responding to the received first feedback information, and refreshing the electronic drawing board interface;

and sending a synchronization instruction to the server, wherein the synchronization instruction is used for instructing the server to send a synchronization request to each second display device so as to refresh the electronic drawing board interface of each second display device.

In a second aspect, the present application also provides a display device, including:

a display configured to display an electronic drawing board interface, the electronic drawing board interface including a control area and a drawing area, the control area including at least one control for performing an input operation on the drawing area, the drawing area being used to present input content;

a communicator configured to establish a communication connection with a server so that the display apparatus draws as a second display apparatus in cooperation with a first display apparatus connected to the server;

a controller configured to:

receiving a revocation request sent by the first display device, wherein the revocation request comprises an inquiry instruction for revoking historical drawing operation, and the revocation request is used for requesting the second display device to feed back first feedback information for indicating the revocation after the revocation of the historical drawing operation is agreed;

in response to an instruction for approving the cancellation request input by a user, sending first feedback information to the first display device, so that the first display device refreshes an electronic drawing board interface of the first display device when receiving the first feedback information, so as to cancel the historical drawing operation displayed by the first display device, and sending a synchronization instruction to the server, wherein the synchronization instruction is used for instructing the server to send a synchronization request to the second display device;

receiving a synchronization request sent by the server;

and refreshing the electronic drawing board interface.

In a third aspect, the present application provides a collaborative drawing method applied to a first display device, including:

establishing communication connection with a server so that the first display device and at least one second display device connected with the server draw in a cooperation manner;

According to the technical scheme, when a user inputs a control instruction for indicating that the historical drawing operation is cancelled to the first display device, the input device for inputting the historical drawing operation is detected in response to the control instruction; if the input device is a second display device which performs collaborative drawing with the first display device, sending a revocation request to the second display device to request the second display device to feed back first feedback information which is used for indicating that revocation is granted; and in response to receiving the first feedback information, refreshing the electronic drawing board interface, and sending a synchronization instruction to the server to instruct the server to send a synchronization request to each display device participating in collaborative drawing so as to refresh the electronic drawing board interface of each second display device. The problem that the user can cancel the content input by other display equipment at will when canceling operation is carried out is solved, and the user experience is facilitated.

Drawings

In order to more clearly explain the technical solution of the present application, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious to those skilled in the art that other drawings can be obtained according to the drawings without creative efforts.

FIG. 1 is a usage scenario of a display device in some embodiments;

fig. 2 is a block diagram of a hardware configuration of a display device in some embodiments;

FIG. 3 is a diagram of software configuration in a display device in some embodiments;

FIG. 4 is a diagram of a user interface displayed by a display device in some embodiments;

FIG. 5 is a schematic diagram of an application list in some embodiments;

fig. 6 is a schematic diagram of an electronic palette in some embodiments;

FIG. 7 is an invitation interface diagram in some embodiments;

FIG. 8 is a diagram of a collaborative drawing scenario in some embodiments;

FIG. 9 is a diagram of information interaction in performing undo operations in some embodiments;

FIG. 10 is a schematic diagram of an operational chain in some embodiments;

fig. 11 is a schematic diagram of an electronic palette interface in some embodiments;

fig. 12 is a schematic diagram of an electronic palette interface in some embodiments;

fig. 13 is a schematic diagram of an electronic palette interface in some embodiments;

fig. 14 is a schematic diagram of an electronic palette interface in some embodiments;

FIG. 15 is an interface diagram of a display device participating in collaborative drawing before undo operations in some embodiments;

FIG. 16 is an interface diagram of a display device participating in collaborative drawing after undo operations in some embodiments;

FIG. 17 is a diagram of information interaction in performing a undo operation in one embodiment;

FIG. 18 is a diagram of information interaction in performing a undo operation in one embodiment;

FIG. 19 is a diagram of information interaction in performing a undo operation in one embodiment;

FIG. 20 is an information interaction diagram that illustrates a resume historical drawing operation in some embodiments;

FIG. 21 is an interface diagram of a display device participating in collaborative drawing before a restore operation in some embodiments;

FIG. 22 is an interface diagram of a display device participating in collaborative drawing after a restore operation in some embodiments;

FIG. 23 is a flow diagram of a collaborative drawing method in some embodiments;

FIG. 24 is a flow diagram of a collaborative drawing method in some embodiments.

Detailed Description

Reference will now be made in detail to embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following examples do not represent all embodiments consistent with the present application. But merely as exemplifications of systems and methods consistent with certain aspects of the application, as recited in the claims.

To make the purpose and embodiments of the present application clearer, the following will clearly and completely describe the exemplary embodiments of the present application with reference to the attached drawings in the exemplary embodiments of the present application, and it is obvious that the described exemplary embodiments are only a part of the embodiments of the present application, and not all of the embodiments.

It should be noted that the brief descriptions of the terms in the present application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of the present application. These terms should be understood in their ordinary and customary meaning unless otherwise indicated.

The terms "first," "second," "third," and the like in the description and claims of this application and in the above-described drawings are used for distinguishing between similar or analogous objects or entities and not necessarily for describing a particular sequential or chronological order, unless otherwise indicated. It is to be understood that the terms so used are interchangeable under appropriate circumstances.

The terms "comprises" and "comprising," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a product or apparatus that comprises a list of elements is not necessarily limited to all elements expressly listed, but may include other elements not expressly listed or inherent to such product or apparatus.

The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and/or software code that is capable of performing the functionality associated with that element.

Fig. 1 is a schematic diagram of a usage scenario of a display device according to an embodiment. As shown in fig. 1, the display apparatus 200 is also in data communication with a server 400, and a user can operate the display apparatus 200 through the smart device 300 or the control device 100.

In some embodiments, the control apparatus 100 may be a remote controller, and the communication between the remote controller and the display device 200 includes at least one of an infrared protocol communication or a bluetooth protocol communication, and other short-distance communication methods, and the display device 200 is controlled by a wireless or wired method. The user may control the display apparatus 200 by inputting a user instruction through at least one of a key on a remote controller, a voice input, a control panel input, and the like.

In some embodiments, the smart device 300 may also be used to control the display device 200. For example, the display device 200 is controlled using a camera application running on the smart device.

In some embodiments, the smart device 300 and the display device 200 may also be used for communication of data.

In some embodiments, the display device 200 may also be controlled in a manner other than the control apparatus 100 and the smart device 300, for example, the voice instruction control of the user may be directly received by a module configured inside the display device 200 to obtain a voice instruction, or may be received by a voice control apparatus provided outside the display device 200.

In some embodiments, the display device 200 is also in data communication with a server 400. The display device 200 may be allowed to be communicatively connected through a Local Area Network (LAN), a Wireless Local Area Network (WLAN), and other networks. The server 400 may provide various contents and interactions to the display apparatus 200. The server 400 may be a cluster or a plurality of clusters, and may include one or more types of servers.

Fig. 2 illustrates a hardware configuration block diagram of a display device according to an exemplary embodiment.

In some embodiments, the display apparatus 200 includes at least one of a tuner 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface.

In some embodiments the controller comprises a central processor, a video processor, an audio processor, a graphics processor, a RAM, a ROM, a first interface to an nth interface for input/output.

In some embodiments, the display 260 includes a display screen component for displaying pictures, and a driving component for driving image display, a component for receiving image signals from the controller output, displaying video content, image content, and menu manipulation interface, and a user manipulation UI interface, etc.

In some embodiments, the display 260 may be at least one of a liquid crystal display, an OLED display, and a projection display, and may also be a projection device and a projection screen.

In some embodiments, communicator 220 is a component for communicating with external devices or servers according to various communication protocol types. For example: the communicator may include at least one of a Wifi module, a bluetooth module, a wired ethernet module, and other network communication protocol chips or near field communication protocol chips, and an infrared receiver. The display apparatus 200 may establish transmission and reception of control signals and data signals with the control device 100 or the server 400 through the communicator 220.

In some embodiments, the detector 230 is used to collect signals of the external environment or interaction with the outside. For example, detector 230 includes a light receiver, a sensor for collecting ambient light intensity; alternatively, the detector 230 includes an image collector, such as a camera, which can be used to collect external environment scenes, attributes of the user, or user interaction gestures, or the detector 230 includes a sound collector, such as a microphone, which is used to receive external sounds.

In some embodiments, the external device interface 240 may include, but is not limited to, the following: high Definition Multimedia Interface (HDMI), analog or data high definition component input interface (component), composite video input interface (CVBS), USB input interface (USB), RGB port, and the like. The interface may be a composite input/output interface formed by the plurality of interfaces.

In some embodiments, the controller 250 controls the operation of the display device 200 and responds to user operations through various software control programs stored in the memory. The controller 250 controls the overall operation of the display apparatus 200. For example: in response to receiving a user command for selecting a UI object displayed on the display 260, the controller 250 may perform an operation related to the object selected by the user command.

In some embodiments, the object may be any one of selectable objects, such as a hyperlink, an icon, or other actionable control. The operations related to the selected object are: displaying an operation connected to a hyperlink page, document, image, or the like, or performing an operation of a program corresponding to the icon.

In some embodiments the controller comprises at least one of a Central Processing Unit (CPU), a video processor, an audio processor, a Graphics Processing Unit (GPU), a RAM Random Access Memory (RAM), a ROM (Read-Only Memory), a first to nth interface for input/output, a communication Bus (Bus), and the like.

A CPU processor. The system is used for executing the operating system and the camera application instructions stored in the memory and executing various camera applications, data and contents according to various interaction instructions received from the outside so as to finally display and play various audio and video contents. The CPU processor may include a plurality of processors. E.g. comprising a main processor and one or more sub-processors.

In some embodiments, a graphics processor for generating various graphical objects, such as: at least one of an icon, an operation menu, and a user input instruction display figure. The graphic processor comprises an arithmetic unit, which performs operation by receiving various interactive instructions input by a user and displays various objects according to display attributes; the system also comprises a renderer for rendering various objects obtained based on the arithmetic unit, wherein the rendered objects are used for being displayed on a display.

In some embodiments, a user may enter user commands on a Graphical User Interface (GUI) displayed on display 260, and the user input interface receives the user input commands through the Graphical User Interface (GUI). Alternatively, the user may input the user command by inputting a specific sound or gesture, and the user input interface receives the user input command by recognizing the sound or gesture through the sensor.

In some embodiments, a "user interface" is a media interface for interaction and information exchange between a camera application or operating system and a user that enables conversion between an internal form of information and a user-acceptable form. A commonly used presentation form of the User Interface is a Graphical User Interface (GUI), which refers to a User Interface related to computer operations and displayed in a graphical manner. It may be an interface element such as an icon, a window, a control, etc. displayed in the display screen of the electronic device, where the control may include at least one of an icon, a button, a menu, a tab, a text box, a dialog box, a status bar, a navigation bar, a Widget, etc. visual interface elements.

In some embodiments, user interface 280 is an interface that may be used to receive control inputs (e.g., physical buttons on the body of display device 200, or the like).

In some embodiments, the system of display device 200 may include a Kernel (Kernel), a command parser (shell), a file system, and a camera application. The kernel, shell, and file system together make up the basic operating system structure that allows users to manage files, run programs, and use the system. After power-on, the kernel starts, activates kernel space, abstracts hardware, initializes hardware parameters, etc., runs and maintains virtual memory, scheduler, signals and inter-process communication (IPC). And after the kernel is started, loading the Shell and the user camera application. The camera application is compiled into machine code after being started, and a process is formed.

Referring to fig. 3, in some embodiments, the system is divided into four layers, which are, from top to bottom, a camera Application (Applications) layer (abbreviated as "Application layer"), a camera Application Framework (Application Framework) layer (abbreviated as "Framework layer"), an Android runtime (Android runtime) and system library layer (abbreviated as "system runtime library layer"), and a kernel layer.

In some embodiments, at least one camera application runs in the camera application layer, and the camera applications may be a Window (Window) program of an operating system, a system setting program, a clock program, or the like; or a camera application developed by a third party developer. In specific implementation, the camera application package in the camera application layer is not limited to the above example.

The framework layer provides an Application Programming Interface (API) and a programming framework for the camera application of the camera application layer. The camera application framework layer includes some predefined functions. The camera application framework layer acts as a processing center that decides to let the camera applications in the application layer act. The camera application can access resources in the system and obtain services of the system in execution through the API interface.

As shown in fig. 3, in the embodiment of the present application, the camera application framework layer includes a manager (Managers), a Content Provider (Content Provider), and the like, where the manager includes at least one of the following modules: an Activity Manager (Activity Manager) is used for interacting with all activities running in the system; the Location Manager (Location Manager) is used for providing the system service or application with the access of the system Location service; a Package Manager (Package Manager) for retrieving various information related to the camera application Package currently installed on the device; a Notification Manager (Notification Manager) for controlling display and clearing of Notification messages; a Window Manager (Window Manager) is used to manage the icons, windows, toolbars, wallpapers, and desktop components on a user interface.

In some embodiments, the activity manager is used to manage the lifecycle of the various camera applications and the usual navigation fallback functions, such as controlling the exit, opening, fallback, etc. of the camera applications. The window manager is used for managing all window programs, such as obtaining the size of a display screen, judging whether a status bar exists, locking the screen, intercepting the screen, controlling the change of the display window (for example, reducing the display window, displaying a shake, displaying a distortion deformation, and the like), and the like.

In some embodiments, the system runtime layer provides support for the upper layer, i.e., the framework layer, and when the framework layer is used, the android operating system runs the C/C + + library included in the system runtime layer to implement the functions to be implemented by the framework layer.

In some embodiments, the kernel layer is a layer between hardware and software. As shown in fig. 3, the core layer comprises at least one of the following drivers: audio drive, display driver, bluetooth drive, camera drive, WIFI drive, USB drive, HDMI drive, sensor drive (like fingerprint sensor, temperature sensor, pressure sensor etc.) and power drive etc..

In some embodiments, the display device 200 may have a touch interaction function, and a user may touch the display 260 with a finger to perform an interaction operation, instead of a keyboard, a mouse, a remote controller, or the like, so that the human-computer interaction is more straightforward. Based on the display device 200, the display device 200 can support a touch interaction function by adding a touch component. In general, the touch-sensing component may constitute a touch screen together with the display 260. A user can input different control instructions on the touch screen through touch operation. For example, the user may input a click, slide, long press, double click, etc. touch command, and different touch commands may represent different control functions.

In order to implement the different control commands, the touch assembly may generate different electrical signals when a user inputs different touch actions, and transmit the generated electrical signals to the controller 250. The controller 250 may perform feature extraction on the received electrical signal to determine a control function to be performed by the user based on the extracted features.

For example, when a user inputs a click touch action at any program icon position in the application program interface, the touch component senses the touch action and generates an electrical signal. After receiving the electrical signal, the controller 250 may first determine a duration of a level corresponding to a touch action in the electrical signal, and when the duration is less than a preset time threshold, recognize that a click touch instruction is input by the user. The controller 250 then extracts the positional features generated by the electrical signals to determine the touch position. And when the touch position is within the display range of the application icon, determining that the user inputs a click touch instruction at the position of the application icon. Accordingly, the click touch command is used to execute a function of running a corresponding application program in the current scene, so that the controller 250 may start running the corresponding application program.

For another example, when the user inputs a sliding motion in the media asset presentation page, the touch component also sends the sensed electrical signal to the controller 250. The controller 250 first determines the duration of the signal corresponding to the touch action in the electrical signal. When the determined duration is longer than the preset time threshold, the position change condition generated by the signal is judged, and obviously, for the interactive touch action, the generation position of the signal changes, so that the sliding touch instruction input by the user is determined. The controller 250 determines the sliding direction of the sliding touch instruction according to the change condition of the position of the signal generation, and controls to turn pages of the display frame in the media asset display page so as to display more media asset options. Further, the controller 250 may extract features such as a sliding speed and a sliding distance of the sliding touch instruction, and perform a page-turning screen control according to the extracted features, so as to achieve a hand-following effect.

Similarly, for the touch instruction such as double click, long press, etc., the controller 250 may execute the corresponding control function according to the preset interaction rule by extracting different features and determining the type of the touch instruction through feature judgment. In some embodiments, the touch component also supports multi-touch, such that a user can input touch actions on the touch screen through multiple fingers, e.g., multi-finger clicks, multi-finger long presses, multi-finger swipes, and the like.

The touch control action can be matched with a specific application program to realize a specific function. For example, after the user opens the drawing board application, the display 260 may present a drawing area, the user may draw a specific touch action track in the drawing area through a sliding touch instruction, and the controller 250 determines a touch action pattern through a touch action detected by the touch component and controls the display 260 to display in real time to satisfy the demonstration effect.

In some embodiments, multiple applications may be installed in the display device 200, and fig. 4 shows a user interface displayed by the display in some embodiments, and a user may click "my applications" in the user interface to trigger the display of the application list. All applications installed by the display apparatus 200 are included in the application list.

FIG. 5 illustrates a schematic diagram of an application list in some embodiments. As shown in fig. 6. The display device 200 may be installed with a drawing board application, a player application, a video chat application, a camera application, and a mirror application. When the application list is displayed in the display, a user may select one of the applications and open the application to trigger the interface displaying the application, for example, the user may select to open the drawing board application, trigger the display to display the interface corresponding to the drawing board application, and perform drawing operation in the interface corresponding to the drawing board application to display corresponding content in the interface.

In some embodiments, when a user launches a sketchpad application, the controller may control the electronic sketchpad to be displayed in the display. An interactive area corresponding to one or more functions of the sketchpad application is displayed on the electronic sketchpad interface, and the interactive area can display text, images, icons, button buttons, pull-down menus, check boxes, selectable lists and the like. The user can make contact with the touch screen at a position where interaction is needed, so as to interact with the interaction area. The display apparatus 200 detects the contact and responds to the detected contact by performing a corresponding operation.

In some embodiments, fig. 6 is an exemplary electronic palette interface, as shown in fig. 6, which includes a drawing area 610 and a control area 620, wherein the drawing area 610 is an area in which content can be input. The control area 620 may display entry corresponding to one or more functions of the drawing board application, including a brush item 621, an eraser item 622, a delete item 623, a screen recording item 624, a collaborative drawing item 625, a restore item 626, a cancel item 627, and the like, and a user may click on each item to trigger the display device 200 to display a corresponding item interface, and may also set parameters of some items on the item interface. For example, a user can click a painting brush item, a display is triggered to display a painting brush toolbar, items such as a painting brush type, a painting brush color, a line type and a line thickness can be selected based on the painting brush toolbar, after the selection is completed, the painting brush tool is in a picking state, and the user can control the painting brush tool to perform corresponding painting operation in a painting area.

In some embodiments, when the user clicks the collaborative drawing item 625, the display device 200 may display an invitation interface as shown in fig. 7 in response to a click action input by the user, a list of users to be invited is displayed in the invitation interface, the list of users to be invited includes a plurality of user ID/device ID items that can invite collaborative drawing, and the display device 200, in response to a user selecting a corresponding item, may access other display devices according to the user ID/device ID of the selected item to invite the corresponding display devices to perform collaborative drawing, where collaborative drawing refers to displaying the same drawing content in an electronic picture of each display device participating in collaborative drawing, for example, a display device initiating the collaborative drawing may be used as a first display device, a display device accepting an invitation to collaborate drawing may be used as a second display device, when the first display device and the second display device participate in collaborative drawing together, after a target track is drawn in an electronic drawing board interface of first display equipment, the first display equipment can send the drawing operation of drawing the target track to second display equipment, so that the second display equipment refreshes the electronic drawing board interface of the second display equipment and displays the target track.

In the related art, when the first display device and the second display device participate in collaborative drawing together, each drawing operation input in the electronic drawing board interfaces of the first display device and the second display device is stored in the memory of the first display device according to the input time sequence, after the first display device receives the cancel/restore instruction, the controller reads only the drawing operation of the last step each time, for example, the memory of the first display device stores N operations together, and the cancel/restore operation can be executed only from the nth time. And no matter whether the most further drawing operation is input by the first display device or not, the first display device directly cancels/recovers the drawing operation and sends a cancellation/recovery instruction to the second display device so as to cancel/recover the corresponding drawing operation in the electronic drawing board interface of the second display device, thereby reducing the experience of a user who uses the second display device to perform collaborative drawing.

In addition, if the user wants to cancel a certain drawing operation input before, for example, the user wants to cancel the mth drawing operation, where M is between 1 and N, all drawing operations from M to N must be cancelled in sequence, which is not favorable for the user experience.

Therefore, in order to solve the above technical problem and increase the experience of the user, an embodiment of the present application provides a display device, where the display device establishes a communication connection with a server through a built-in communicator, so that the display device is used as a first display device to draw a picture in cooperation with at least one second display device connected to the server.

It can be understood that the first display device and the second display device in this application may be the same display device, for example, the first display device and the second display device are both smart televisions, and the first display device and the second display device may also be different display devices, for example, the first display device is a smart television, and the second display device is a terminal device such as a mobile phone and a tablet computer. For convenience of description, the controller, the memory, and the communicator of the first display device in the embodiment of the present application are respectively referred to as a first controller, a first memory, and a first communicator, and the electronic palette interface presented by the first display device is referred to as a first electronic palette interface; the controller, the memory and the communicator of the second display device are respectively called a second controller, a second memory and a second communicator, and the electronic drawing board interface presented by the second display device is called a second electronic drawing board interface.

Fig. 8 is a scene diagram of a collaborative painting according to an embodiment of the present application, and as shown in fig. 8, a first communicator establishes a communication connection with a server, a second communicator of at least one second display device also establishes a communication connection with the server, and the first display device and each second display device complete information interaction through the communication connections established between the first communicator, the server, and the second communicator.

In some embodiments, the first controller may receive a variety of control instructions input by a user. Each control instruction corresponds to a control function, and part of the control instructions can be used for revoking the content displayed on the first electronic drawing board interface.

As shown in fig. 9, when the user inputs a control instruction to the first controller instructing to cancel the history drawing operation, the first controller may detect an input device that inputs the history drawing operation in response to the control instruction.

The historical drawing operation is a drawing operation input in the first electronic drawing board interface or the second electronic drawing board interface in advance, and when a user inputs the drawing operation in the first electronic drawing board interface, the first controller responds to the drawing operation input in the first electronic drawing board interface by the user and stores the drawing operation in the first memory. The drawing operation input by the user on the first electronic drawing board interface refers to an event that the user changes the content in the first electronic drawing board interface through operation, for example, the user picks up a drawing tool, and a touch track left on the first electronic drawing board interface can be regarded as a drawing operation; the user picks up the eraser tool, and the erasing of part of the content displayed in the first electronic drawing board interface can be regarded as a drawing operation; and clicking the revocation item by the user to revoke the content displayed on the first electronic drawing board interface by a certain input operation of the user can be regarded as a drawing operation. And the first memory stores the drawing operation input by the user on the first electronic drawing board interface, and then sends a synchronization instruction to the server, wherein the synchronization instruction is used for instructing the server to send the drawing operation input by the user on the first electronic drawing board interface to each second display device.

And after the second display device receives the drawing operation input by the user on the first electronic drawing board interface, the second controller controls to store the drawing operation into the second memory, the second electronic drawing board interface is refreshed according to the drawing operation stored in the second memory, and the refreshed second electronic drawing board interface and the first electronic drawing board interface display the same drawing content. Similarly, the second controller may store the drawing operation input by the user on the second electronic drawing board interface into the second memory in response to the drawing operation input by the user on the second electronic drawing board interface, and send a synchronization instruction to the server to instruct the server to send the drawing operation input by the user on the second electronic drawing board interface to the first display device. And after the first display device receives the drawing operation input by the user on the second electronic drawing board interface, the first controller controls to store the drawing operation into the first memory, and refreshes the first electronic drawing board interface according to the drawing operation stored in the first memory so as to display the corresponding content.

The first memory and the second memory are both stored with operation record lists, and the operation record lists are used for storing drawing operations input by users on the first electronic drawing board interface and the second electronic drawing board interface.

When the operation record list stores the drawing operations input by the user on the first electronic drawing board interface and the second electronic drawing board interface, the first controller and the second controller mark the drawing operations according to the device identification of the input device of the drawing operations, and store the drawing operations in the operation record list according to the time sequence input by the drawing operations.

Wherein the operation record list may be an operation chain, the operation chain includes a plurality of nodes, each drawing operation of the user may be stored in one node of the operation chain, as shown in fig. 10, for the first display device, after the user inputs the drawing operation a in the first electronic drawing board interface, the first controller marks the drawing operation a with the first display device identifier and stores the drawing operation a in the first node of the operation chain, and then, after receiving the drawing operation B input in the second electronic drawing board interface by the user, the first controller marks the drawing operation B with the second display device identifier and stores the drawing operation B in the second node of the operation chain, so that each time the user completes a drawing operation in the first electronic drawing board interface and the second electronic drawing board interface, the first controller marks the drawing operation with the device identifier input the drawing operation, and sequentially storing the marked drawing operations on nodes of the operation chain according to the sequence of the input time, wherein the information stored in each node may include brush type information, brush color information, brush width information, input shape information and the like input by a user, and the application is not limited.

In addition, for the second display device, the manner in which the second controller controls the operation record list storing the drawing operations input in the first electronic drawing board interface and the second electronic drawing board interface into the second memory is the same as the manner in which the first controller controls the drawing operations input in the first electronic drawing board interface and the second electronic drawing board interface to be stored in the first memory, and details are not repeated in this application.

In some embodiments, the first electronic drawing board interface includes a first drawing area and a first control area, where the first drawing area is drawn in a partial or entire area of the first layer, the first control area is drawn in a partial or entire area of the second layer, the first layer and the second layer may be arranged in parallel on the first electronic drawing board interface, the second layer may also be arranged on the upper layer of the first layer in a floating manner, and a user may select the second layer and move the second layer to any position on the upper layer of the first layer by touch-control dragging.

In some embodiments, as further shown in fig. 6, the first control area includes an undo item 627 for indicating an undo operation on the input historical drawing operation, the user clicks the undo item 627 to input an undo instruction on the historical drawing operation to the first controller, the first controller generates a third layer after receiving the undo instruction input by the user, and draws the third layer according to the operation record list stored in the first memory, and the third layer is arranged on the upper layer of the second layer in a floating manner to display relevant options of the operation record list in the first electronic drawing board interface.

The related option of the operation record list may be a thumbnail list displayed in the form of a picture as shown in fig. 11, the thumbnail list includes history drawing operations input by each display device participating in the collaborative drawing and displayed in the form of a thumbnail, and any one of the lists corresponds to one history drawing operation. The content executed by each historical drawing operation is marked or displayed in the thumbnail, for example, when the historical drawing operation input by the user is a line or a graphic, each historical drawing operation input by the user is enclosed in a text frame to prompt the user to know the drawing operation corresponding to the current thumbnail. Alternatively, when the history drawing operation input by the user is to store a filter or fill a color, a thumbnail corresponding to the history drawing operation may be displayed, and a screen in which the filter or the color is stored may be displayed on the thumbnail. In addition, the input device ID/user ID for each history drawing operation is marked on the thumbnail, so that the user can intuitively know the input device for each history drawing operation. The thumbnail list is arranged in the time axis, the two sides of the time axis are respectively provided with the rolling control, and the time axis can be controlled to roll by clicking the rolling control, so that the complete thumbnail list is browsed. After a user selects the historical drawing operation in the operation record list, the first controller responds to the selection operation, obtains the equipment identifier for marking the first target historical event, and detects input equipment for inputting the historical drawing operation according to the equipment identifier.

In some embodiments, after receiving a control instruction indicating to cancel the history drawing operation, which is input by the user, the first controller may display an operation object of the selectable history drawing operation in the first electronic drawing board interface as shown in fig. 12, and after receiving a selection operation of the operation object of the history drawing operation by the user, detect an input device for inputting the history drawing operation in response to the selection operation. After the first controller receives a control instruction which is input by a user and indicates that the historical drawing operation is cancelled, the first controller controls an operation object of the selectable historical drawing operation to be drawn on the first image layer so as to display the operation object of the selectable historical drawing operation in the first electronic drawing board interface, the user can click any operation object and other selection operations, and in response to the selection operations of the user, the first controller controls a selection frame to be drawn on the first image layer so as to prompt the user that the operation object is selected.

In some embodiments, after receiving a control instruction indicating that the historical drawing operation is cancelled, which is input by a user, the first controller may generate a fourth layer, and draw the fourth layer according to the identification of each display device participating in the collaborative drawing, so as to display the user list shown in fig. 13 in the first electronic drawing board interface. After the user selects the identifier of the display device in the user list, the first controller responds to the selection operation, obtaining historical drawing operation input by the selected display equipment in an electronic drawing board interface of the selected display equipment in an operation record list, arranging the historical drawing operation pieces according to the time sequence input to the electronic drawing board interface of the selected display equipment to generate the operation record list corresponding to the selected display equipment, and redrawing a fourth layer according to the operation record list of the selected display equipment, to display the options related to the operation record list as shown in fig. 14 on the first electronic drawing board interface, after the user selects the historical drawing operation in the operation record list, the first controller may directly determine the selected display device as the input device in response to the selection operation.

In some embodiments, if the first controller detects that the input device which inputs the history drawing operation to be revoked is the second display device, a revocation request including a query instruction to revoke a specified history drawing operation to request the input device to feed back first feedback information indicating the revocation approval after the revocation of the history drawing operation is approved is transmitted to the input device. The first display device may directly send a revocation request to the input device through a local area network, bluetooth transmission, or the like, or may send the revocation request to a server and instruct the server to forward the revocation request to the input device.

After the input device receives a revocation request sent by the first display device, the second controller displays a revocation selection page as shown in fig. 15 on the second electronic drawing board interface in response to the received revocation request, wherein the revocation selection page is used for selecting whether to receive the revocation request, the revocation selection confirmation interface comprises a receiving option and a rejecting option, and when the receiving option/rejecting option is selected by the user, the second controller controls to generate feedback information for indicating the approval/rejection of revocation and to send corresponding feedback information to the first display device.

In some embodiments, after the first display device receives feedback information indicating that the historical drawing operations are approved to be cancelled, sent by the second display device, the first controller deletes the historical drawing operations from the operation record list in response to the received feedback information, redraws the first image layer according to the drawing operations in the updated operation record list, and sends a synchronization instruction to the server, where the synchronization instruction is used to instruct the server to send a synchronization request to each second display device connected to the server to refresh the second electronic drawing board interface of each second display device, as shown in fig. 16, so that the second display device and the first display device display the same content.

In some embodiments, the first controller detects whether any other history drawing operation is deleted in the operation record list before the revocation operation is performed on the history drawing operation in response to the received feedback information indicating that the revocation of the history drawing operation is granted; if yes, judging that a revocation record list exists in the first display device, wherein the revocation record list is used for storing the historical drawing operation deleted in the operation record list; if not, judging that a revocation record list does not exist, generating a revocation record list, and storing the historical drawing operation into the revocation record list.

The revocation list may be a recovery chain including a plurality of nodes, and the history drawing operation deleted from the operation record list by the user is sequentially stored at the nodes of the recovery chain according to the deletion time, for example, the history drawing operation a, the history drawing operation B, and the history drawing operation C are sequentially revoked by the user, and are sequentially deleted from the operation record list/operation chain and sequentially stored at the nodes of the recovery chain, that is, the history drawing operation a is stored at a first node of the recovery chain, the history drawing operation B is stored at a second node of the recovery chain, and the history drawing operation C is stored at a third node of the recovery chain.

In some embodiments, when the first controller stores the history drawing operation in the revocation list, the history drawing operation is marked with the device identifier of the first display device, so that each second display device needs to send a recovery request to the first display device when performing the recovery operation on the history drawing operation, the recovery request includes a query instruction for recovering the history drawing operation, when the first display device agrees with the recovery request, the first controller feeds back feedback information indicating that the recovery is agreed to the second display device, and after receiving the feedback information, the second controller can perform the recovery operation on the history drawing operation.

In some embodiments, as shown in fig. 17, an operation record list and a revocation record list may also be stored in the server, and after receiving the synchronization instruction sent by the first display device, the server deletes the historical drawing operation from the operation record list in the server and stores the historical drawing operation into the revocation record list of the server, so as to refresh the second electronic drawing board interface of each second display device according to the updated operation record list of the server, so that each second electronic drawing board interface and the first electronic drawing board interface display the same content.

In some embodiments, as shown in fig. 18, the second memory of the second display device may also store an operation record list and a revocation record list, after the second controller receives a request for revoking the history drawing operation sent by the second display device, if the user sends an approval to revoke the instruction to the second display device, the second controller may delete and store the history drawing operation from the operation record list in the revocation record list in response to the instruction sent by the user, and then refresh the electronic drawing board interface and send a synchronization instruction to the server to instruct the server to refresh pages of the first display device and the rest of the display devices participating in the collaborative drawing, so that the display devices participating in the collaborative drawing display the same content.

In some embodiments, referring to fig. 19, if the first controller detects that the input device inputting the history drawing operation to be cancelled is the first display device, the history drawing operation is directly deleted from the operation record list and stored in the cancellation record list, the first electronic drawing board interface is refreshed according to the updated operation record list, and a synchronization instruction is sent to the server, the synchronization instruction is used for instructing the server to send a synchronization request to each second display device so as to refresh the second electronic drawing board interface of each second display device, so that the second display device and the first display device display the same content.

In some embodiments, the user may click on a recovery item on the first electronic drawing board interface to input a recovery instruction to the first controller, and the first controller triggers an associated option of displaying a revocation record list on the first electronic drawing board interface in response to the recovery instruction input by the user, wherein the revocation record list includes all the drawing operations deleted in the operation record list.

The revocation list can be a thumbnail list displayed in the form of a picture, and the thumbnail list includes history drawing operations to be restored, which are displayed in the form of thumbnails, wherein any one of the revocation lists corresponds to one history drawing operation to be restored in the revocation list, and content executed when each history drawing operation is restored is marked or displayed in the thumbnail. Alternatively, when the history drawing operation to be resumed is a storage filter or a color fill operation, there is also a thumbnail image corresponding to the history drawing operation, and a screen in which the storage filter or the color fill operation is displayed in the thumbnail image. The thumbnail list is arranged in the time axis, the two sides of the time axis are respectively provided with the rolling control, and the time axis can be controlled to roll by clicking the rolling control, so that the complete thumbnail list is browsed.

Fig. 20 is an information interaction diagram for resuming a history drawing operation according to some embodiments, fig. 21 is an interface diagram before a resuming operation is performed on a display device participating in collaborative drawing according to an embodiment of the present application, fig. 22 is an interface diagram after a resuming operation is performed on a display device participating in collaborative drawing according to an embodiment of the present application, as shown in fig. 20, when a user wants to resume a history drawing operation, a selection operation may be performed on at least one cancelled target history drawing operation option in a thumbnail list shown in fig. 21, a first controller detects an input device performing a cancelling operation on the history drawing operation in response to the selection operation by the user, sends a request for resuming the target history drawing operation to the input device, and deletes the history drawing operation on a resume chain, that is, deletes the history drawing operation selected by the user in the cancel record list after the request is successful, and storing the history drawing operation selected by the user in the operation record list, then refreshing the first electronic drawing board interface and sending a synchronization instruction to the server, where the synchronization instruction is used to instruct the server to send a synchronization request to each second display device connected to the server to refresh the second electronic drawing board interface of each second display device, so that the second display device and the first display device display the same content as shown in fig. 22.

In some embodiments, after the first display device sends a request for restoring the second target history drawing operation to the input device, if the user sends an agreement to a restoration instruction to the second display device, the second controller may delete and store the history drawing operation from the operation record list in a revocation record list in response to the instruction sent by the user, and then refresh the electronic drawing board interface and send a synchronization instruction to the server to instruct the server to refresh pages of the first display device and the rest of the display devices participating in the collaborative drawing, so that the display devices participating in the collaborative drawing display the same content.

In some embodiments, if the first controller detects that the input device inputting the historical drawing operation to be restored is the first display device, the historical drawing operation to be restored is directly deleted from the revocation record list and stored in the operation record list, the first electronic drawing board interface is refreshed according to the updated operation record list, and a synchronization instruction is sent to the server, and the synchronization instruction is used for instructing the server to send a synchronization request to each second display device so as to refresh the second electronic drawing board interface of each second display device, so that the second display device and the first display device display the same content.

In some embodiments, the present application further provides a display device, where the display device establishes a communication connection with a server through a built-in communicator, so as to execute all interactive contents when the second display device participates in the collaborative drawing in the foregoing embodiments, and details are not repeated in the present application.

In some embodiments, as shown in fig. 23 and fig. 24, the present application further provides a collaborative drawing method applied to the first display device in the above embodiments, the method including the steps of:

s1: establishing communication connection with a server so that the first display device and at least one second display device connected with the server are cooperated to draw pictures;

s2: acquiring a control instruction which is input by a user and indicates to cancel historical drawing operation;

s3: detecting an input device for inputting the historical drawing operation in response to the control instruction;

in some embodiments, the input device for detecting the input of the historical drawing operation further comprises: displaying an operation object of the selectable historical drawing operation in the electronic drawing board interface; receiving a selection operation input by a user aiming at the operation object; in response to the selection operation, an input device that inputs the history drawing operation is detected.

In some embodiments, the input device for detecting the input of the historical drawing operation further comprises: displaying an operation record list on the electronic drawing board interface, wherein the operation record list comprises at least one historical drawing operation input on the electronic drawing board interface of the first display device and at least one historical drawing operation input on the electronic drawing board interface of each second display device; receiving a selection operation input by a user aiming at the operation record list; in response to the selection operation, an input device that inputs the history drawing operation is detected.

In some embodiments, the input device for detecting the input of the historical drawing operation further comprises: displaying a user list on the electronic drawing board interface, wherein the user list comprises identifications of all display devices participating in collaborative drawing; receiving a selection operation of a user on the identifier of the display device in the user list; and responding to the selected operation, refreshing the electronic drawing board interface to display an operation record list of the selected display equipment on the refreshed electronic drawing board interface, wherein the operation record list of the selected display equipment comprises the historical drawing operation input on the electronic drawing board interface of the selected display equipment.

In some embodiments, the input device that detects the input of the historical drawing operation further includes: if the input device is the first display device, refreshing the electronic drawing board interface; and sending a synchronization instruction to the server, wherein the synchronization instruction is used for instructing the server to send a synchronization request to each second display device so as to refresh the electronic drawing board interface of each second display device.

S4: if the input device is the second display device, sending an undo request to the input device, wherein the undo request comprises a query instruction for undoing the historical drawing operation, and the undo request is used for requesting the second display device to feed back first feedback information indicating the undoing agreement after the second display device agrees to undoing the historical drawing operation;

s5: responding to the received first feedback information, and refreshing the electronic drawing board interface;

in some embodiments, said sending a revocation request to said input device further comprises: receiving second feedback information which is fed back by the second display device and used for indicating that revocation is refused; and responding to the received second feedback information, and displaying a feedback result on the electronic drawing board interface.

In some embodiments, refreshing the electronic palette interface further comprises: deleting the cancelled historical drawing operation in the operation record list; storing the revoked history drawing operation in a revocation record list; and refreshing the electronic drawing board interface according to the deleted historical drawing operation in the operation record list.

In some embodiments, the method further comprises: acquiring a recovery instruction which is input by a user and indicates to recover the historical drawing operation; detecting, in response to the restoration instruction, an input device that inputs an instruction to cancel the historical drawing operation control instruction; if the input device is a second display device, sending a recovery request to the second display device, wherein the recovery request comprises an inquiry instruction for recovering the historical drawing operation, and the recovery request is used for requesting the second display device to feed back third feedback information for indicating recovery after the second display device agrees to recover the historical drawing operation; in response to receiving the third feedback information, refreshing the electronic drawing board interface; and sending a synchronization instruction to the server to instruct the server to send a synchronization request to each second display device.

In some embodiments, said in response to receiving said third feedback information, further comprises: deleting the historical drawing operation in the revocation record list; storing the historical drawing operation in the operation record list.

S6: and sending a synchronization instruction to the server, wherein the synchronization instruction is used for instructing the server to send a synchronization request to each second display device so as to refresh the electronic drawing board interface of each second display device.

In specific implementation, the present invention further provides a computer storage medium, where the computer storage medium may store a program, and the program may include some or all of the steps in the embodiments of the collaborative drawing method provided by the present invention when executed. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM) or a Random Access Memory (RAM).

Those skilled in the art will readily appreciate that the techniques of the embodiments of the present invention may be implemented using software plus any required general purpose hardware platform. Based on such understanding, the technical solutions in the embodiments of the present invention may be essentially or partially implemented in the form of a software product, which may be stored in a storage medium, such as ROM/RAM, magnetic disk, optical disk, etc., and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method according to the embodiments or some parts of the embodiments.

The same and similar parts in the various embodiments in this specification may be referred to each other. In particular, for the embodiment of the display device, since it is substantially similar to the embodiment of the method, the description is simple, and for the relevant points, refer to the description in the embodiment of the method.

The above-described embodiments of the present invention should not be construed as limiting the scope of the present invention.

Claims

1. A display device, comprising:

a controller configured to:

2. The display device according to claim 1, wherein in the input device step of detecting input of the history drawing operation, the controller is further configured to:

displaying an operation object of the selectable historical drawing operation in the electronic drawing board interface;

receiving a selection operation input by a user aiming at the operation object;

in response to the selection operation, an input device that inputs the history drawing operation is detected.

3. The display device according to claim 1, wherein in the input device step of detecting input of the history drawing operation, the controller is further configured to:

displaying an operation record list on the electronic drawing board interface, wherein the operation record list comprises at least one historical drawing operation input on the electronic drawing board interface of the first display device and at least one historical drawing operation input on the electronic drawing board interface of each second display device;

receiving a selection operation input by a user aiming at the operation record list;

4. The display device according to claim 1, wherein in the input device step of detecting input of the history drawing operation, the controller is further configured to:

displaying a user list on the electronic drawing board interface, wherein the user list comprises identifications of all display devices participating in collaborative drawing;

receiving a selection operation of a user on the identifier of the display device in the user list;

and responding to the selected operation, refreshing the electronic drawing board interface to display an operation record list of the selected display equipment on the refreshed electronic drawing board interface, wherein the operation record list of the selected display equipment comprises the historical drawing operation input on the electronic drawing board interface of the selected display equipment.

5. The display device of claim 4, wherein in the step of refreshing the electronic palette interface, the controller is further configured to:

deleting the cancelled historical drawing operation in the operation record list;

storing the revoked history drawing operation in a revocation record list;

and refreshing the electronic drawing board interface according to the deleted historical drawing operation in the operation record list.

6. The display device of claim 1, wherein after the step of sending a revocation request to the input device, the controller is further configured to:

receiving second feedback information which is fed back by the second display device and used for indicating that revocation is refused;

and responding to the received second feedback information, and displaying a feedback result on the electronic drawing board interface.

7. The display device according to claim 1, wherein after the step of detecting the input device for inputting the historical drawing operation, the controller is further configured to:

if the input device is the first display device, refreshing the electronic drawing board interface;

8. The display device according to claim 5, wherein the controller is further configured to:

acquiring a recovery instruction which is input by a user and indicates to recover the historical drawing operation;

detecting, in response to the restoration instruction, an input device that inputs an instruction to cancel the historical drawing operation control instruction;

if the input device is a second display device, sending a recovery request to the second display device, wherein the recovery request comprises an inquiry instruction for recovering the historical drawing operation, and the recovery request is used for requesting the second display device to feed back third feedback information indicating that the recovery is agreed after the recovery of the historical drawing operation is agreed;

in response to receiving the third feedback information, refreshing the electronic drawing board interface;

and sending a synchronization instruction to the server to instruct the server to send a synchronization request to each second display device.

9. The display device of claim 8, wherein in response to receiving the third feedback information, the controller is further configured to:

deleting the historical drawing operation to be recovered in the revocation record list;

storing the historical drawing operation to be recovered in the operation record list.

10. A display device, comprising:

a communicator configured to establish a communication connection with a server to cause the display apparatus to draw as a second display apparatus in cooperation with a first display apparatus connected to the server;

a controller configured to:

receiving a revocation request sent by the first display device, wherein the revocation request comprises an inquiry instruction for revoking historical drawing operations, and the revocation request is used for requesting the second display device to feed back first feedback information for indicating the revocation after the revocation of the historical drawing operations is agreed;

in response to an instruction for agreeing to the cancel request input by a user, sending first feedback information to the first display device to enable the first display device to refresh an electronic drawing board interface of the first display device when receiving the first feedback information so as to cancel the historical drawing operation displayed by the first display device, and sending a synchronization instruction to the server by the first display device, wherein the synchronization instruction is used for instructing the server to send a synchronization request to the second display device;

receiving a synchronization request sent by the server;

and refreshing the electronic drawing board interface.

11. A collaborative drawing method is applied to a first display device and is characterized by comprising the following steps: