WO2017185785A1

WO2017185785A1 - Method and apparatus for front-facing touch screen device and computer storage medium

Info

Publication number: WO2017185785A1
Application number: PCT/CN2016/112487
Authority: WO
Inventors: 梁舟
Original assignee: 努比亚技术有限公司
Priority date: 2016-04-29
Filing date: 2016-12-27
Publication date: 2017-11-02
Also published as: CN105892753B; CN105892753A

Abstract

An intelligent lasso method for a front-facing touch screen device, comprising: acquiring track information input regarding an image by a user on a touch screen (401); according to a first image edge line of the image and a pre-set adjustment rule, adjusting the track information (402); and according to the adjusted track information, outputting a second image edge line (403). Meanwhile, further disclosed are an intelligent lasso apparatus for a front-facing touch screen device and a computer storage medium.

Description

Method, device and computer storage medium for touch screen device

Technical field

The present application relates to the field of image processing technologies, and in particular, to an intelligent lasso method, apparatus, and computer storage medium for a touch screen device.

Background technique

A practical technique widely used in image processing is the intelligent lasso tool. The existing intelligent lasso tool is based on the Intelligent Scissors method proposed by Mortensen et al. When using this method, the user needs to give the fine contour edge information of the selected image.

However, if the smart lasso tool is to be applied to the touch screen device, the prior art is not suitable for the touch screen device because the finger is not convenient to perform the dotted line on the touch screen. In addition, the current technology algorithms require a large number of global calculations, with picture size correlation, and computational overhead.

Summary of the invention

In view of this, embodiments of the present invention are directed to an intelligent lasso method, apparatus, and computer storage medium for a touch screen device, so as to reduce the accuracy constraint on the user's selection and dash operation, and ensure the accuracy of the lasso. At the same time improve efficiency.

In order to achieve the above object, the technical solution of the present application is implemented as follows:

In a first aspect, an embodiment of the present invention provides an intelligent lasso device for a touch screen device, the device comprising:

An obtaining module, configured to acquire trajectory information input by the user for the image on the touch screen;

An adjustment module, configured to adjust the track information according to a first image edge line of the image and a preset adjustment rule;

And an output module, configured to output a second image edge line according to the adjusted track information.

In the above solution, the obtaining module is specifically configured to:

Recording a trajectory that the user has swept on the touch screen for the image, according to a preset sampling frequency Rateing the trajectory to generate a plurality of support points, and using coordinates of the plurality of support points as the trajectory information.

In the above solution, the adjustment module is specifically configured to:

Obtaining coordinates of a point on the edge line of the first image closest to the support point as an adsorption support point;

The trajectory information is adjusted to coordinates of a plurality of the adsorption support points.

In the above solution, the adjustment module is specifically configured to:

Obtaining coordinates of a point on the edge line of the first image closest to the support point according to a movement cost function; wherein the moving cost function is:

ShiftCost(T_i)=omega_d*distance(T_i,C)+omega_GM*Gradient Magnitude(C)(i=1)

ShiftCost(T_i)=omega_d*distance(T_i,C)+omega_GM*Gradient Magnitude(C)+omega_theta*cos(theta(T_i-1->T_i,S_i-1->C))(i=2,3... , n);

Where C means that each parameter omega in the function is constant, T_i->T_i+1, S_i->C represents the vector formed by the two-point connection, and theta(T_i->T_i+1, S_i->C) represents two Angle between vectors;

When the point C obtains the optimum value, the coordinate S_i:=C of the point C is taken as the adsorption support point.

In the above solution, the output module is specifically configured to:

The coordinates of each of the adsorption support points are successively connected in accordance with the edge of the image and the second image edge line is output.

In a second aspect, an embodiment of the present invention provides a smart lasso method for a touch screen device, the method comprising:

Obtaining trajectory information input by the user for the image on the touch screen;

Adjusting the track information according to a first image edge line of the image and a preset adjustment rule;

And outputting a second image edge line according to the adjusted track information.

In the above solution, the acquiring the trajectory information input by the user for the image on the touch screen includes:

In the above solution, the adjusting the track information according to the first image edge line of the image and the preset adjustment rule includes:

In the above solution, acquiring coordinates of a point on the edge line of the first image closest to the support point as an adsorption support point includes:

ShiftCost(T_i)=omega_d*distance(T_i,C)+omega_GM*Gradient Magnitude(C)(i=1)

In the above solution, outputting the second image edge line according to the adjusted track information, including:

In a third aspect, an embodiment of the present invention provides a computer storage medium, the storage medium comprising a set of instructions, when executed, causing at least one processor to perform operations including:

The smart lasso method, device and computer storage medium for the touch screen device provided by the embodiment of the present invention acquires the track information input by the user for the image on the touch screen; according to the first image edge line of the image and the preset adjustment Rule, adjusting the track information; and outputting a second image edge line according to the adjusted track information. In this way, the fine image contour edge information can be output by the relatively rough contour edge input by the user, which reduces the accuracy constraint on the user's dotted line operation, and improves the efficiency while ensuring the lasso accuracy.

DRAWINGS

1 is a schematic structural diagram of hardware of an optional mobile terminal that implements various embodiments of the present application;

2 is a schematic structural diagram of a communication system that can be operated by a mobile terminal according to an embodiment of the present invention;

3 is a schematic structural diagram of an intelligent lasso device for a touch screen device according to an embodiment of the present invention;

4 is a schematic diagram of acquiring an image contour by a smart lasso tool in the prior art;

FIG. 5 is a schematic diagram of acquiring an image contour by using an intelligent lasso device facing a touch screen device according to an embodiment of the present invention; FIG.

FIG. 6 is a flowchart of a smart lasso method for a touch screen device according to an embodiment of the present invention.

detailed description

The technical solutions in the embodiments of the present invention will be clearly and completely described in the following with reference to the accompanying drawings.

A mobile terminal implementing various embodiments of the present application will now be described with reference to FIG. In the following description, the use of suffixes such as "module", "component" or "unit" for indicating an element is merely an explanation for facilitating the present invention, and does not have a specific meaning per se. Therefore, "module" and "component" can be used in combination.

The mobile terminal can be implemented in various forms. For example, the terminals described in the present invention may include, for example, mobile phones, smart phones, notebook computers, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (PADs), portable multimedia players (PMPs), navigation devices, and the like. Mobile terminals and fixed terminals such as digital TVs, desktop computers, and the like. In the following, it is assumed that the terminal is a mobile terminal. However, those skilled in the art will appreciate that configurations in accordance with embodiments of the present invention can be applied to fixed type terminals in addition to components that are specifically for mobile purposes.

FIG. 1 is a schematic diagram showing the hardware structure of a mobile terminal that implements various embodiments of the present application.

The mobile terminal 100 may include a wireless communication unit 110, an audio/video (A/V) input unit 120, a user input unit 130, a sensing unit 140, an output unit 150, a memory 160, an interface unit 170, a controller 180, and a power supply unit 190. and many more. Figure 1 shows a mobile terminal having various components, but it should be understood that not all illustrated components are required to be implemented, and more or fewer components may be implemented instead, and the components of the mobile terminal will be described in detail below. .

Wireless communication unit 110 typically includes one or more components that permit radio communication between mobile terminal 100 and a wireless communication system or network. For example, the wireless communication unit may include at least one of a broadcast receiving module 111, a mobile communication module 112, a wireless internet module 113, a short-range communication module 114, and a location information module 115.

The broadcast receiving module 111 receives a broadcast signal and/or broadcast associated information from an external broadcast management server via a broadcast channel. The broadcast channel can include a satellite channel and/or a terrestrial channel. The broadcast management server may be a server that generates and transmits a broadcast signal and/or broadcast associated information or a server that receives a previously generated broadcast signal and/or broadcast associated information and transmits it to the terminal. The broadcast signal may include a TV broadcast signal, a radio broadcast signal, a data broadcast signal, and the like. Moreover, the broadcast signal may further include a broadcast signal combined with a TV or radio broadcast signal. The broadcast associated information may also be provided via a mobile communication network, and in this case, the broadcast associated information may be received by the mobile communication module 112. The broadcast signal may exist in various forms, for example, it may exist in the form of Digital Multimedia Broadcasting (DMB) Electronic Program Guide (EPG), Digital Video Broadcasting Handheld (DVB-H) Electronic Service Guide (ESG), and the like. . The broadcast receiving module 111 can receive a signal broadcast by using various types of broadcast systems. In particular, the broadcast receiving module 111 can use forward link media (MediaFLO) by using, for example, multimedia broadcast-terrestrial (DMB-T), digital multimedia broadcast-satellite (DMB-S), digital video broadcast-handheld (DVB-H) The digital broadcasting system of the @) data broadcasting system, the terrestrial digital broadcasting integrated service (ISDB-T), and the like receives digital broadcasting. The broadcast receiving module 111 can be constructed It is suitable for various broadcasting systems that provide broadcast signals as well as the above-described digital broadcasting systems. The broadcast signal and/or broadcast associated information received via the broadcast receiving module 111 may be stored in the memory 160 (or other type of storage medium).

The mobile communication module 112 transmits the radio signals to and/or receives radio signals from at least one of a base station (e.g., an access point, a Node B, etc.), an external terminal, and a server. Such radio signals may include voice call signals, video call signals, or various types of data transmitted and/or received in accordance with text and/or multimedia messages.

The wireless internet module 113 supports wireless internet access of the mobile terminal. The module can be internally or externally coupled to the terminal. The wireless Internet access technologies involved in the module may include WLAN (Wireless LAN) (Wi-Fi), Wibro (Wireless Broadband), Wimax (Worldwide Interoperability for Microwave Access), HSDPA (High Speed Downlink Packet Access), etc. .

The short range communication module 114 is a module for supporting short range communication. Some examples of short-range communication technologies include BluetoothTM, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wide Band (UWB), ZigbeeTM, and the like.

The location information module 115 is a module for checking or acquiring location information of the mobile terminal. A typical example of the location information module 115 is a GPS (Global Positioning System). According to the current technology, the GPS module 115 calculates distance information and accurate time information from three or more satellites and applies triangulation to the calculated information to accurately calculate three-dimensional current position information based on longitude, latitude, and altitude. Currently, the method for calculating position and time information uses three satellites and corrects the calculated position and time information errors by using another satellite. Further, the GPS module 115 is capable of calculating speed information by continuously calculating current position information in real time.

The A/V input unit 120 is for receiving an audio or video signal. The A/V input unit 120 may include a camera 121 and a microphone 122 that processes image data of still pictures or video obtained by the image capturing device in a video capturing mode or an image capturing mode. The processed image frame can be displayed on the display unit 151. The image frames processed by the camera 121 may be stored in the memory 160 (or other storage medium) or transmitted via the wireless communication unit 110, and may be according to the mobile terminal. The configuration provides two or more cameras 121. The microphone 122 can receive sound (audio data) via the microphone 122 in an operation mode of a telephone call mode, a recording mode, a voice recognition mode, and the like, and can process such sound as audio data. The processed audio (voice) data can be converted to a format output that can be transmitted to the mobile communication base station via the mobile communication module 112 in the case of a telephone call mode. The microphone 122 can implement various types of noise cancellation (or suppression) algorithms to cancel (or suppress) noise or interference generated during the process of receiving and transmitting audio signals.

The user input unit 130 may generate key input data according to a command input by the user to control various operations of the mobile terminal. The user input unit 130 allows the user to input various types of information, and may include a keyboard, a pot, a touch pad (eg, a touch sensitive component that detects changes in resistance, pressure, capacitance, etc. due to contact), a scroll wheel , rocker, etc. In particular, when the touch panel is superimposed on the display unit 151 in the form of a layer, a touch screen can be formed.

The sensing unit 140 detects the current state of the mobile terminal 100 (eg, the open or closed state of the mobile terminal 100), the location of the mobile terminal 100, the presence or absence of contact (ie, touch input) by the user with the mobile terminal 100, and the mobile terminal. The orientation of 100, the acceleration or deceleration movement and direction of the mobile terminal 100, and the like, and generates a command or signal for controlling the operation of the mobile terminal 100. For example, when the mobile terminal 100 is implemented as a slide type mobile phone, the sensing unit 140 can sense whether the slide type phone is turned on or off. In addition, the sensing unit 140 can detect whether the power supply unit 190 provides power or whether the interface unit 170 is coupled to an external device. Sensing unit 140 may include proximity sensor 141 which will be described below in connection with a touch screen.

The interface unit 170 serves as an interface through which at least one external device can connect with the mobile terminal 100. For example, the external device may include a wired or wireless headset port, an external power (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device having an identification module, and an audio input/output. (I/O) port, video I/O port, headphone port, and more. The identification module may be stored to verify various information used by the user using the mobile terminal 100 and may include a User Identification Module (UIM), a Customer Identification Module (SIM), a Universal Customer Identity Module (USIM), and the like. In addition, a device having an identification module (hereinafter referred to as "identification device") may take the form of a smart card, and thus, The identification device can be connected to the mobile terminal 100 via a port or other connection device. The interface unit 170 can be configured to receive input from an external device (eg, data information, power, etc.) and transmit the received input to one or more components within the mobile terminal 100 or can be used at the mobile terminal and external device Transfer data between.

In addition, when the mobile terminal 100 is connected to the external base, the interface unit 170 may function as a path through which power is supplied from the base to the mobile terminal 100 or may be used as a transmission of various command signals allowing input from the base to the mobile terminal 100 The path to the terminal. Various command signals or power input from the base can be used as signals for identifying whether the mobile terminal is accurately mounted on the base. Output unit 150 is configured to provide an output signal (eg, an audio signal, a video signal, an alarm signal, a vibration signal, etc.) in a visual, audio, and/or tactile manner. The output unit 150 may include a display unit 151, an audio output module 152, an alarm unit 153, and the like.

The display unit 151 can display information processed in the mobile terminal 100. For example, when the mobile terminal 100 is in a phone call mode, the display unit 151 can display a user interface (UI) or a graphical user interface (GUI) related to a call or other communication (eg, text messaging, multimedia file download, etc.). When the mobile terminal 100 is in a video call mode or an image capturing mode, the display unit 151 may display a captured image and/or a received image, a UI or GUI showing a video or image and related functions, and the like.

Meanwhile, when the display unit 151 and the touch panel are superposed on each other in the form of a layer to form a touch screen, the display unit 151 can function as an input device and an output device. The display unit 151 may include at least one of a liquid crystal display (LCD), a thin film transistor LCD (TFT-LCD), an organic light emitting diode (OLED) display, a flexible display, a three-dimensional (3D) display, and the like. Some of these displays may be configured to be transparent to allow a user to view from the outside, which may be referred to as a transparent display, and a typical transparent display may be, for example, a TOLED (Transparent Organic Light Emitting Diode) display or the like. According to a particular desired embodiment, the mobile terminal 100 may include two or more display units (or other display devices), for example, the mobile terminal may include an external display unit (not shown) and an internal display unit (not shown) . The touch screen can be used to detect touch input pressure as well as touch input position and touch input area.

The audio output module 152 may convert audio data received by the wireless communication unit 110 or stored in the memory 160 when the mobile terminal is in a call signal receiving mode, a call mode, a recording mode, a voice recognition mode, a broadcast receiving mode, and the like. The audio signal is output as sound. Moreover, the audio output module 152 can provide audio output (eg, call signal reception sound, message reception sound, etc.) associated with a particular function performed by the mobile terminal 100. The audio output module 152 can include a speaker, a buzzer, and the like.

The alarm unit 153 can provide an output to notify the mobile terminal 100 of the occurrence of an event. Typical events may include call reception, message reception, key signal input, touch input, and the like. In addition to audio or video output, the alert unit 153 can provide an output in a different manner to notify of the occurrence of an event. For example, the alarm unit 153 can provide an output in the form of vibrations, and when a call, message, or some other incoming communication is received, the alarm unit 153 can provide a tactile output (eg, vibration) to notify the user of it. By providing such a tactile output, the user is able to recognize the occurrence of various events even when the user's mobile phone is in the user's pocket. The alarm unit 153 can also provide an output of the notification event occurrence via the display unit 151 or the audio output module 152.

The memory 160 may store a software program or the like that performs processing and control operations performed by the controller 180, or may temporarily store data (for example, a phone book, a message, a still image, a video, and the like) that has been output or is to be output. Moreover, the memory 160 can store data regarding vibrations and audio signals of various manners that are output when a touch is applied to the touch screen.

The memory 160 may include at least one type of storage medium including a flash memory, a hard disk, a multimedia card, a card type memory (eg, SD or DX memory, etc.), a random access memory (RAM), a static random access memory ( SRAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), programmable read only memory (PROM), magnetic memory, magnetic disk, optical disk, and the like. Moreover, the mobile terminal 100 can cooperate with a network storage device that performs a storage function of the memory 160 through a network connection.

The controller 180 typically controls the overall operation of the mobile terminal. For example, the controller 180 performs the control and processing associated with voice calls, data communications, video calls, and the like. In addition, the controller 180 can include A multimedia module 181 for reproducing (or playing back) multimedia data may be constructed within the controller 180 or may be configured to be separate from the controller 180. The controller 180 may perform a pattern recognition process to recognize a handwriting input or a picture drawing input performed on the touch screen as a character or an image.

The power supply unit 190 receives external power or internal power under the control of the controller 180 and provides appropriate power required to operate the various components and components.

The various embodiments described herein can be implemented in a computer readable medium using, for example, computer software, hardware, or any combination thereof. For hardware implementations, the embodiments described herein may be through the use of application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays ( An FPGA, a processor, a controller, a microcontroller, a microprocessor, at least one of the electronic units designed to perform the functions described herein, in some cases, such an embodiment may be at the controller 180 Implemented in the middle. For software implementations, implementations such as procedures or functions may be implemented with separate software modules that permit the execution of at least one function or operation. The software code can be implemented by a software application (or program) written in any suitable programming language, which can be stored in memory 160 and executed by controller 180.

So far, the mobile terminal has been described in terms of its function. Hereinafter, for the sake of brevity, a slide type mobile terminal among various types of mobile terminals such as a folding type, a bar type, a swing type, a slide type mobile terminal, and the like will be described as an example. Therefore, the present invention can be applied to any type of mobile terminal, and is not limited to a slide type mobile terminal.

The mobile terminal 100 as shown in FIG. 1 may be configured to operate using a communication system such as a wired and wireless communication system and a satellite-based communication system that transmits data via frames or packets.

A communication system in which the mobile terminal according to the present application can operate will now be described with reference to FIG.

Such communication systems may use different air interfaces and/or physical layers. For example, air interfaces used by communication systems include, for example, Frequency Division Multiple Access (FDMA), Time Division Multiple Access (TDMA), Code Division Multiple Access (CDMA), and Universal Mobile Telecommunications System (UMTS) (in particular, Long Term Evolution (LTE)). ), global Mobile communication system (GSM) and so on. As a non-limiting example, the following description relates to a CDMA communication system, but such teachings are equally applicable to other types of systems.

Referring to FIG. 2, a CDMA wireless communication system can include a plurality of mobile terminals 100, a plurality of base stations (BS) 270, a base station controller (BSC) 275, and a mobile switching center (MSC) 280. The MSC 280 is configured to interface with a public switched telephone network (PSTN) 290. The MSC 280 is also configured to interface with a BSC 275 that can be coupled to the base station 270 via a backhaul line. The backhaul line can be constructed in accordance with any of a number of known interfaces including, for example, E1/T1, ATM, IP, PPP, Frame Relay, HDSL, ADSL, or xDSL. It will be appreciated that the system as shown in FIG. 2 can include multiple BSCs 275.

Each BS 270 can serve one or more partitions (or regions), each of which is covered by a multi-directional antenna or an antenna directed to a particular direction radially away from the BS 270. Alternatively, each partition may be covered by two or more antennas for diversity reception. Each BS 270 can be configured to support multiple frequency allocations, and each frequency allocation has a particular frequency spectrum (eg, 1.25 MHz, 5 MHz, etc.).

The intersection of partitioning and frequency allocation can be referred to as a CDMA channel. BS 270 may also be referred to as a Base Transceiver Subsystem (BTS) or other equivalent terminology. In such a case, the term "base station" can be used to generally refer to a single BSC 275 and at least one BS 270. A base station can also be referred to as a "cell station." Alternatively, each partition of a particular BS 270 may be referred to as a plurality of cellular stations.

As shown in FIG. 2, a broadcast transmitter (BT) 295 transmits a broadcast signal to the mobile terminal 100 operating within the system. A broadcast receiving module 111 as shown in FIG. 1 is provided at the mobile terminal 100 to receive a broadcast signal transmitted by the BT 295. In Figure 2, several Global Positioning System (GPS) satellites 300 are shown. The satellite 300 helps locate at least one of the plurality of mobile terminals 100.

In Figure 2, a plurality of satellites 300 are depicted, but it is understood that useful positioning information can be obtained using any number of satellites. The GPS module 115 as shown in Figure 1 is typically configured to cooperate with the satellite 300 to obtain desired positioning information. Instead of GPS tracking technology or in addition to GPS tracking technology, other techniques that can track the location of the mobile terminal can be used. Additionally, at least one GPS satellite 300 can selectively or additionally process satellite DMB transmissions.

As a typical operation of a wireless communication system, BS 270 receives reverse link signals from various mobile terminals 100. Mobile terminal 100 typically participates in calls, messaging, and other types of communications. Each reverse link signal received by a particular base station 270 is processed within a particular BS 270. The obtained data is forwarded to the relevant BSC 275. The BSC provides call resource allocation and coordinated mobility management functions including a soft handoff procedure between the BSs 270. The BSC 275 also routes the received data to the MSC 280, which provides additional routing services for interfacing with the PSTN 290. Similarly, PSTN 290 interfaces with MSC 280, MSC 280 interfaces with BSC 275, and BSC 275 controls BS 270 accordingly to transmit forward link signals to mobile terminal 100.

Various embodiments of the present application are proposed based on the above-described mobile terminal hardware structure and communication system.

In various embodiments of the present application, the user input is appropriately adjusted according to the edge feature of the image by acquiring the input trajectory of the user on the touch screen for the image; the final contour information is generated according to the adjusted user input and output, and the final The contour information is the lasso area selected by the user. Thus, the fine image contour edge information can be outputted by the rougher contour edge input by the user, which reduces the accuracy constraint on the user's dotted line operation, and the guarantee sleeve Improve accuracy while improving accuracy.

FIG. 3 is a schematic structural diagram of an intelligent lasso device for a touch screen device according to an embodiment of the present invention. As shown in FIG. 3, the apparatus provided in this embodiment may include: an obtaining module 31, an adjusting module 32, and an output module 33.

The obtaining module 31 is configured to acquire trajectory information input by the user for the image on the touch screen;

The adjusting module 32 is configured to adjust the track information according to the first image edge line of the image and a preset adjustment rule;

The output module 33 is configured to output a second image edge line according to the adjusted track information.

Specifically, the acquiring module 31 is specifically configured to: record a trajectory that the user has swept on the touch screen for the image, sample the trajectory according to a preset sampling frequency, and generate at least one support point; The coordinates of the at least one support point are used as the trajectory information.

Since the touch screen is input on the touch screen of the touch screen device by the user's finger, it is a rough contour edge. In this embodiment, the acquisition module 31 inputs the user. The trajectory is sampled, and the sampling point is used as a support point, that is, the acquired trajectory information, and the trajectory information is appropriately adjusted according to the edge feature of the image itself to generate a finer image contour.

Specifically, the adjusting module 32 is specifically configured to: acquire coordinates of a point on the edge line of the first image that is closest to the support point as an adsorption support point; and adjust the track information into multiple The coordinates of the adsorption support point.

In an actual application, the acquiring module 31 records the trajectory information T that the finger swipes on the touch screen panel, and records a series of “support points” T_0, T_1, . . . , T_n, etc. according to a certain sampling frequency; The adjustment module 32 performs "adsorption" adjustment on each of the support points T_i to obtain an adsorption support point.

Specifically, the adjusting module 32 is specifically configured to: acquire, according to a movement cost function, coordinates of a point on the edge line of the first image that is closest to the support point; wherein the moving cost function is:

ShiftCost(T_i)=omega_d*distance(T_i,C)+omega_GM*Gradient Magnitude(C)(i=1)

Where C is the coordinate of the point on the edge line of the first image, omega is constant, T_i is the support point, T_i->T_i+1, S_i->C respectively represent the vector formed by the two-point connection, theta(T_i- >T_i+1, S_i->C) indicates the angle between the two vectors, and Gradient Magnitude indicates the calculation function of the gradient magnitude;

That is to say, in an actual application, the adjustment module 32 performs "adsorption" adjustment on each support point, moves each support point to the nearest image edge line, and correspondingly records coordinates to form a new set of "adsorption support". Point "S_0, S_1, ..., S_n, etc.; for a certain support point T_i, calculate the moving cost function of a candidate point C moving to the adjacent area Shift Search Region:

ShiftCost(T_i)=omega_d*distance(T_i,C)+omega_GM*Gradient Magnitude(C)(i=1)

Each omega is an empirical parameter, T_i->T_i+1, S_i->C represents the vector formed by the two-point connection, and theta(T_i->T_i+1, S_i->C) represents the angle between the two vectors.

The adjustment module is configured to select a point C on the edge line of the first image corresponding to the optimal calculation result based on the calculation result of the movement cost function, and use the coordinate of the point C as the coordinate of the adsorption support point. When the optimal value is obtained at a certain point C, the coordinates of C are recorded, that is, S_i:=C.

For speed optimization, the Shift Search Region should be limited to a small local area containing T_i-1, T_i instead of the whole picture.

After the adsorption support point is obtained, the output module 33 sequentially forms the coordinates of each of the adsorption support points according to the edge of the image and outputs the second image edge line.

Specifically, the output module is configured to sequentially connect the adjusted adsorption support points S_i to the final image edge line P according to the edge of the image;

Calculating the adsorption support point S_i, and a local cost function of the candidate point N in the adjacent area;

When the local cost function of the candidate point N obtains the optimal value, the coordinates of the record N are taken as P_tmp, the P_tmp is sequentially outputted as the obtained image edge line, and the image edge line is taken as the second image edge line.

Each adjusted "adsorption support point" S_i is successively connected to the image edge to form a final image edge line P; for a certain adsorption support point S_i, a local cost function of a candidate point N in the neighborhood NeighborSearchRegion is calculated.

LocalCost(S_i,N)=omega_z*Laplacian Operator(N)+omega_G*Gradient Magnitude(N)+omega_D*Gradient Dircetion(S_i,N)

Each omega is an empirical parameter.

When the optimal value is obtained at a certain point N, the coordinates of N are recorded, that is, P_tmp:=N.

For speed optimization, the Neighbor Search Region should be as small as possible and can be limited to eight pixels.

In the local area of the Path Connect Search Region containing S_i-1 and S_i,

Let LocalCost(S_i-1)=0, remember that LocalCost achieves the best advantage of P_tmp;

Do{

Record P_tmp in order in P

Find the best advantage of LocalCost near P_tmp and remember it as a new P_tmp

}while(P_tmp reaches S_i-1)

The image edge lines, that is, the second image edge lines, are obtained by sequentially outputting the respective pixel coordinates in P.

Among them, the local area reduction of Path Connect Search Region in the original method is limited to the adjacent area of S_i-1 and S_i, which can greatly improve the efficiency of the algorithm and also better maintain the original method.

As shown in FIG. 4 and FIG. 5, the black solid dots in the figure represent the pen point/pen drop point, and the black bold line segment represents a certain edge of the image;

In the current technology, the image edge line is formed as shown in FIG. 4, and there is a “burr” phenomenon near/away from the edge; in this embodiment, the image edge line is obtained as shown in FIG. 5, and the landing pen point is “adsorbed” to form an adsorption. The point (black hollow point), which forms the edge line based on the adsorption point, eliminates the "burr phenomenon".

The smart lasso device for the touch screen device provided by the embodiment can output fine image contour edge information by means of a relatively rough contour edge input by the user, thereby reducing the accuracy constraint on the user's selection and dashing operation, and ensuring Lasso accuracy increases efficiency at the same time.

In an actual application, the obtaining module 31, the adjusting module 32, and the output module 33 may each be a Central Processing Unit (CPU) or a microprocessor (Micro Processor Unit) located in a smart lasso device facing the touch screen device. , MPU), digital signal processor (DSP) or Field Programmable Gate Array (FPGA) implementation.

FIG. 4 is a flowchart of a smart lasso method for a touch screen device according to an embodiment of the present invention. As shown in FIG. 4, the method provided in this embodiment may be specifically used in a terminal device configured with a touch screen. Specifically, the method provided in this embodiment may include:

Step 401: Acquire trajectory information input by the user for the image on the touch screen.

In this step, since the touch screen is input on the touch screen of the touch screen device by the user's finger, it is a rough contour edge. In this embodiment, the user inputs the track. Sampling is performed, and the sampling point is used as a supporting point, that is, the trajectory information is acquired, and the trajectory information is adjusted according to the edge feature of the image itself, and a fine image contour is generated based on the adjusted trajectory.

Specifically, the trajectory that the user has swept on the touch screen is recorded, and the trajectory is sampled according to a preset sampling frequency to generate a plurality of support points, and coordinates of the plurality of support points are used as Track information.

Step 402: Adjust the track information according to the first image edge line of the image and a preset adjustment rule.

Obtaining coordinates of a point on the edge line of the first image closest to the support point as an adsorption support point; adjusting the trajectory information to coordinates of the plurality of adsorption support points.

In an actual application, acquiring coordinates of a point on the edge line of the first image closest to the support point as an adsorption support point includes:

Obtaining coordinates of a point on the edge line of the first image closest to the support point according to a movement cost function; wherein the moving cost function is: ShiftCost(T_i)=omega_d*distance(T_i, C)+ omega_GM*Gradient Magnitude(C)(i=1)

Where C is the parameter in the function, omega is a constant, T_i->T_i+1, S_i->C represents the vector formed by the two-point connection, theta(T_i->T_i+1, S_i->C) The angle between the two vectors; when the point C obtains the optimal value, the coordinate S_i:=C of the point C is taken as the adsorption support point.

In practical applications, a series of "support points" T_0, T_1, ..., T_n, etc. are generated by recording a track information T that is swiped by a finger on the touch screen panel at a certain sampling frequency; the adjustment module 32 is Each support point T_i is separately "adsorbed" to obtain an adsorption support point.

Specifically, according to the moving cost function, acquiring coordinates of a point on the edge line of the first image closest to the support point; wherein the moving cost function is:

ShiftCost(T_i)=omega_d*distance(T_i,C)+omega_GM*Gradient Magnitude(C)(i=1)

Where C is the coordinate of the point on the edge line of the first image, omega is constant, T_i is the coordinate of the support point, T_i->T_i+1, S_i->C is the vector formed by the two-point connection, theta(T_i ->T_i+1, S_i->C) represents the angle between the two vectors, and Gradient Magnitude represents the calculation function of the gradient magnitude;

Based on the calculation result of the movement cost function, the point C on the edge line of the first image corresponding to the optimal calculation result is selected, and the coordinate of the point C is taken as the coordinate of the adsorption support point; specifically, the coordinates of the point C are obtained. S_i:=C as the adsorption support point.

That is to say, in the actual application, the “adsorption” adjustment is performed on each support point, and each support point is moved to the nearest image edge line, and corresponding coordinates are recorded to form a new set of “adsorption support points” S_0, S_1. ,..., S_n, etc.; for a certain support point T_i, calculate the moving cost function of a candidate point C moving to the adjacent area Shift Search Region:

ShiftCost(T_i)=omega_d*distance(T_i,C)+omega_GM*Gradient Magnitude(C)(i=1)

When the optimal value is obtained at a certain point C, the coordinates of C are recorded, that is, S_i:=C.

For the speed optimization meter, according to the moving cost function, acquiring the coordinates of the points on the edge line of the first image in a local area including T_i-1, T_i;

In other words, the Shift Search Region should be limited to a small local area containing T_i-1, T_i instead of the entire image. The size of a small partial area may be set according to actual conditions, for example, may be 1/10 size of the entire image, or other sizes, and is not exhaustive here.

Step 403: Output a second image edge line according to the adjusted track information.

Forming the coordinates of each of the adsorption support points successively according to the edge of the image and outputting the second image Like the edge line.

In practical applications, after the adsorption support point is obtained, the coordinates of each of the adsorption support points are successively connected according to the edge of the image and the second image edge line is output.

Specifically, the coordinates of each of the adsorption support points are sequentially connected according to an image edge and the second image edge line is output, including:

Each adjusted adsorption support point S_i is successively connected to the edge of the image to form a final image edge line P;

That is to say, each adjusted "adsorption support point" S_i is successively connected to the image edge to form a final image edge line P; for a certain adsorption support point S_i, a local cost function of a candidate point N in the neighborhood NeighborSearchRegion is calculated;

Wherein, the local cost function can be:

Among them, each omega is a preset empirical parameter, Gradient Magnitude represents the calculation function of the gradient magnitude, and Gradient Dircetion represents the calculation function of the gradient direction.

Further, for the speed optimization meter, the neighboring area is set to be adjacent to a preset number of pixels. For example, the Neighbor Search Region should be as small as possible and can be limited to eight pixels. In the local area of Path Connect Search Region containing S_i-1 and S_i, let LocalCost(S_i-1)=0, and note that LocalCost achieves the best advantage as P_tmp;

Do{

Record P_tmp in order in P;

Find the best advantage of LocalCost near P_tmp and record it as new P_tmp;

}while(P_tmp reaches S_i-1)

Embodiments of the present invention also provide a computer storage medium, the storage medium comprising a set of instructions that, when executed, cause at least one processor to perform operations including:

The smart lasso method for the touch screen device provided by the embodiment can output fine image contour edge information by means of a relatively rough contour edge input by the user, thereby reducing the accuracy constraint on the user's selection and dash operation, and ensuring Lasso accuracy increases efficiency at the same time.

It is to be understood that the term "comprises", "comprising", or any other variants thereof, is intended to encompass a non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes those elements. It also includes other elements that are not explicitly listed, or elements that are inherent to such a process, method, article, or device. An element that is defined by the phrase "comprising a ..." does not exclude the presence of additional equivalent elements in the process, method, item, or device that comprises the element.

The serial numbers of the embodiments of the present invention are merely for the description, and do not represent the advantages and disadvantages of the embodiments.

Through the description of the above embodiments, those skilled in the art can clearly understand that the foregoing embodiment method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be through hardware, but in many cases, the former is better. Implementation. Based on such understanding, the technical solution of the present invention, which is essential or contributes to the prior art, may be embodied in the form of a software product stored in a storage medium (such as ROM/RAM, disk, CD), package A number of instructions are included to cause a terminal device (which may be a cell phone, computer, server, air conditioner, or network device, etc.) to perform the methods described in various embodiments of the present invention.

The present invention has been described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (system), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or FIG. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing device to produce a machine for the execution of instructions for execution by a processor of a computer or other programmable data processing device. Means for implementing the functions specified in one or more of the flow or in a block or blocks of the flow chart.

The computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture comprising the instruction device. The apparatus implements the functions specified in one or more blocks of a flow or a flow and/or block diagram of the flowchart.

These computer program instructions can also be loaded onto a computer or other programmable data processing device such that a series of operational steps are performed on a computer or other programmable device to produce computer-implemented processing for execution on a computer or other programmable device. The instructions provide steps for implementing the functions specified in one or more of the flow or in a block or blocks of a flow diagram.

The above are only the preferred embodiments of the present invention, and are not intended to limit the scope of the invention, and the equivalent structure or equivalent process transformations made by the description of the present invention and the drawings are directly or indirectly applied to other related technical fields. The same is included in the scope of patent protection of the present invention.

Industrial applicability

Embodiments of the present invention provide an intelligent lasso device, a method, and a computer storage medium for a touch screen device, which acquire track information for an image input by a user on a touch screen; according to a first image edge line of the image and a preset adjustment rule And adjusting the track information; and outputting a second image edge line according to the adjusted track information. In this way, it is possible to rely on the rougher contour edges of the user input. Outputting fine image contour edge information reduces the accuracy constraint on the user's point-dash operation, and improves the efficiency while ensuring the lasso accuracy.

Claims

An intelligent lasso device for a touch screen device, the device comprising:

Obtaining a module, configured to acquire trajectory information input by the user for the image on the touch screen;

The adjusting module is configured to adjust the track information according to the first image edge line of the image and a preset adjustment rule;

And an output module configured to output a second image edge line according to the adjusted track information.
The device according to claim 1, wherein the acquiring module is configured to record a trajectory that the user has swept on the touch screen for the image, and sample the trajectory according to a preset sampling frequency to generate at least a support point; the coordinates of the at least one support point are taken as the track information.
The apparatus according to claim 2, wherein the adjustment module is configured to acquire coordinates of a point on the edge line of the first image closest to the support point as an adsorption support point; Adjusted to the coordinates of a plurality of said adsorption support points.
The apparatus according to claim 3, wherein the adjustment module is configured to acquire coordinates of a point on the edge line of the first image closest to the support point as an adsorption support point according to a movement cost function.
The apparatus of claim 3 wherein said moving cost function is:

ShiftCost(T_i)=omega_d*distance(T_i,C)+omega_GM*Gradient Magnitude(C)(i=1)

ShiftCost(T_i)=omega_d*distance(T_i,C)+omega_GM*Gradient Magnitude(C)+omega_theta*cos(theta(T_i-1->T_i,S_i-1->C))(i=2,3... , n);

Where C is the coordinate of the point on the edge line of the first image, omega is constant, T_i is the support point, T_i->T_i+1, S_i->C respectively represent the vector formed by the two-point connection, theta(T_i- >T_i+1, S_i->C) indicates the angle between the two vectors, and Gradient Magnitude indicates the calculation function of the gradient magnitude;

The adjusting module is configured to select an optimal based on a calculation result of the moving cost function The point C on the edge line of the first image corresponding to the calculation result is the coordinate of the point C as the coordinate of the adsorption support point.
The apparatus according to claim 5, wherein the adjustment module is configured to acquire coordinates of a point on the edge line of the first image in a local area including T_i-1, T_i according to a movement cost function .
The apparatus according to claim 4, wherein the output module is configured to sequentially form coordinates of each of the adsorption support points in accordance with an image edge and output the second image edge line.
The apparatus according to claim 7, wherein the output module is configured to sequentially connect the respective adjusted adsorption support points S_i to the final image edge line P according to the edge of the image;

Calculating the adsorption support point S_i, and a local cost function of the candidate point N in the adjacent area;

When the local cost function of the candidate point N obtains the optimal value, the coordinates of the record N are taken as P_tmp, the P_tmp is sequentially outputted as the obtained image edge line, and the image edge line is taken as the second image edge line.
The apparatus of claim 8 wherein said local cost function is:

LocalCost(S_i,N)=omega_z*Laplacian Operator(N)+omega_G*Gradient Magnitude(N)+omega_D*Gradient Dircetion(S_i,N);

Among them, each omega is a preset empirical parameter, Gradient Magnitude represents the calculation function of the gradient magnitude, and Gradient Dircetion represents the calculation function of the gradient direction.
The apparatus of claim 9, wherein the output module is configured to be disposed adjacent to a predetermined number of pixels.
A smart lasso method for a touch screen device, the method comprising:

Obtaining trajectory information input by the user for the image on the touch screen;

Adjusting the track information according to a first image edge line of the image and a preset adjustment rule;

And outputting a second image edge line according to the adjusted track information.
The method according to claim 11, wherein the acquiring the trajectory information input by the user for the image on the touch screen comprises:

Recording, by the user, a trajectory drawn on the touch screen by the image, sampling the trajectory according to a preset sampling frequency to generate at least one support point; and using coordinates of the at least one support point as the trajectory information .
The method of claim 12, wherein the adjusting the track information according to the first image edge line of the image and a preset adjustment rule comprises:

Obtaining coordinates of a point on the edge line of the first image closest to the support point as an adsorption support point;

The trajectory information is adjusted to coordinates of a plurality of the adsorption support points.
The method according to claim 13, wherein acquiring coordinates of a point on the edge line of the first image closest to the support point as the adsorption support point comprises:

According to the moving cost function, coordinates of points on the edge line of the first image closest to the support point are acquired as adsorption support points.
The method of claim 14 wherein said moving cost function is:

ShiftCost(T_i)=omega_d*distance(T_i,C)+omega_GM*Gradient Magnitude(C)(i=1)

ShiftCost(T_i)=omega_d*distance(T_i,C)+omega_GM*Gradient Magnitude(C)+omega_theta*cos(theta(T_i-1->T_i,S_i-1->C))(i=2,3... , n);

Where C is the coordinate of the point on the edge line of the first image, omega is constant, T_i is the support point, T_i->T_i+1, S_i->C respectively represent the vector formed by the two-point connection, theta(T_i- >T_i+1, S_i->C) indicates the angle between the two vectors, and Gradient Magnitude indicates the calculation function of the gradient magnitude;

Correspondingly, the acquiring the coordinates of the point on the edge line of the first image closest to the support point as the adsorption support point includes:

Based on the calculation result of the movement cost function, the point C on the edge line of the first image corresponding to the optimal calculation result is selected, and the coordinates of the point C are taken as the coordinates of the adsorption support point.
The method of claim 15 wherein the method further comprises:

Obtaining the first part in a local area including T_i-1, T_i according to a moving cost function The coordinates of the point on the edge of the image.
The method according to claim 14, wherein outputting the second image edge line according to the adjusted track information comprises:

The coordinates of each of the adsorption support points are successively connected according to the edge of the image and the second image edge line is output.
The method according to claim 17, wherein the forming the coordinates of each of the adsorption support points in a line by image edge and outputting the second image edge line comprises:

Each adjusted adsorption support point S_i is successively connected to the edge of the image to form a final image edge line P;

Calculating the adsorption support point S_i, and a local cost function of the candidate point N in the adjacent area;

When the local cost function of the candidate point N obtains the optimal value, the coordinates of the record N are taken as P_tmp, the P_tmp is sequentially outputted as the obtained image edge line, and the image edge line is taken as the second image edge line.
The method of claim 18 wherein said local cost function is:

LocalCost(S_i,N)=omega_z*Laplacian Operator(N)+omega_G*Gradient Magnitude(N)+omega_D*Gradient Dircetion(S_i,N);

Among them, each omega is a preset empirical parameter, Gradient Magnitude represents the calculation function of the gradient magnitude, and Gradient Dircetion represents the calculation function of the gradient direction.
A computer storage medium comprising a set of instructions that, when executed, cause at least one processor to perform operations comprising:

Obtaining trajectory information input by the user for the image on the touch screen;

Adjusting the track information according to a first image edge line of the image and a preset adjustment rule;

And outputting a second image edge line according to the adjusted track information.