CN110609756A

CN110609756A - Reading method, electronic device, computer equipment and storage medium

Info

Publication number: CN110609756A
Application number: CN201910756055.8A
Authority: CN
Inventors: 刘行行
Original assignee: Ping An Life Insurance Company of China Ltd
Current assignee: Ping An Life Insurance Company of China Ltd
Priority date: 2019-08-15
Filing date: 2019-08-15
Publication date: 2019-12-24

Abstract

The invention discloses a reading method, an electronic device, computer equipment and a storage medium.A multithreading module and a thread inherited from the multithreading module are arranged, and a calling function corresponding to the multithreading module is arranged in the thread; setting a thread pool for storing running threads in an application program; and starting the application program to call the threads in the thread pool so that the application program reads data in the multithreading module according to the multithreading mode. The reading method, the electronic device, the computer and the storage medium provided by the invention have the advantages that the multithreading module and the corresponding calling function are arranged, so that the threads can run in parallel, the multithreading data reading mode is realized, and the data reading time can be effectively shortened.

Description

Reading method, electronic device, computer equipment and storage medium

Technical Field

The present invention relates to the field of computer technologies, and in particular, to a reading method, an electronic device, a computer device, and a storage medium.

Background

Python is a high-level scripting language that combines interpretive, compiled, interactive, and object-oriented programming, designed by Guido van Rossum in the late eighties and early nineties, at the national institute of mathematics and computer science in the Netherlands.

Due to the simplicity, readability and extensibility of Python language, the number of research institutes using Python for scientific computing is increasing abroad, and some well-known universities have adopted Python to teach programming courses. For example, the programming base of the university of kanji merlon, the computer science of the massachusetts institute of technology, and the programming guide teach in the Python language. Many open-source scientific computing software packages provide calling interfaces of Python, such as the famous computer vision library OpenCV, the three-dimensional visualization library VTK and the medical image processing library ITK. The Python-specific scientific computing expansion libraries are more, for example, the following 3 very classical scientific computing expansion libraries: NumPy, SciPy, and matplotlib, which provide fast array processing, numerical operations, and graphics functions for Python, respectively. Therefore, the development environment formed by the Python language and the plurality of expansion libraries thereof is very suitable for engineering technology, scientific research personnel to process experimental data, make charts and even develop scientific computing application programs.

In Windows system applications using Python compilation, the program defaults to a single thread to handle interactions with the user and feedback on the response of the backend server. Currently, a Central Processing Unit (CPU) is the core of a computer, and it takes all tasks of the computer. Modern operating systems adopt a multi-channel programming mechanism, and multiple processes can be executed concurrently, for example, when music is played by using a music player, a browser can be used to browse a web page at the same time. However, for the same task, the reading mode is still a single-threaded mode, that is, when tasks such as reading need to be executed, the reading mode is still traversed in a single-threaded cycle, and when the data volume is large, the reading time is very large.

Disclosure of Invention

In view of this, the present invention provides a reading method, an electronic apparatus, a computer device and a storage medium, which can effectively shorten the time period for reading data.

First, in order to achieve the above object, the present invention provides a reading method, including the steps of:

setting a multithreading module and a thread inherited to the multithreading module, and setting a calling function corresponding to the multithreading module in the thread;

setting a thread pool for storing running threads in an application program; and

and starting the application program to call the threads in the thread pool so that the application program reads data in the multithreading module according to the multithreading mode.

Further, the step of setting a multithreading module further comprises:

defining classes of multi-threaded modules according to the projects; and

and setting a multithreading module according to the class modification application program source code.

Further, the step of setting a calling function corresponding to the multithreading module in the thread further includes:

importing parameters related to a multithreading module in the calling function;

and setting an initialization function in the thread, and importing corresponding parameters into the thread through the initialization function.

Further, the step of starting the application program to call the thread in the thread pool further comprises:

and enabling the application program to call the thread from the thread pool through a thread running function.

Further, the thread pool is set in an initialization function of the application program.

Further, before the step of starting the application program to call the thread in the thread pool, the method further includes:

setting a global interpreter lock in an application program, and controlling the running of threads in the application program through the global interpreter lock.

Further, after the step of reading the data in the multi-threaded module by the application program according to the multi-threaded mode, the method further includes:

and encapsulating a result obtained by reading the data in the multithreading module, and setting a calling interface.

In addition, to achieve the above object, the present invention also provides an electronic device, including:

the device comprises a first setting module, a second setting module and a control module, wherein the first setting module is suitable for setting a multithreading module and a thread inherited to the multithreading module, and a calling function corresponding to the multithreading module is set in the thread;

the second setting module is suitable for setting a thread pool for storing the running threads in the application program; and

and the starting module is suitable for starting the application program to call the threads in the thread pool so that the application program reads the data in the multithreading module according to the multithreading mode.

Further, the first setup module is adapted to define classes of multi-threaded modules according to projects; and setting a multithreading module according to the class modification application program source code.

Further, the first setting module is adapted to import a parameter related to a multithreading module in the calling function; and setting an initialization function in the thread, and importing corresponding parameters into the thread through the initialization function.

Further, the start module is adapted to cause the application to call a thread from the thread pool by a thread run function.

Furthermore, the electronic device further comprises a third setting module, which is adapted to set a global interpreter lock in the application program, and control the running of the thread in the application program through the global interpreter lock.

Furthermore, the electronic device further comprises an encapsulation module which is suitable for encapsulating results obtained by reading data in the multithreading module and setting a calling interface.

To achieve the above object, the present invention further provides a computer device, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

To achieve the above object, the present invention also provides a computer-readable storage medium having stored thereon a computer program which, when being executed by a processor, carries out the steps of the above method.

Compared with the prior art, the reading method, the electronic device, the computer equipment and the storage medium provided by the invention have the advantages that the threads can run in parallel by setting the multithreading module and the corresponding calling function, so that a multithreading data reading mode is realized, and the data reading time can be effectively shortened.

Drawings

FIG. 1 is a schematic flow diagram of a reading method according to an exemplary embodiment of the present invention;

FIG. 2 is a flow chart diagram illustrating a reading method in accordance with an exemplary embodiment of the present invention;

FIG. 3 is a flow chart diagram illustrating a reading method in accordance with an exemplary embodiment of the present invention;

FIG. 4 is a flow chart diagram illustrating a reading method in accordance with an exemplary embodiment of the present invention;

FIG. 5 is a flow chart diagram illustrating a reading method in accordance with an exemplary embodiment of the present invention;

FIG. 6 is a flow chart diagram illustrating a reading method in accordance with an exemplary embodiment of the present invention;

FIG. 7 is a schematic diagram illustrating program modules of an electronic device in accordance with an exemplary embodiment of the present invention;

fig. 8 is a schematic diagram of a hardware architecture of an electronic device according to an exemplary embodiment of the present invention.

Reference numerals:

the implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

It should be noted that the description relating to "first", "second", etc. in the present invention is for descriptive purposes only and is not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In addition, technical solutions between various embodiments may be combined with each other, but must be realized by a person skilled in the art, and when the technical solutions are contradictory or cannot be realized, such a combination should not be considered to exist, and is not within the protection scope of the present invention.

Referring to fig. 1, a flow chart of a reading method according to an exemplary embodiment of the present invention is shown, the method includes the following steps:

step S110, a multithreading module and a thread inherited from the multithreading module are set, and a calling function corresponding to the multithreading module is set in the thread;

step S120, a thread pool for storing the running threads is set in the application program; and

step S130, the application program is started to call the thread in the thread pool, so that the application program reads the data in the multithreading module according to the multithreading mode.

Setting a multithreading module and a thread inherited to the multithreading module, and setting a calling function corresponding to the multithreading module in the thread. In an embodiment of the present invention, the reading method is applied to Python, which is a high-level scripting language combining interpretability, compilability, interactivity and object-oriented, and in Python, a control Thread activity running separately can be depicted by a multithreading (Thread) class, and there are two ways to specify this activity: 1) passing through a constructor of the callable object; 2) method of running a function (Run ()) by overriding a subclass.

The following briefly introduces a common approach to multi-threaded modules.

1) Start function (Start ())

Objects in Python need to Start thread activity with Start ().

2) Run function (Run ())

The activity of a thread is described by Run (), of course this method can be overridden in subclasses.

3) Connection function (Join ())

The Python multi-thread class also provides a connection function method, so that one thread can wait for the other thread to finish executing and then continue to run. The method may also set a timeout (Time Out) parameter to avoid endless waiting. Since two threads complete sequentially, appearing as one thread, it is called a merge of threads.

The timeout is set by passing a parameter to the connection function, i.e. the connection function is not blocking the process beyond a specified time. In the actual application test, it is found that not all threads end within the timeout period, but the sequential execution checks whether the timeout period expires, for example, the timeout period is set to 2s, and in the case that the previous thread does not complete, the execution of the connection function by the subsequent thread will reset the timeout of 2s from the end time of the previous thread.

As shown in fig. 2, in an embodiment of the present invention, the step of setting the multithreading module may include the following steps:

step S201, defining the class of a multithreading module according to the project; and

and S202, setting a multithreading module according to the class modification application program source code.

The core function of the multi-thread module is start _ new _ thread (), through which a new thread can be generated.

The start _ new _ thread () function is a key function of the multi-thread module, and its syntax is the same as the built-in application function (application ()), and its parameters are: a function, parameters of a function, and optionally parameters of a keyword. The difference is that instead of running the function in the main thread, a new thread is created to run the function.

In an embodiment of the present invention, the multithreading module is a Thread module, and certainly, the Threading module may also be used, and the Threading module may be obtained by packaging the Thread module, and compared with the Thread module, the Threading module may be used more conveniently. The Threading module is specifically described below by a constructor.

And (3) constructing a function: class reading, thread (group, target, name, args (), kwargs { }, daemon ═ None)

It is worth noting that if a subclass rewrites the constructor, it must be ensured that the base class's constructor initialization function (thread) is called before doing something else.

After the multithreading module is set, a thread inherited to the multithreading module needs to be created, and a calling function corresponding to the multithreading module is set in the thread. A process is an executing program, and each process execution has an execution sequence which is an execution path or a control unit. A thread is an independent control unit in a process, and the thread controls the execution of the process. There is at least one thread in a process.

There are many ways to create a thread, just to name one way by way of example: inheriting a multithreading module class, comprising the steps of:

one), the definition class inherits the multithreading module class.

Second), a method for duplicating running function (Run ()) in the multithread module class. The purpose is as follows: and storing the self-defined code in the function running method so as to enable the thread to run.

Third), call thread's Start function (Start ()) method. The two roles of the start-up function method are: 1) starting thread, 2) calling a running function method.

As shown in fig. 3, in an embodiment of the present invention, the step of setting a calling function corresponding to the multithread module in the thread may include the following steps:

step S301, importing parameters related to a multithreading module into the calling function;

step S302, an initialization function is set in the thread, and corresponding parameters are led into the thread through the initialization function.

Generally, when a function needs to be called, data is input through a parameter of the function, and a final calculation result is returned through a return value of the function. But in the multi-threaded mode, there is a great difference between the passing and returning of data. Since the execution and termination of a thread is unpredictable, data cannot be returned through function arguments and return (return) statements as a function does when passing and returning data. To this end, the present invention provides the following several methods for transferring function parameters to threads.

1) Transferring parameters by using constructor

When creating a Thread, an instance of the Thread class or subclass thereof must be created. Data is transferred into the thread through the construction method of the thread class before the function method is called and started, and the transferred data use class variable is stored for the thread to use (to use in the function method is executed).

When the instantiation thread starts the class where the thread is located, the parameters required by the thread are transmitted to the constructor of the class and are assigned to the member variables of the class, so that the variables can be used in the thread, and the purpose of transmitting the parameters to the thread is achieved.

Because the method is used for transferring data while creating the thread object, the data is already in place before the thread runs, and the phenomenon that the data is transferred after the thread runs is avoided. If more complex data is to be communicated, a data structure such as a collection, class, etc. may be used.

Although it is safe to transmit data by using the construction method, it causes inconvenience if the data to be transmitted is large. Because Java has no default parameters, heavy load is required to be used to realize the effect similar to the default parameters, so that the construction method is too complex, and the number of the construction methods is greatly increased. Therefore, to avoid this, data must be passed through class methods or class variables.

2) Data transfer by variables and methods

There are generally two opportunities to transfer data into an object, the first being to transfer the data in through a construction method when building the object, and the other being to define a series of public (public) methods or variables (also referred to as fields) in a class. After the objects are established, the values are assigned one by one through the object instances.

3) Passing thread parameters using thread pool (ThreadPool) class

The thread pool class provides a thread pool (which may be viewed as a container of threads) maintained by the system. The thread pool class has a static method Queue User WorkItem (Queue User WorkItem) that queues a thread for execution and specifies objects that contain data used by the thread. The thread pool class is adapted to spend most of the application program's time waiting for an event to occur and then give a response.

4) Instantiating a multithreaded class with Parameterized Thread Start (Parameterized Thread Start) delegation

The parameterized Thread starts an instance of the initialized multithreading class, specifies a delegate that allows objects to pass to the Thread at Thread start-up, and passes parameters to the threadlnstance.start method using NET Framework version 2.0 constructor public Thread start of the newly added multithreading.

In step S120, a thread pool for storing the running threads is set in the application.

The running of a program is the execution of a process, and a process can have a plurality of threads inside. In an operating system, a thread is the smallest unit which can be scheduled, the creation and destruction of the thread can enable an application program to quickly respond to operation, but the thread cannot be generated without limitation, because the creation of the thread and the destruction of the thread have corresponding performance overhead, and the frequent creation and destruction obviously reduce the performance of the application program.

When a large number of threads exist in the system, the system schedules each thread in a time slice round robin manner, so that in the case of a large number of threads, there is no way to get each thread absolutely parallel unless the number of threads is less than or equal to the number of cores of the CPU, which is generally not possible. Because a great deal of threads are created and destroyed in the process, the method is a behavior of consuming performance very much, so that a thread management tool, namely a thread pool, can be used for managing the threads through the thread pool, the resource consumption can be reduced, the response speed can be increased, the manageability of the threads can be improved, and the running efficiency of the program can be improved.

As shown in fig. 4, in an embodiment of the present invention, the step of starting the application program to call the thread in the thread pool may include the following steps:

step S401, the application program calls a thread from the thread pool through a thread running function.

The multithreading mode can maximize the computing power of the multicore computer, but if no control is performed, a burden is imposed on the system. To avoid creating threads repeatedly, the presence of a thread pool may allow threads to be multiplexed. Generally speaking, when work comes, a thread is taken from a thread pool, after the work is completed, the thread is not directly closed, but is returned to the thread pool for other tasks, an application program can call the thread from the thread pool to execute the task through a thread running function, and in one embodiment of the invention, data in a multithreading module is read by calling the thread in the thread pool.

Of course, the thread pool is set in the initialization function of the application program, and at the beginning of creating the thread pool, the thread pool is not provided with any thread by default. When a task comes, creating a thread execution task, and after the thread number reaches the Size of a core pool (corePool Size, which represents the maximum thread number allowed to run simultaneously in the thread pool), placing the arrived task in a task cache queue, and if the adding is successful, waiting for an idle thread to take out the task for execution; if the addition fails (typically the task cache queue is full), an attempt is made to create a new thread to execute the task; if the number of threads in the current thread Pool reaches the maximum Pool Size (maximum Pool Size), a task rejection policy is adopted for processing.

According to the reading method provided by the invention, the threads can run in parallel by setting the multithreading module and the corresponding calling function, so that a multithreading data reading mode is realized, and the data reading time can be effectively shortened.

As shown in fig. 5, in an embodiment of the present invention, before the step of starting the application program to invoke a thread in the thread pool, the method may further include the following steps:

step S501, setting a global interpreter lock in an application program, and controlling the running of threads in the application program through the global interpreter lock.

The Global Interpreter Lock is called Global Interpreter Lock, GIL for short.

The premise for current threads to want secure access to Python objects is to acquire a GIL to support multi-thread security. Without GIL, problems occur even with the simplest operations in a multi-threaded program. For example, two threads simultaneously increment the reference count of an object, which may only be incremented once instead of twice. Thus, there is one rule: only threads that have acquired the global interpreter lock can operate the Python object or call Python/C API (Application program interface) functions. To support multi-threaded Python programming, the interpreter regularly releases and reclaims locks, by default, once every 100 byte instruction set cycles (which may be set by the function sys. I/O slices like file reads and writes are also released and recycled with the GIL so that other threads can run while the thread requesting the I/O operation waits for the I/O operation to complete.

The execution of the Python code is controlled by a Python virtual machine (also called interpreter main loop, C Python version) which executes in a multi-threaded environment in the following manner:

1) setting GIL;

2) switching to a thread to run;

3) and operation:

a) a specified number of bytecode instructions, or;

b) thread active yield control (function time can be called. sleep (0));

4) setting the thread to be in a sleep state;

5) unlocking the GIL;

6) and repeating all the steps again.

As shown in fig. 6, in an embodiment of the present invention, after the step of the application program reading the data in the multi-thread module according to the multi-thread mode, the method may further include the following steps:

step S601, encapsulating a result obtained by reading data in the multithread module, and setting a call interface.

Reading data in a multi-threaded module typically requires subsequent operations, such as sorting the data. In order to facilitate the subsequent process to call the result, the result may be packaged (for example, packaged in an xml format, etc.), and a call interface is provided for others to call.

An interface may be understood as a more specific "class", and a method defined in the interface may be called using a variable of the interface type, and then the method of the upstream request and the downstream message may be encapsulated using the interface, so that only the method defined in the interface may be called. The callback interface may be used to transmit data and return the returned data to the appropriate location for processing.

The invention further provides an electronic device. Fig. 7 is a schematic diagram illustrating program modules of the electronic device 20 according to an exemplary embodiment of the invention.

The electronic device 20 includes:

a first setting module 201 adapted to set a multithread module and a thread inherited from the multithread module, in which a call function corresponding to the multithread module is set;

a second setting module 202 adapted to set a thread pool for storing the running threads in the application; and

the starting module 203 is adapted to start the application program to call the threads in the thread pool, so that the application program reads the data in the multithreading module according to the multithreading mode.

Further, the first setting module 201 is adapted to import a parameter related to a multithread module in the calling function; and setting an initialization function in the thread, and importing corresponding parameters into the thread through the initialization function.

Further, the starting module 203 is adapted to enable the application program to call a thread from the thread pool by a thread running function.

Further, the electronic device 20 further includes a third setting module, which is adapted to set a global interpreter lock in the application program, and control the running of the thread in the application program through the global interpreter lock.

Further, the electronic device 20 further includes an encapsulation module adapted to encapsulate a result obtained by reading the data in the multithreading module, and set a call interface.

The electronic device provided by the invention has the advantages that the threads can run in parallel by setting the multithreading module and the corresponding calling function, so that a multithreading data reading mode is realized, and the data reading time can be effectively shortened.

To achieve the above object, the present invention further provides a computer device 20, which comprises a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

To achieve the above object, the present invention further provides a computer device 20, which includes a memory 21, a processor 22, and a computer program stored on the memory 21 and executable on the processor 22, wherein the processor 22 implements the steps of the above method when executing the computer program. The computer program may be stored in the memory 24.

The present invention also provides a computer device, such as a smart phone, a tablet computer, a notebook computer, a desktop computer, a rack server, a blade server, a tower server or a rack server (including an independent server or a server cluster composed of a plurality of servers) capable of executing programs, and the like. The computer device of the embodiment at least includes but is not limited to: memory, processor, etc. communicatively coupled to each other via a system bus.

The present embodiment also provides a computer-readable storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, a server, an App application mall, etc., on which a computer program is stored, which when executed by a processor implements corresponding functions. The computer readable storage medium of the present embodiment is used for storing the electronic device 20, and when executed by the processor 22, the reading method of the present invention is implemented.

Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (such as a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present invention.

The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.

Claims

1. A method of reading, the method comprising the steps of:

2. The reading method of claim 1, wherein the step of providing a multi-threaded module further comprises:

defining classes of multi-threaded modules according to the projects; and

3. The reading method of claim 1, wherein the step of setting a calling function in the thread corresponding to the multi-threaded module further comprises:

4. The reading method of claim 1, wherein the step of launching the application to invoke a thread in the thread pool further comprises:

5. A reading method according to claim 4, wherein the thread pool is provided in an initialisation function of the application.

6. The reading method of claim 1, wherein the step of launching the application to invoke a thread in the thread pool is preceded by the step of:

7. The method of reading in accordance with claim 1, wherein the step of the application program reading data in the multi-threaded module according to the multi-threaded mode further comprises:

8. An electronic device, comprising:

9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the steps of the reading method according to any of claims 1 to 7 are implemented by the processor when executing the computer program.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that: the computer program when executed by a processor implements the steps of the reading method of any one of claims 1 to 7.