CN114047943A - Information processing method, apparatus, device, storage medium and computer program product - Google Patents

Abstract

The present disclosure provides an information processing method, apparatus, device, storage medium and computer program product, and relates to the field of computer technology, in particular to the field of software development. The specific implementation scheme is as follows: obtain at least one item of file information and version information of a specified version in the remote repository; archive the at least one item of information to obtain an archive package of the specified version; the type of the archive package of the specified version is The first type or the second type, the archive package of the first type carries the file information of the specified version, and the archive package of the second type carries the file information and version information of the specified version; send the archive package of the specified version to the third-party storage system ; a third-party storage system for storing archive packages. The technical solution of the present disclosure can archive the file information of the specified version and at least one item of version information in the remote version library and store it in a third-party storage system, so as to reduce the read and write pressure of the remote version library and improve the overall read and write performance and throughput.

Description

Information processing method, apparatus, device, storage medium and computer program product

technical field

The present disclosure relates to the field of computer technology, in particular to the field of software development, and in particular to an information processing method, apparatus, device, storage medium and computer program product.

Background technique

The current practice of DevOps (the collective term for processes, methods and systems) has been widely used in the field of software research and development. In DevOps practice scenarios, Git (distributed version control system) is currently the most popular code hosting tool, and every developer can Copy a complete Git repository from a remote location locally by git clone.

The existing source code distribution method needs to obtain the source code from the remote Git repository. Due to the particularity of the Git protocol, the main methods for obtaining the source code from the remote Git repository are git clone and git pull/fetch. Both methods need to create a Git repository locally, and the amount of data to be pulled is large, and all historical versions of the specified branch are included by default. In various DevOps practice scenarios such as compilation, construction, automated testing, release, and launch, the read and write performance of a single Git repository will drop sharply in the case of high concurrency, especially for some large-scale Git repositories, the read success rate There will be a significant decline, and the continuous failure and retries will further increase the read and write pressure on the remote Git repository, resulting in a vicious circle, further lengthening the time to download the source code, and also affecting the performance of other push requests.

SUMMARY OF THE INVENTION

The present disclosure provides an information processing method, apparatus, device, storage medium, and computer program product.

According to a first aspect of the present disclosure, an information processing method is provided, comprising:

Obtain at least one of the file information and version information of the specified version in the remote repository;

The at least one item of information is archived to obtain an archive package of a specified version; the type of the archive package of the specified version is the first type or the second type, the archive package of the first type carries the file information of the specified version, and the archive package of the second type is of the first type or the second type. The archive package carries the file information and version information of the specified version;

Send the archive package of the specified version to the third-party storage system; the third-party storage system is used to store the archive package.

According to a second aspect of the present disclosure, there is provided an information processing apparatus, comprising:

an information acquisition module, used to acquire at least one item of text information and version information of a specified version in the remote repository;

An archiving module, configured to archive the at least one item of information to obtain an archive package of a specified version; the type of the archive package of the specified version is the first type or the second type, and the archive package of the first type carries the file information of the specified version , the second type of archive package carries the file information and version information of the specified version;

The archive package storage module sends the archive package of the specified version to the third-party storage system; the third-party storage system is used to store the archive package.

According to a third aspect of the present disclosure, there is provided an electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the information processing method provided by any embodiment of the present disclosure.

According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause a computer to execute the information processing method provided by any embodiment of the present disclosure.

According to a fifth aspect of the present disclosure, there is provided a computer program product, including a computer program, which when executed by a processor implements the information processing method provided by any embodiment of the present disclosure.

The technical solution of the present disclosure can achieve at least the following beneficial effects:

The technical solution of the present disclosure can archive at least one item of file information and version information of a specified version in a remote version library, and store the archived package obtained after archiving in a third-party storage system, and the third-party storage system can store the archived package. , to provide users with archive packages when needed, and then provide corresponding version information, which can reduce the read and write pressure on the remote repository and improve the overall read and write performance and throughput; archive packages can carry version information of a specified version or not. The version information of the specified version can be applied to scenarios that need to restore the version information of the original version or scenarios that do not need to restore the version information of the original version to meet different usage requirements.

It should be understood that what is described in this section is not intended to identify key or critical features of embodiments of the disclosure, nor is it intended to limit the scope of the disclosure. Other features of the present disclosure will become readily understood from the following description.

Description of drawings

The accompanying drawings are used for better understanding of the present solution, and do not constitute a limitation to the present disclosure. in:

1 is a schematic flowchart of an information processing method provided by an embodiment of the present disclosure;

2 is a schematic diagram of the principle of obtaining a specified version based on a Git shallow bundle in an embodiment of the present disclosure;

3 is a schematic partial flowchart of another information processing method provided by an embodiment of the present disclosure;

4 is a schematic diagram of an example of an information processing method provided by an embodiment of the present disclosure;

5 is a schematic structural framework diagram of an information processing apparatus provided by an embodiment of the present disclosure;

6 is a schematic structural framework diagram of an address generation module in an embodiment of the present disclosure;

FIG. 7 is a schematic structural framework diagram of an electronic device provided by an embodiment of the present disclosure.

Detailed ways

Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding and should be considered as exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted from the following description for clarity and conciseness.

First, several terms involved in the present disclosure are introduced as follows:

DevOps: A combination of Development and Operations, a collective term for a set of processes, methods, and systems used to facilitate communication, collaboration, and integration between development (application/software engineering), technical operations, and quality assurance departments. DevOps requires the adoption of highly standardized and automated processes and tools to support the efficient collaboration of various roles, so as to achieve the goal of faster, faster, more frequent and more reliable software delivery.

Git: An open source distributed version control system that can handle version management of small to very large projects efficiently and at high speed.

Cloud-native: An approach to building and running applications that take advantage of the cloud computing model. Cloud-native technology enables organizations to build and run elastically scalable applications in new and dynamic environments such as public, private, and hybrid clouds . Representative technologies of cloud native include containers, service mesh, microservices, immutable infrastructure and declarative API (Application Programming Interface). These techniques enable the construction of loosely coupled systems that are fault-tolerant, easy to manage, and easy to observe. Combined with reliable automation, cloud-native technologies make it easy for engineers to make frequent and predictable breaking changes to the system.

In the cloud-native era, DevOps itself also widely adopts technical architecture systems such as large-scale distributed construction and large-scale distributed continuous integration/delivery in the context of large-scale automated processes and tools. In the cloud-native era, containerization is a very important manifestation. The challenges brought by containerization to source code hosting mainly come from statelessness and high-concurrency and high-performance access.

The traditional source code distribution method mainly creates Git warehouses locally through two methods: git clone and git pull/fetch. What is really needed is the source code of a specific version, and the transmission and download of a large number of irrelevant version history records will occupy computing resources and reduce R&D efficiency. In addition, in large-scale DevOps scenarios, a large amount of unnecessary data reading and transmission will result in low source code download efficiency and waste of computing resources and network resources. Therefore, the traditional source code distribution method cannot meet the actual requirements of large-scale DevOps practice for high concurrency and high performance.

There are mainly the following solutions to this problem:

The first solution is to add read nodes on the remote server to cope with the high concurrent access pressure to the source code of large-scale DevOps automation tasks.

The second solution is to increase the source code cache on the local client to relieve the reading pressure on the remote server.

Solution 3: Upgrade the Git protocol on the remote server to pull a single version to reduce unnecessary waste of resources.

But the above scheme has the following problems:

In solution 1, there is a delay in data synchronization between the master node and the read node, and the data synchronization is poor. At the same time, limited by the principle of Git itself and the characteristics of processing a large number of small files in practice, a single Git service instance can support The number of concurrency is very limited.

Solution 2 is not suitable for the cloud-native era. Services running in containers in the cloud-native era usually mount third-party storage services. Therefore, due to the fact that the Git workspace can only have a view corresponding to a given version at any given time, the services in the container It is not possible to see multiple versions of the view in the same workspace at the same time, so it is impossible to use multiple versions of the source code in the same space at the same time.

Solution 3 cannot obtain the specified version and cannot completely solve the high concurrent read problem of the remote Git repository. First, changes to the remote Git repository may occur at any time, and the client cannot know the depth of the clone or the time of the version, so it cannot accurately obtain the specified version; secondly, the third solution must be from the latest commit information of the version, that is, the branch head pointer (HEAD) The commit information (commit) pointed to starts to obtain the source code of the remote Git repository. In order to obtain the source code of a certain version, all the commit information from the current latest commit information back to the specified commit information (specified commit) must be obtained together, so it is impossible to obtain the source code of the remote Git repository. Completely solve the high concurrent read problem of the remote Git repository, but there is still a lot of waste of resources.

According to an embodiment of the present disclosure, the present disclosure provides an information processing method, which can be applied to a Git server or other servers. As shown in FIG. 1 , the method includes:

S101: Acquire at least one item of file information and version information of a specified version in a remote repository.

The remote version repository in the embodiment of the present disclosure may be a Git repository of a remote Git server, which is hereinafter referred to as a remote Git repository.

In the embodiment of the present disclosure, the file information of the specified version may include information of at least one of a text file (ie, a source code file) and a binary file, and the version information of the specified version may include a history record of the specified version.

Optionally, the operation of obtaining the file information and/or version information of the specified version in the remote repository can be realized through the archiving task construction service, or through other services, and the corresponding service can be set according to the actual situation. The embodiment does not limit this.

S102: Archive the file information of the specified version and at least one item of version information to obtain an archive package of the specified version.

The archive package of the specified version is of the first type or the second type. The archive package of the first type carries the file information of the specified version, and the archive package of the second type carries the file information and version information of the specified version.

In the scenario of archiving based on the first type, the file information of the specified version can be archived to obtain the archive package of the first type of the specified version. In the scenario of archiving based on the second type, the file information of the specified version can be archived. And the version information is archived to obtain the second type of archive package of the specified version. Both the first type of archive package and the second type of archive package can be used to restore (restore through file information) files of a specified version, and the second type of archive package can also be used to restore (restore through version information) version records of a specified version.

Optionally, the operation of archiving the file information and/or version information of a specified version can be implemented through an archiving task processing service, or through other services, and a corresponding service can be set according to the actual situation. This is not limited.

S103: Send an archive package of a specified version to a third-party storage system.

A third-party storage system is used to store archived packages.

Optionally, the operation of sending the specified version of the archive package to the third-party storage system can be implemented through the archive storage and distribution service, or through other services, and the corresponding service can be set according to the actual situation. This is not limited.

The technical solution of the present disclosure can archive at least one item of file information and version information of a specified version in a remote version library, and store the archived package obtained after archiving in a third-party storage system, and the third-party storage system can store the archived package. , to provide users with archive packages when needed, and then provide corresponding version information, which can reduce the read and write pressure on the remote repository and improve the overall read and write performance and throughput; archive packages can carry version information of a specified version or not. The version information of the specified version can be applied to scenarios that need to restore the version information of the original version or scenarios that do not need to restore the version information of the original version to meet different usage requirements.

The third-party storage system in the embodiment of the present disclosure can be set on a remote Git server or other servers, and the third-party storage system is used to store and deliver archive packages, which can relieve the read and write pressure of the remote repository in high concurrency scenarios. Eliminate single-point dependencies on remote repositories.

In an optional implementation manner, acquiring at least one item of file information and version information of a specified version in the remote repository includes: in the case of a version change in the remote repository, acquiring the changed version The version of the file information and at least one item of version information is used as at least one item of information in the specified version of the file information and version information.

Optionally, whether there is a version change in the remote repository can be determined by monitoring the version in the remote repository or polling, or by other methods, and the remote version can be determined. Check if the version in the library has changed.

In one example, the archiving task construction service can monitor the remote repository to determine whether the source code signal of each version in the remote repository has changed. When the file information of multiple versions has changed, it is considered that the one or more versions have changed. The archive task construction service obtains the basic information of the one or more changed versions (as the specified version), and creates an archive task based on the basic information. , and send the archiving task to the next service, such as the archiving task processing service; the archiving task processing service can obtain at least one of the file information and version information of the changed version from the remote repository, and the obtained information can be based on The type of file that actually needs to be archived is determined. Wherein, the basic information of the changed version may include the submission information of the changed version.

In one example, the archiving task construction service may poll the remote repository, and periodically check the change log of the remote repository to determine whether the file information of each version in the remote repository has changed; When the file information of one or more versions in the remote repository has changed, it is considered that the one or more versions have changed, and the archive task build service can obtain one or more changed versions in the remote repository (as the specified version). ), create an archive task based on the basic information, and send the archive task to the next service, such as the archive task processing service; the archive task processing service can obtain the file information and version information of the changed version from the remote repository At least one item of information, the acquired information can be determined according to the type of archive that actually needs to be archived. Wherein, the basic information of the changed version may include the submission information of the changed version. In one example, the change version has the specified commit information.

The above implementation of determining version changes by monitoring or polling and then creating an archiving task is an active archiving method, which can obtain version change information in real time. When the changes of the local client are sent to the remote repository, the archiving task The build service can quickly monitor the change, and then respond quickly to create an archiving task, which helps to improve the efficiency of source code archiving.

In another optional implementation manner, acquiring at least one item of file information and version information of a specified version in the remote repository includes: in response to an archive instruction sent by the client, acquiring the remote file specified by the archive instruction At least one item of file information and version information of the version in the version repository of the terminal is used as at least one item of information of the file information and version information of the specified version.

Optionally, the archive instruction carries specified commit information, and the specified commit information can be used to determine one or more versions in the remote repository specified by the archive instruction.

In one example, after receiving the archiving instruction sent by the client, the archiving task construction service obtains the specified commit information carried by the archiving instruction, and obtains one or more specified commit information from the remote repository according to the specified commit information. Basic information of a version (as a specified version), create an archive task based on the basic information, and send the archive task to the next service, such as the archive task processing service, which can obtain the changed files from the remote repository. At least one item of the file information of the version and the version information, the acquired information can be determined according to the type of archive that actually needs to be archived. The basic information of the version specified by the filing instruction may include submission information.

The method of creating an archiving task based on the archiving instruction sent by the client is a passive archiving method, which can realize the archiving of the source code in a targeted manner based on the actual needs of the user.

Based on the above content, the method of obtaining at least one of the file information and version information of the specified version in the remote repository in the embodiment of the present disclosure can be applied to both the application scenario of active archiving and the application scenario of passive archiving , which is more flexible and can meet different archiving needs.

Optionally, the first type may be Git archive, which is used to archive file information. The archive package of Gitarchive only corresponds to the complete file content of a specified version, and does not have any information for restoring the specified version.

In an optional implementation, the second type does not support Git's shallow clone protocol, which may be referred to as Git shallow clone for short. For example, the second type may be a Git bundle, which is used for archiving file information and version information. The archive package of the Git bundle carries the necessary information for restoring the native file of the specified version. The required Git object (object), through the Git bundle archive, can incrementally add new submission information to the existing submission information history of the local Git warehouse, thereby reducing the need for the complete submission information history request of the remote Git warehouse .

In another optional implementation, the second type supports Git's shallow clone protocol. For example, the second type may be Git shallow bundle, a special binary file format used to implement shallow clone compatible archive of file information and version information. The archive package of Git shallow bundle carries the necessary information for restoring the native file of the specified version. The archive package of Git shallow bundle contains the version content that needs to be archived, including version reference (ref), commit information, tree (tree), object ( object), etc.

The embodiments of the present disclosure provide archive packages of several types (eg, Git archive, Git bundle, and Git shallowbundle), which can realize one-time calculation and permanent storage, thereby reducing a large number of unnecessary repetitions caused by frequent reading and writing of remote repository It can reduce the waste of resources such as CPU (Central Process Unit, central processing unit), memory, network, etc., and can provide more user read and write support under the same server resources, so as to alleviate the high load of the remote version library. Read and write pressure in concurrent scenarios.

The above-mentioned several types of archive packages provided by the embodiments of the present disclosure can be adapted to various storage services, so that the archive packages can be stored in various read-optimized storage services, and distributed through CDN (Content Delivery Network, Content Delivery Network). ) or other means for read acceleration, which can effectively improve the user's access speed to the file information and/or version information read operation, and ensure the user's read success rate. In the actual DevOps scenario, the traffic has obvious characteristics of more reads and fewer writes. The above solutions that can be adapted to various read-optimized storage services can effectively guarantee the success rate of DevOps in the stage of obtaining file information and/or version information. Reduce DevOps time-consuming in the acquisition phase of file information and/or version information.

The second type in the embodiment of the present disclosure may be a type that supports the shallow clone protocol of Git, for example, a Gitshallow bundle, which can provide one or more specified versions in a remote repository, which can well satisfy users' high-speed, high-speed, Read exactly the requirements of any subrepository in the remote repository.

In a large number of R&D scenarios, especially in the CI (Continuous Integration, continuous integration)/CD (Continuous Delivery, continuous delivery) process, a large amount of historical submission information (historical commit) of the remote repository has no practical effect on the execution of actual tasks. , the user just wants to obtain the file information and/or version information of the specified version in the remote repository, and check, scan, build, compile, publish, deploy and other processes for the file information and/or version information. These common processes are usually only related to specific versions of file information and/or version information. The reading and transmission of a large number of historical records is a huge waste of computing and network resources, and also restricts the development of large-scale DevOps practices. Git shallow bundle supports the native Git shallow clone feature, and users can get the latest commit information from the remote end by executing the pull (git pull/fetch) command.

In addition, the traditional solution is usually directly based on Git shallow clone to speed up the acquisition of the commit information of the remote repository. This scheme can only obtain the latest commit information on the specified branch through the --depth<depth> parameter <depth>+1 historical commit information , or through --shallow-since<date> to get the commit information after a certain time, but through --depth<depth> and --shallow-since<date> can not accurately get the depth and version time defect. Based on the Git shallow bundle solution in the embodiment of the present disclosure, the user can natively obtain the Git in the remote repository that contains the specified commit information (which can be composed of any number of commit information required by the user) Shallowclone the local repository file, that is, any subversion of the remote repository, makes up for the shortcomings of the native Git shallow clone, enhances the flexibility of Git in reading the subversion history, and is more suitable for large-scale DevOps.

Referring to FIG. 2 , the solution for obtaining a specified version by means of Git shallow bundle in the embodiment of the present disclosure will be further introduced below.

The first line of Figure 2 shows the version of the remote repository in a branch, including the version of the head pointer of the branch, the version containing commit information 1 (commit 1), and the version containing commit information 2 (commit 2) . The second row of Figure 2 shows a schematic diagram of locally obtaining the version containing commit 2 through Git shallow clone, and the third row of Figure 2 shows a schematic diagram of locally obtaining the version containing commit 2 and commit 1 through Git shallow clone , as can be seen from the schematic diagrams in the second and third lines, when locally obtaining the file information and/or version information of the specified version by means of Git shallow clone, it is necessary to obtain the latest commit information from the branch, that is, the head pointer (HEAD) The commit information pointed to starts backtracking to all commit information of the specified commit information (commit 2 and/or commit 1), with unnecessary backtracking.

The fourth row of FIG. 2 shows a schematic diagram of locally obtaining a version containing commit2 by means of the Git bundle in the embodiment of the present disclosure, and the fifth row of FIG. 2 shows the local acquisition of the version containing commit2 by means of the Git bundle in the embodiment of the present disclosure. The schematic diagrams of versions 2 and commit 1, as can be seen from the schematic diagrams in the fourth and fifth lines, when locally obtaining the file information and/or version information of the specified version through Git bundle, the latest commit information from the branch is also required. That is, the commit information pointed to by the head pointer (HEAD) starts to trace back to all the commit information carried by the Git bundle archive itself, and when the version of the specified commit information (commit 2 and/or commit 1) is used locally, the local repository must A prerequisite for the normal use of the Git bundle archive for the version with the historical commit information of the specified commit information (commit 2 and/or commit 1), otherwise the Git bundle archive cannot be used normally.

The sixth row of FIG. 2 shows a schematic diagram of locally obtaining the version containing commit 2 through the Git shallow bundle in the embodiment of the present disclosure, and the seventh row of FIG. 2 shows the local method of using the Git shallow bundle in the embodiment of the present disclosure. A schematic diagram of obtaining a version including commit 2 and commit 1, as can be seen from the schematic diagrams in the sixth and seventh lines, the file information and/or file information of the specified version can be directly obtained through the Git shallow bundle method in the embodiment of the present disclosure. Or version information, there is no need to backtrack one by one from the latest commit information of the branch, which can greatly reduce unnecessary backtracking and reduce the waste of resources. At the same time, the version of the specified commit information (commit 2 and/or commit 1) can be used locally. There are versions of the historical commit information for the specified commit information (commit 2 and/or commit 1) in the local repository.

In an optional implementation manner, as shown in FIG. 3 , archiving at least one of the file information and version information of a specified version includes the following steps S301-S303:

S301 , adding initial information of the second type to the first file to be archived formed by the file information of the specified version.

The second type of initial information includes format information for archiving file information and version information, such as information in bundle format.

In an example, the first file to be archived may be a .pack file, which contains file information of a specified version, and the second type of initial information may be information in a bundle format, and bundles may be introduced in the file header of the .pack file format information.

S302, if the specified version has the specified commit information, add the specified commit information of the specified version to the second to-be-archived file supporting the shallow clone protocol of Git.

In one example, the second file to be archived may be a .git/shallow file, that is, a file named shallow in the .git directory of the remote Git repository, and the file supports the shallow clone protocol of Git, such as SHA1 (Secure HashAlgorithm 1, The secure hash algorithm 1) hash value, MD5 (Message-Digest Algorithm 5, Message Digest Algorithm 5) hash value, etc., may also be other types of key information, which are not listed one by one in the embodiments of the present disclosure.

In one example, the SHA1 hash value (as the specified commit information) can be written into the .git/shallow file, so that the locally restored Git repository can allow the local repo with only one or more specified commit information. The root node of the repository has no parent commit.

S303: Archive the first to-be-archived file and the second to-be-archived file after the information is added.

Optionally, the above steps S301-S303 can be implemented by an archiving task processing service, which can perform archiving through the above steps S301-S303, and send the obtained archive package to the next service, such as an archive storage and distribution service.

The archive package obtained after archiving in the above way can be delivered to the recipient through file transmission methods such as network transmission, storage medium transmission, etc., and the recipient can restore it locally to contain the specified submission information (the specified submission information can be specified by the user. A local shallow clone version file of any number of commit messages.

In an optional implementation manner, the information processing method provided by the embodiment of the present disclosure further includes:

In response to the archive package access request sent by the client, the archive package access address is generated according to the storage address of the archive package requested by the archive package access request in the third-party storage system; the archive package access address is fed back to the client.

Optionally, the archive package access request sent by the client carries specified submission information, one or more versions specified by the specified submission information, and the one or more versions of the archive package are the ones requested by the archive package access request. Archive the package.

The above method of generating and feeding back the access address of the archive package based on the client-based access request for the archive package can be implemented by the archive storage and distribution service. After the archive storage and distribution service generates the access address of the archive package in the above manner, the archive package access address is fed back to the client. The client can download the corresponding archive package from the third-party storage system through the archive package access address. The third-party storage system can send the corresponding archive package to the client through the CDN, so as to improve the speed and success rate of the user reading the archive package.

The archive package access address in the embodiment of the present disclosure may be a temporary access address or a permanent access address. When the archive package access address is a temporary access address, the validity period of the access address may be set according to actual requirements.

Based on the above method, the user can obtain the archive package stored in the third-party storage system through the archive package access address, and obtain the file information and/or version information of the corresponding version, without relying on the remote version repository for read and write operations to obtain the version of the file. information and/or version information, which can reduce the read and write pressure on the remote repository and improve the read and write performance and efficiency.

In an optional implementation manner, the process of generating the archive access address in response to the archive package access request sent by the client specifically includes:

Perform permission authentication on the archive package access request; if the permission authentication for the archive package access request is passed, send the storage address of the archive package requested by the archive package access request to the client in the third-party storage system, and generate the archive package address.

The operation of performing authority authentication on the archive package access request can be implemented through authentication and audit services, or through other services, and corresponding services can be set according to actual conditions, which are not limited in the embodiments of the present disclosure.

Optionally, the archive package access request sent by the client carries user information, and the user information can be used to implement authority authentication for the archive package access request.

In one example, after receiving the archive package access request sent by the client, the archive storage and distribution service sends the archive package access request to the authentication and audit service, and the authentication and audit service authenticates the user information carried in the archive package access request Whether it has the permission to access the archive package in the third-party storage system, and whether the permission is consistent with the permission to access the version in the remote repository, if it is authenticated that the user information carried in the access request for the archive package has access to the archive package in the third-party storage system If the permission is the same as the permission to access the version in the remote repository, the permission authentication of the access request to the archive package passes, otherwise the permission authentication for the access request to the archive package fails.

Optionally, in the case that the authorization authentication for the archive package access request fails, the authentication failure information may be fed back to the client. In one example, in the case that the authorization authentication for the access request to the archive package fails, the authentication and audit service feeds back the authentication failure information to the archive storage and distribution service, and the archive storage and distribution service feeds back the authentication failure information to the client . In one example, after each authorization authentication is completed, the authentication and auditing service may store a record of this authorization authentication for subsequent use.

Based on the above-mentioned implementation manner, the technical solutions of the embodiments of the present disclosure can perform authority control on the access request of the archive package of the client, so as to ensure that the sending user has the access authority to the archive package in the third-party storage system, so as to improve the security of the access; Make sure that the user's access rights to archived packages in the third-party storage system are the same as the access rights to the version in the remote repository to improve the effectiveness of access.

The authentication and auditing services in the embodiments of the present disclosure can be bypassed, decoupling the read/write and access control of the archive package, and can be accessed in a low-cost manner without changing the original access control design architecture. Among all kinds of Git hosting services, it has extremely high practicability and compatibility. Even if the authentication and audit services fail, it will not affect the user's read request, which improves the stability of the user's read operation.

In an optional implementation manner, performing authority authentication on the archive package access request, including:

Authenticate the archive package access request through any valid authority authentication channel among the multiple authority authentication channels.

In an example, authentication and auditing services can be set in multiple computer rooms or network segments, and the authentication and auditing services in each computer room or network segment serve as an authorization authentication channel. When the authentication and auditing services of a computer room or network segment appear In the event of a fault (that is, the authority authentication channel is invalid), the traffic of authority authentication can be quickly switched to the authentication and audit services of other non-faulty computer rooms or network segments (that is, the authority authentication channel is valid).

Based on the above embodiments, the setting of multiple authority authentication channels is helpful for disaster recovery switching. When a certain authority authentication channel fails, it can quickly switch to other valid authority authentication channels, which can improve the stability of authority authentication, and prevent certain The failure of the authorization authentication channel interrupts authorization authentication, so as to ensure that the normal read requests of users are not affected.

FIG. 4 shows a schematic diagram of implementing the information processing method provided by the present disclosure through various services. An example of the information processing method provided by the present disclosure will be described below with reference to FIG. 4 .

Referring to the example shown in FIG. 4 , the information processing method provided by the present disclosure can be implemented by services such as archive task construction service, archive task processing service, archive storage and distribution service, authentication and audit service, etc., specifically involving the following processes;

When the file information of the software version changes on the local client, the client sends the change to the remote Git repository through the Git write operation.

The archive task construction service obtains the specified version (the version specified by the archive command or the changed version of the remote Git warehouse) when the version specified by the archive command sent by the client or the version changed in the remote Git warehouse is determined. Basic information, create an archive task based on the basic information, and send the created archive task to the archive task processing service.

The archiving task processing service obtains one or more specified versions of file information and at least one item of version information based on the archiving task, wherein, for the archiving task based on the first type, the file information of the specified version can be obtained, and for the file information based on the second type. type of archiving task, obtain the file information and version information of the specified version; package at least one item of the file information and version information of each specified version to generate a corresponding binary file, and use the binary file as an archive package or The binary compressed file obtained by compressing the binary file is used as an archive package, an archive package upload task is created, and the archive package and the archive package upload task are sent to the archive storage and distribution service.

The archive storage and distribution service writes archive packages to third-party storage systems for storage based on archive package upload tasks.

When the user needs to obtain the archive package in the third-party storage system, the client sends an archive package access request (Git read operation), and the archive storage and distribution service sends the archive package access request to the authentication and audit service. The archive package access request is authenticated, the authentication result is fed back to the archive storage and distribution service, and the archive storage and distribution service is fed back to the client. In the case that the authority authentication fails, the fed back authentication result may include the reason for the authentication failure, the suggestion for re-authentication, etc. FIG. 4 only shows the feedback archive package access address as an example when the authority authentication is passed.

The client reads the corresponding archive package in the third-party storage system through the archive package access address.

The above process shown in FIG. 4 can be processed asynchronously with the remote Git repository, so as to reduce the impact of the above process on the read and write performance of the remote Git repository.

In Figure 4, the client's read and write requests can be implemented through the client-side extension. The client-side extension is mainly responsible for encapsulating request operations such as construction, download, and restoration, which can reduce the cost of user learning and use. When applied to the scenario of Git hosting service, the client extension is the Git client extension. The Git client extension is mainly provided according to the command when the user executes the Git clone (clone) or pull (fetch/pull) command. The information is converted into a request for obtaining an archive package (that is, an archive package access request) to access the archive storage and distribution service. After completing the authorization authentication and feeding back the archive package access address, the Git client extension downloads the archive package according to the archive package access address; if the download is successful , decompress the archive and restore it to the native local Git repository and checkout the workspace; if the download fails, it will automatically transfer to the native Git client to perform native Git clone or pull ( fetch/pull) commands. This method can make the use of the client more convenient and transparent.

According to an embodiment of the present disclosure, the present disclosure further provides an information processing apparatus. As shown in FIG. 5 , the apparatus includes: an information acquisition module 501 , an archive module 502 and an archive package storage module 503 .

The information acquisition module 501 is configured to acquire at least one item of information among file information and version information of a specified version in a remote repository.

The archiving module 502 is used to archive at least one item of information in the file information of the specified version and the version information to obtain the archived package of the specified version; the type of the archived package of the specified version is the first type or the second type, the first type The archive package of the first type carries the file information of the specified version, and the archive package of the second type carries the file information and version information of the specified version.

The archive package storage module 503 sends an archive package of a specified version to a third-party storage system; the third-party storage system is used for storing the archive package.

In an optional implementation, the second type supports a shallow clone protocol for distributed version control systems.

In an optional embodiment, the information acquisition module 501 is specifically configured to: in the case of a version change in the remote version repository, obtain at least one item of file information and version information of the changed version, as at least one item of file information and version information of the specified version; or, in response to the archive instruction sent by the client, obtain at least one of the file information and version information of the version in the remote repository specified by the archive instruction Item information, as at least one item of file information and version information of the specified version.

In an optional implementation manner, the archiving module 502 is specifically configured to: add initial information of the second type to the first to-be-archived file formed by the file information of the specified version; in the case that the specified version has the specified submission information, Add the specified submission information of the specified version to the second to-be-archived file that supports the shallow clone protocol; archive the first to-be-archived file and the second to-be-archived file after the information is added; wherein the second type of initial information includes the file Format information for archiving information and version information.

In an optional implementation manner, the information processing apparatus provided by the embodiment of the present disclosure further includes: an address generation module and an address feedback module.

The address generation module is configured to, in response to the archive package access request sent by the client, generate the archive package access address according to the storage address of the archive package requested by the archive package access request in the third-party storage system.

The address feedback module is used to feed back the archive package access address to the client.

In an optional implementation manner, as shown in FIG. 6 , the address generation module includes: an authority authentication unit 601 and an address generation unit 602 .

The authority authentication unit 601 is configured to perform authority authentication on the archive package access request.

The address generation unit 602 is configured to send, to the client, the storage address of the archived package requested by the archived package access request in the third-party storage system, and generate the archived package access address in the case of passing the authorization authentication of the archived package access request .

In an optional implementation manner, the authority authentication unit 601 is specifically configured to: perform authority authentication on the archive package access request through any valid authority authentication channel among multiple authority authentication channels.

For the functions of the units or modules in the apparatuses in the embodiments of the present disclosure, reference may be made to the corresponding descriptions in the foregoing method embodiments, and details are not described herein again.

In the technical solution of the present disclosure, the acquisition, storage and application of the user's personal information involved are all in compliance with the provisions of relevant laws and regulations, and do not violate public order and good customs.

According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a non-transitory computer-readable storage medium, and a computer program product.

An electronic device provided by an embodiment of the present disclosure includes: at least one processor, and a memory connected in communication with the at least one processor; wherein, the memory stores an instruction that can be executed by the at least one processor, and the instruction is executed by the at least one processor , so that at least one processor can execute any information processing method provided by the embodiments of the present disclosure.

The non-transitory computer-readable storage medium provided by the embodiments of the present disclosure stores computer instructions thereon, and the computer instructions are used to cause the computer to execute any one of the information processing methods provided by the embodiments of the present disclosure.

The computer program product provided by the embodiments of the present disclosure includes a computer program, which, when executed by a processor, implements any one of the information processing methods provided by the embodiments of the present disclosure.

FIG. 7 shows a schematic block diagram of an example electronic device 700 that may be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are by way of example only, and are not intended to limit implementations of the disclosure described and/or claimed herein.

As shown in FIG. 7 , the device 700 includes a computing unit 701 that can be executed according to a computer program stored in a read only memory (ROM) 702 or loaded into a random access memory (RAM) 703 from a storage unit 708 Various appropriate actions and handling. In the RAM 703, various programs and data necessary for the operation of the device 700 can also be stored. The computing unit 701 , the ROM 702 , and the RAM 703 are connected to each other through a bus 704 . An input/output (I/O) interface 705 is also connected to bus 704 .

Various components in the device 700 are connected to the I/O interface 705, including: an input unit 706, such as a keyboard, mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a magnetic disk, an optical disk, etc. ; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, and the like. The communication unit 709 allows the device 700 to exchange information/data with other devices through a computer network such as the Internet and/or various telecommunication networks.

Computing unit 701 may be various general-purpose and/or special-purpose processing components with processing and computing capabilities. Some examples of computing units 701 include, but are not limited to, central processing units (CPUs), graphics processing units (GPUs), various specialized artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processing processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 executes the various methods and processes described above, such as an information processing method. For example, in some embodiments, an information processing method may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as storage unit 708 . In some embodiments, part or all of the computer program may be loaded and/or installed on device 700 via ROM 702 and/or communication unit 709 . When the computer program is loaded into the RAM 703 and executed by the computing unit 701, one or more steps of the information processing method described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform the information processing method by any other suitable means (eg, by means of firmware).

Various implementations of the systems and techniques described herein above may be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips system (SOC), load programmable logic device (CPLD), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include being implemented in one or more computer programs executable and/or interpretable on a programmable system including at least one programmable processor that The processor, which may be a special purpose or general-purpose programmable processor, may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device an output device.

Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer or other programmable data processing apparatus, such that the program code, when executed by the processor or controller, performs the functions/functions specified in the flowcharts and/or block diagrams. Action is implemented. The program code may execute entirely on the machine, partly on the machine, partly on the machine and partly on a remote machine as a stand-alone software package or entirely on the remote machine or server.

In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in connection with the instruction execution system, apparatus or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media may include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media would include one or more wire-based electrical connections, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), fiber optics, compact disk read only memory (CD-ROM), optical storage, magnetic storage, or any suitable combination of the foregoing.

To provide interaction with a user, the systems and techniques described herein may be implemented on a computer having a display device (eg, a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user ); and a keyboard and pointing device (eg, a mouse or trackball) through which a user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (eg, visual feedback, auditory feedback, or tactile feedback); and can be in any form (including acoustic input, voice input, or tactile input) to receive input from the user.

The systems and techniques described herein may be implemented on a computing system that includes back-end components (eg, as a data server), or a computing system that includes middleware components (eg, an application server), or a computing system that includes front-end components (eg, a user computer having a graphical user interface or web browser through which a user may interact with implementations of the systems and techniques described herein), or including such backend components, middleware components, Or any combination of front-end components in a computing system. The components of the system may be interconnected by any form or medium of digital data communication (eg, a communication network). Examples of communication networks include: Local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.

A computer system can include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server arises by computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, a distributed system server, or a server combined with blockchain.

It should be understood that steps may be reordered, added or deleted using the various forms of flow shown above. For example, the steps described in the present disclosure can be executed in parallel, sequentially, or in different orders. As long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, there is no limitation herein.

The above-mentioned specific embodiments do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present disclosure should be included within the protection scope of the present disclosure.

Claims (17)

1. An information processing method comprising:

acquiring at least one item of information in the file information and the version information of a specified version in a remote version library;

archiving the at least one item of information to obtain an archiving package of the specified version; the type of the archive packet of the specified version is a first type or a second type, the archive packet of the first type carries the file information of the specified version, and the archive packet of the second type carries the file information and the version information of the specified version;

sending the archive package of the specified version to a third-party storage system; the third-party storage system is used for storing the archive package.

2. The information processing method according to claim 1, wherein the second type supports a shallow clone protocol of a distributed version control system.

3. The information processing method of claim 2, wherein the archiving the at least one item of information comprises:

adding the initial information of the second type in a first file to be archived, which is formed by the file information of the specified version; the second type of initial information includes format information for archiving file information and version information;

under the condition that the specified version has specified submission information, adding the specified submission information of the specified version in a second file to be archived, which supports the shallow clone protocol;

and archiving the first file to be archived and the second file to be archived after the information is added.

4. The information processing method according to any one of claims 1 to 3, wherein the acquiring at least one of version information and file information of a specified version in a remote version repository includes:

acquiring at least one item of information of file information and version information of a version which is changed under the condition that the version is changed in the remote version library, wherein the at least one item of information is used as at least one item of information of the file information and the version information of the specified version;

or, in response to an archive instruction sent by a client, acquiring at least one of file information and version information of a version in the remote version library specified by the archive instruction as the at least one of file information and version information of the specified version.

5. The information processing method according to any one of claims 1 to 3, further comprising:

responding to an archive package access request sent by a client, and generating an archive package access address according to a storage address of an archive package requested to be accessed by the archive package access request in the third-party storage system;

and feeding back the access address of the archive package to the client.

6. The information processing method according to claim 5, wherein the generating an archive package access address according to a storage address of the archive package requested to be accessed by the archive package access request in the third-party storage system comprises:

performing authority authentication on the archive package access request;

and under the condition that the authority authentication of the archive package access request passes, sending the storage address of the archive package requested to be accessed by the archive package access request in the third-party storage system to the client to generate an archive package access address.

7. The information processing method of claim 6, wherein the performing rights authentication on the archive package access request comprises:

and performing authority authentication on the access request of the archive package through any effective authority authentication channel in a plurality of authority authentication channels.

8. An information processing apparatus comprising:

the information acquisition module is used for acquiring at least one item of information in the text information and the version information of the specified version in the remote version library;

the filing module is used for filing the at least one item of information to obtain a filing package of the specified version; the type of the archive packet of the specified version is a first type or a second type, the archive packet of the first type carries the file information of the specified version, and the archive packet of the second type carries the file information and the version information of the specified version;

the archive package storage module is used for sending the archive package of the specified version to a third-party storage system; the third-party storage system is used for storing the archive package.

9. The information processing apparatus according to claim 8, wherein the second type supports a shallow clone protocol of a distributed version control system.

10. The information processing apparatus according to claim 9, wherein the archive module is specifically configured to:

adding the initial information of the second type in a first file to be archived, which is formed by the file information of the specified version; under the condition that the specified version has specified submission information, adding the specified submission information of the specified version in a second file to be archived, which supports the shallow clone protocol; archiving the first file to be archived and the second file to be archived after information is added;

the second type of initial information includes format information that archives file information and version information.

11. The information processing apparatus according to any one of claims 8 to 10, wherein the information acquisition module is specifically configured to:

acquiring at least one item of information of file information and version information of a version which is changed under the condition that the version is changed in the remote version library, wherein the at least one item of information is used as at least one item of information of the file information and the version information of the specified version; or, in response to an archive instruction sent by a client, acquiring at least one of file information and version information of a version in the remote version library specified by the archive instruction as the at least one of file information and version information of the specified version.

12. The information processing apparatus according to any one of claims 8 to 10, further comprising:

the address generation module is used for responding to an archive package access request sent by a client and generating an archive package access address according to the storage address of an archive package requested to be accessed by the archive package access request in the third-party storage system;

and the address feedback module is used for feeding back the access address of the archive package to the client.

13. The information processing apparatus according to claim 12, wherein the address generation module includes:

the authority authentication unit is used for carrying out authority authentication on the access request of the filing package;

and the address generating unit is used for sending the storage address of the archive package requested to be accessed by the archive package access request in the third-party storage system to the client side to generate the archive package access address under the condition that the authority authentication of the archive package access request passes.

14. The information processing apparatus according to claim 13, wherein the authority authentication unit is specifically configured to:

and performing authority authentication on the access request of the archive package through any effective authority authentication channel in a plurality of authority authentication channels.

15. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the information processing method of any one of claims 1 to 7.

16. A non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the information processing method according to any one of claims 1 to 7.

17. A computer program product comprising a computer program which, when executed by a processor, implements an information processing method according to any one of claims 1 to 7.

Images