CN115994236B - Collaborative processing method and system for aviation data - Google Patents
Collaborative processing method and system for aviation data Download PDFInfo
- Publication number
- CN115994236B CN115994236B CN202310288970.5A CN202310288970A CN115994236B CN 115994236 B CN115994236 B CN 115994236B CN 202310288970 A CN202310288970 A CN 202310288970A CN 115994236 B CN115994236 B CN 115994236B
- Authority
- CN
- China
- Prior art keywords
- picture
- group
- index information
- slice
- pictures
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The invention discloses a collaborative processing method and a collaborative processing system for aviation data.A storage server forms continuous pictures acquired by the same camera into a picture group, a check picture is added, index information of an upper picture and a lower picture is added in a picture attribute field, and the index information is stored in different hard disks; the method comprises the steps of establishing and maintaining a picture group index file, wherein the index file comprises index information of a first picture and a check picture of a picture group, and attribute information such as source cameras and generation time of the first picture and a last picture of the picture group, so that the storage reliability is improved, and meanwhile, the information of the index file is reduced. When the number of hard disks is reduced, the restored pictures are independently generated into slice groups, and the original slice groups provide pointing information, so that the reliability and the retrievability of the restored pictures are still ensured. The invention ensures the reliability and retrievability of the stored pictures through the cooperative processing of the multiple hard disks.
Description
Technical Field
The application belongs to the technical field of data storage, and particularly relates to a cooperative processing method and system for aviation data.
Background
Along with the continuous expansion of the aviation market scale, the pressure of the aviation company for efficiently managing various resources is increased, and how to effectively allocate resources in complex and changeable environments, so that the operation management cost is reduced, and the problem to be solved by the aviation company is urgent.
Airline operations management systems are often required to collect various data, and reliable storage and management of data is one of the important issues that have always been addressed. The airline operation management system comprises a plurality of cameras mounted on the apron and used for shooting an airplane driving into a stand and generating an airplane picture. The picture can be used for identifying information such as airplane registration number, airplane type and the like and is used for operation management.
However, at present, the storage reliability of the picture data of the operation management system of the airline company depends on the technology of a hard disk layer, such as RAID, erasure codes and the like, special equipment support is needed, and the universality is relatively weak.
Disclosure of Invention
The purpose of the application is to provide a collaborative processing method and a collaborative processing system for aviation data, a reliable technology is provided from an application layer, reliable storage of picture data is realized, and the problem that the universality is poor due to the fact that the technology of a hard disk layer is needed in the prior art is solved.
In order to achieve the above purpose, the technical scheme of the application is as follows:
the collaborative processing method of aviation data is applied to a storage server, the storage server divides a managed hard disk into a first hard disk group for storing camera acquisition pictures and a reserved hard disk for storing check pictures, and divides a logic block space of each hard disk into continuous storage units with the same size, and the collaborative processing method of aviation data comprises the following steps:
establishing a picture group corresponding to the same camera and the pictures with the same size, determining the number of the pictures included in the picture group according to the number of the hard disks in the first hard disk group, and dividing the received collected pictures of the camera into the corresponding picture groups according to the picture size and the camera ID;
storing each picture in the picture group to different hard disks in the first hard disk group, and after all pictures in the picture group are stored to different hard disks in the first hard disk group, creating check pictures with the same picture size for the picture group and storing the check pictures in a reserved hard disk; adding index information of a next picture and index information of a last picture in an attribute field of each picture, wherein a last picture comprises index information of a check picture and index information of a penultimate picture, and the check picture comprises index information of a first picture and index information of the last picture;
and establishing and maintaining a picture group index file corresponding to each picture group, wherein the picture group index file comprises index information of a first picture and a check picture in the picture group.
Further, the collaborative processing method of aviation data further comprises the following steps:
receiving a request of a user for extracting a picture, and searching index information of a first picture corresponding to a picture group from a picture group index file;
judging whether the picture to be searched is the picture to be searched according to the index information of the first picture, outputting the first picture if the picture to be searched is the picture, otherwise, sequentially searching other pictures in the picture group and judging.
Further, the collaborative processing method of aviation data further comprises the following steps:
when one of the managed hard disks fails, each picture of the picture group is searched according to the picture group index file, and the pictures stored in the failed hard disk are restored according to the searched pictures.
Further, the collaborative processing method of aviation data further comprises the following steps:
when the number of hard disks managed by the storage server is increased, the original picture group is kept unchanged;
and determining the number of pictures included in the new picture group according to the increased number of hard disks, and dividing the received camera acquisition pictures into corresponding new picture groups according to the picture size and the camera ID.
Further, the collaborative processing method of aviation data further comprises the following steps:
when the number of the hard disks managed by the storage server is reduced by one, restoring the pictures stored by the reduced hard disks in the picture group;
dividing the restored pictures into slices according to the number of the residual hard disks and generating a check slice, so as to form a slice group, respectively storing all slices and check slices into each residual hard disk, adding index information of the next slice and index information of the last slice into an attribute field of each slice, wherein the last slice comprises index information of the check slice and index information of the last-last slice, and the check slice comprises index information of the first slice and index information of the last slice; the first slicing and checking slicing further comprises an original attribute field and a slicing mark of the restored picture;
modifying attribute fields of the front and rear pictures of the restored picture, modifying a start address in index information of a next picture in an attribute field of a next picture into a storage unit address where a first slice of a slice group is located and an offset of the slice relative to the storage unit address, and modifying the start address in index information of a previous picture in the attribute field of the previous picture into a storage unit address where the check slice is located and an offset of the slice relative to the unit address.
The application also provides a collaborative processing system of aviation data, including storage server and camera, the storage server divide into the first hard disk group that is used for storing camera collection picture and is used for storing the reservation hard disk of check-up picture with the hard disk of management to divide into continuous same size's storage unit with the logical block space of every hard disk, the storage server includes:
the image group processing module is used for establishing image groups corresponding to the same camera and the images with the same size, determining the number of the images included in the image groups according to the number of the hard disks in the first hard disk group, and dividing the received image collected by the camera into the corresponding image groups according to the image size and the camera ID;
the storage module is used for storing each picture in the picture group to different hard disks in the first hard disk group, and after all pictures in the picture group are stored to different hard disks in the first hard disk group, checking pictures with the same picture size are created for the picture group and stored in the reserved hard disk; adding index information of a next picture and index information of a last picture in an attribute field of each picture, wherein a last picture comprises index information of a check picture and index information of a penultimate picture, and the check picture comprises index information of a first picture and index information of the last picture;
the index file maintenance module is used for establishing and maintaining a picture group index file corresponding to each picture group, wherein the picture group index file comprises index information of a first picture and a check picture in the picture group.
Further, the storage server further includes:
the picture retrieval module is used for receiving a request of a user for extracting pictures and searching index information of a first picture corresponding to a picture group from a picture group index file; judging whether the picture to be searched is the picture to be searched according to the index information of the first picture, outputting the first picture if the picture to be searched is the picture, otherwise, sequentially searching other pictures in the picture group and judging.
Further, the storage server further includes:
and the picture recovery module is used for searching each picture of the picture group according to the picture group index file when one of the managed hard disks fails, and recovering the picture stored in the failed hard disk according to the searched picture.
Further, the storage server further includes:
the hard disk increase and decrease management module is used for keeping the original picture group unchanged when the number of the hard disks managed by the storage server is increased; and determining the number of pictures included in the new picture group according to the increased number of hard disks, and dividing the received camera acquisition pictures into corresponding new picture groups according to the picture size and the camera ID.
Further, the storage server further includes:
the hard disk increase and decrease management module is used for recovering pictures stored by the hard disks reduced in the picture group when the number of the hard disks managed by the storage server is reduced by one; dividing the restored pictures into slices according to the number of the residual hard disks and generating a check slice, so as to form a slice group, respectively storing all slices and check slices into each residual hard disk, adding index information of the next slice and index information of the last slice into an attribute field of each slice, wherein the last slice comprises index information of the check slice and index information of the last-last slice, and the check slice comprises index information of the first slice and index information of the last slice; the first slicing and checking slicing further comprises an original attribute field and a slicing mark of the restored picture; modifying attribute fields of the front and rear pictures of the restored picture, modifying a start address in index information of a next picture in an attribute field of a next picture into a storage unit address where a first slice of a slice group is located and an offset of the slice relative to the storage unit address, and modifying the start address in index information of a previous picture in the attribute field of the previous picture into a storage unit address where the check slice is located and an offset of the slice relative to the unit address.
The application provides a collaborative processing method and a collaborative processing system for aviation data, wherein a storage server forms continuous pictures with the same size acquired by the same camera into a picture group, check pictures are added, index information of upper and lower pictures is added in a picture attribute field, and the index information is stored in different hard disks; the method comprises the steps that a picture group index file is built and maintained, wherein the picture group index file only comprises index information of a first picture and a check picture of a picture group, and attribute information such as source cameras and generation time of the first picture and a last picture of the picture group, so that storage reliability is improved, and information of the index file is reduced. When the number of hard disks is reduced, the restored pictures are independently generated into slice groups, and the original slice groups provide pointing information, so that the reliability and the retrievability of the restored pictures are still ensured. According to the method and the device, the reliability and the retrievability of the stored pictures are guaranteed through the multi-hard-disk cooperative processing.
Drawings
Fig. 1 is a flowchart of a collaborative processing method for aviation data in the present application.
Fig. 2 is a schematic structural diagram of a storage server according to the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application.
In order to improve the picture storage efficiency, the storage server divides a logic block space of each hard disk into storage units with fixed sizes, and divides the managed hard disk into a first hard disk group for storing pictures collected by a camera and a reserved hard disk for storing check pictures. In order to improve the data recovery capability, a plurality of pictures with the same size are taken as a group according to the number of the hard disks, and check pictures with the same size are established, wherein each picture (including the check pictures) of the same group is distributed on different hard disks, and the check pictures are stored in reserved hard disks. The same group of pictures (except for the check picture) are ordered according to the time of picture generation, with the time of picture generation in between the first and last picture. Meanwhile, a picture group index file is established on the system disk, wherein the picture group index file comprises index information of a first picture and a check picture of a picture group. The attribute field of each picture comprises index information of the picture, index information of the next picture and index information of the previous picture, wherein the index information comprises a storage unit address where the picture is located, an offset relative to the storage unit address, a camera ID of the source and time information. The last picture contains index information of the check picture and the next to last picture, and the check picture contains index information of the first picture and the last picture, thereby forming a picture ring containing bi-directional pointers. When the hard disk where a certain picture is located fails, the system can restore the content of the picture based on other pictures on the ring. And further supporting the automatic adjustment of the expansion of the number of the hard disks and the reduction of the number of the effective hard disks, thereby realizing the reliable storage method of the multi-hard disk collaborative processing.
In one embodiment, as shown in fig. 1, a collaborative processing method of aviation data is proposed, where a hard disk managed by a storage server is divided into a first hard disk group for storing a camera acquisition picture and a reserved hard disk for storing a verification picture, and a logic block space of each hard disk is divided into continuous storage units with the same size. The cooperative processing method of aviation data comprises the following steps:
step S1, establishing a picture group corresponding to the same camera and the pictures with the same size, determining the number of the pictures included in the picture group according to the number of the hard disks in the first hard disk group, and dividing the received camera acquisition pictures into the corresponding picture groups according to the picture size and the camera ID.
In an airline operations management system, there are a plurality of tarmac mounted cameras for taking pictures of an aircraft entering a stand and generating a picture of the aircraft. The pictures can be used for identifying information such as airplane registration numbers, airplane types and the like. The camera generates a large number of pictures, and the pictures acquired by the camera are sent to a storage server for storage through iscsi protocol. The storage server divides at least the managed hard disk into a first hard disk group for storing the camera-captured pictures and a reserved hard disk for storing the verification pictures.
The storage server automatically sets the number of pictures included in the picture group according to the managed number of hard disks. If the number of the hard disks is N, the picture group can be set to be N-1 pictures, one hard disk is reserved for placing the check pictures of the picture group, and the first hard disk group is N-1 hard disks. Of course, the picture can be set to be N-2, a hard disk for placing the check picture is reserved, and a redundant hard disk is provided to prevent a certain hard disk in the picture group from malfunctioning, and the redundant hard disk can be quickly used as a replacement of a failure hard disk. Likewise, the group of pictures can be set to N-3, so that more hard disks can provide redundant backup.
In this embodiment, it is possible that the sizes of the pictures collected by different cameras are different, or that the same camera collects pictures of different sizes. The storage server establishes a picture group corresponding to the same camera and the same-size pictures, so that pictures collected by different cameras are placed in different picture groups, and pictures with different sizes are also placed in different picture groups. For example, the camera 1 and the camera 2 respectively acquire pictures, and the sizes of the acquired pictures are two, namely a size a and a size B. Four picture groups are to be established for putting in the acquisition pictures of the camera 1 size a, the acquisition pictures of the camera 1 size B, the acquisition pictures of the camera 2 size a and the acquisition pictures of the camera 2 size B, respectively. In this application, each image group corresponds to the same size image collected by the same camera, which is not described here in detail.
After receiving the pictures collected by the camera, the storage server divides the received pictures collected by the camera into different picture groups according to the picture size and the camera ID.
According to the storage server, a large number of hard disks are managed, the storage server divides the logic block space of each hard disk into continuous storage units with the same size, and the space of one storage unit can accommodate a plurality of pictures so as to establish a picture index file.
Step S2, storing each picture in the picture group to different hard disks in the first hard disk group, and after all pictures in the picture group are stored to different hard disks in the first hard disk group, creating check pictures with the same picture size for the picture group and storing the check pictures in a reserved hard disk; and adding index information of the next picture and index information of the last picture in an attribute field of each picture, wherein the last picture comprises index information of a check picture and index information of the last-last picture, and the check picture comprises index information of the first picture and index information of the last picture.
After the pictures collected by the camera are put into the picture group, the pictures can be stored into different hard disks in the first hard disk group in real time. Each picture in the picture group should be stored in a different hard disk to avoid that two pictures in the picture group are lost and cannot recover data due to failure of one hard disk. In specific implementation, a corresponding relation between the picture serial numbers in the picture group and the hard disk serial numbers can be established, and each picture in the picture group is stored in a corresponding hard disk during storage, so that each picture is stored in a different hard disk.
After the pictures in the picture group are stored in the hard disk, based on the stored information of the pictures, generating a check picture with the same size according to a RAID5 algorithm, and storing the check picture in the reserved hard disk.
Each picture comprises picture data and an attribute field, the picture attribute field comprises index information of the picture, and the embodiment adds index information of the next picture and index information of the previous picture in the attribute field. The index information of the picture contains the address of the storage unit where it is located, the offset from the address of the storage unit, the camera ID, and time information. One storage unit can store a plurality of pictures, the offset of the first picture is 0, the offset of the second picture is the picture size, and so on.
The last picture contains index information of a check picture and a next to last picture, and the check picture contains index information of the first picture (as "next picture") and the last picture (as "last picture"), thereby forming an image slice ring containing bi-directional pointers.
Step S3, establishing and maintaining a picture group index file corresponding to each picture group, wherein the picture group index file comprises index information of a first picture and a check picture in the picture group.
The storage server stores and maintains a picture group index file corresponding to each picture group in a system disk, the system disk needs to be protected in a key way based on RAID and other means, and the system disk does not store pictures.
The group index file contains index information of the first picture and the check picture of each group of pictures so that the storage server can locate the first picture and the check picture of the group of pictures at one time by indexing.
After the storage of the pictures is realized, the storage server can realize the picture extraction and the image restoration based on the technical scheme of the picture storage, and the specific description is given below through specific embodiments.
In a specific embodiment, the method for collaborative processing of aviation data further includes:
receiving a request of a user for extracting a picture, and searching index information of a first picture corresponding to a picture group from a picture group index file;
judging whether the picture to be searched is the picture to be searched according to the index information of the first picture, outputting the first picture if the picture to be searched is the picture, otherwise, sequentially searching other pictures in the picture group and judging.
Specifically, when the storage server receives a request for extracting a picture from a user, the request for extracting the picture from the user searches a certain picture according to the source camera or/and the picture generation time. After receiving the request, the storage server searches the corresponding picture group of the picture according to the source camera or/and the picture generation time information from the picture group index file, and the picture group index file comprises the index information of the first picture, wherein the index information comprises the camera ID and the time information, so that the picture group index file corresponding to the request is easily searched.
And then, a first picture is obtained from the corresponding picture group, the first picture is extracted, and whether the requirements are met or not, namely whether the camera ID and the time information are met or not is judged from the attribute field of the picture. If not, continuing to find the starting address of the second picture from the index information of the next picture of the first picture, extracting the second picture, and judging whether the attribute field of the second picture meets the requirement. If the first picture is not matched with the second picture, the third picture is extracted according to the starting address of the third picture until a proper picture is found, but the check picture is not extracted.
In a specific embodiment, the method for collaborative processing of aviation data further includes:
when one of the managed hard disks fails, each picture of the picture group is searched according to the picture group index file, and the pictures stored in the failed hard disk are restored according to the searched pictures.
In this embodiment, when a certain hard disk fails, a manager changes the hard disk, so as to trigger the storage server to start image restoration, and the image restoration process is as follows:
the storage server searches a first picture start address of each picture group from a picture group index file of the system disk, and finds a start address of a next picture based on index information of the next picture of the first picture, and sequentially searches each picture. If the hard disk where a certain picture is found to be damaged and the picture data is lost, the storage server searches the starting address of the check picture of the picture group from the picture group index file, and finds the starting address of the last picture from the picture index information of the last picture of the check picture, and sequentially searches each picture. Finally, other pictures of the picture group except for the picture on the damaged hard disk can be aligned, and the damaged picture of the picture group can be restored based on the pictures.
If the storage server finds that the hard disk where the first picture of the picture group is located is damaged, the storage server searches the starting address of the check picture of the picture from the picture group index file, and finds the starting address of the last picture from the last picture index information of the check picture, and sequentially searches each picture. Finally, other pictures except the first picture on the damaged hard disk can be aligned, and damaged pictures of the picture group can be restored based on the pictures.
The specific restoration method for restoring the picture based on the verification picture and other pictures in the picture group is a relatively mature technology in the art, and will not be described herein.
In a specific embodiment, the method for collaborative processing of aviation data further includes:
when the number of hard disks managed by the storage server is increased, the original picture group is kept unchanged;
and determining the number of pictures included in the new picture group according to the increased number of hard disks, and dividing the received camera acquisition pictures into corresponding new picture groups according to the picture size and the camera ID.
Specifically, when the number of hard disks managed by the storage server increases, the original group of pictures does not need to be changed in order to save consumption. The method for setting the number of pictures is the same as the original method by only adjusting the number of pictures contained in the new picture group. And then, when a new picture is received, the new picture is stored according to the same method, and the description is omitted here.
In another specific embodiment, the method for collaborative processing of aviation data further includes:
when the number of the hard disks managed by the storage server is reduced by one, restoring the pictures stored by the reduced hard disks in the picture group;
dividing the restored pictures into slices according to the number of the residual hard disks and generating a check slice, so as to form a slice group, respectively storing all slices and check slices into each residual hard disk, adding index information of the next slice and index information of the last slice into an attribute field of each slice, wherein the last slice comprises index information of the check slice and index information of the last-last slice, and the check slice comprises index information of the first slice and index information of the last slice; the first slicing and checking slicing further comprises an original attribute field and a slicing mark of the restored picture;
modifying attribute fields of the front and rear pictures of the restored picture, modifying a start address in index information of a next picture in an attribute field of a next picture into a storage unit address where a first slice of a slice group is located and an offset of the slice relative to the storage unit address, and modifying the start address in index information of a previous picture in the attribute field of the previous picture into a storage unit address where the check slice is located and an offset of the slice relative to the unit address.
Specifically, when the number of hard disks managed by the storage server is reduced, the hard disk needs to be reduced one by one. The arrangement is started once every time one hard disk is reduced, so that data loss can be avoided.
Taking the example of removing one hard disk, starting the finishing process as follows:
the storage server searches a first picture start address of each picture group from a picture group index file of the system disk, and finds a start address of a next picture based on index information of the next picture of the first picture, and sequentially searches each picture. When the hard disk where a certain picture is located disappears and the picture data is lost, the storage server searches the starting address of the check picture of the picture group from the picture group index file, and finds the starting address of the last picture from the picture index information of the last picture of the check picture, and sequentially searches each picture. Finally, the pictures except the picture on the removed hard disk can be aligned, and the picture on the removed hard disk can be restored based on the pictures. Assuming that the recovered picture sequence number after removal is S, the last picture sequence number is S-1 and the next picture sequence number is s+1.
And slicing the restored picture S, wherein the slicing number is reduced by 1 based on the current hard disk number M, namely M-1. Dividing the picture S into M-1 fragments, and establishing check fragments, wherein the sizes of the fragments are the same, and the fragments are distributed on different hard disks, so that a fragment group related to the picture S is formed. The attribute field of each slice contains index information of the next slice and index information of the last slice, the index information also contains a unit address of the slice and offset relative to the unit address, the last image slice contains index information of the check slice and the last slice, and the check slice contains index information of the first slice and the last image slice, so that an image slice ring containing bidirectional pointers is formed. When a hard disk where a slice is located fails, the system can restore the contents of the slice based on other slices on the ring.
And meanwhile, modifying index information of 'next picture' in an attribute field of the picture with the serial number of S-1, and modifying a starting address in the index information into a unit address where a first slice of a slice group of the S image is located and an offset of the slice relative to the unit address. And modifying index information of a 'previous picture' with a serial number of S+1 picture, and modifying a starting address in the index information into a unit address where the check fragment is located and an offset of the fragment relative to the unit address.
And, the first slice of the slice group of S pictures and the check slice each contain all attribute fields of S pictures, and a slice flag is added, indicating that this is the first slice of M-1 slices or the check slice. When the S picture is wanted to be searched, the attribute information in the first slice and the check slice is found to contain the slice marks, so that the fact that all M-1 slices should be searched is known, and then merging is carried out to recover the complete picture S.
By the design, the storage reliability of the restored picture is guaranteed, namely, any hard disk disappears, and the picture can be restored through the fragments and the verification information on other hard disks. Meanwhile, the generation time period information of the original image group is not used as any change, so that the retrievability is ensured.
In another specific embodiment, the storage server may maintain a "list of available units" in the system disk that holds information for all storage units that have space remaining. The information comprises the number of the hard disk where the storage unit is located, the address of the storage unit, the residual space size of the storage unit, the residual space address segment and the like. When the remaining space of a certain memory cell is 0, the memory cell is deleted from the maintenance list. At the same time, the storage server adds a "used" flag for storage units that already have data but have non-zero remaining space.
According to this principle, the storage server first checks all storage units with "used" flags and accommodated picture data from the "available unit list", and if there is no satisfactory storage unit, then selects an empty storage unit without "used" flag. Selecting a storage unit meeting the requirements as a storage unit for storing pictures, and marking the storage unit with a used mark if the storage unit is not used before use; if the remaining space of the storage unit after the addition of the first picture is 0, the storage unit is deleted from the "available unit list".
Corresponding to the method, the application also provides a collaborative processing system of aviation data, which comprises a storage server and a camera, wherein the storage server divides a managed hard disk into a first hard disk group for storing pictures collected by the camera and a reserved hard disk for storing check pictures, and divides a logic block space of each hard disk into continuous storage units with the same size, and the storage server comprises:
the image group processing module is used for establishing image groups corresponding to the same camera and the images with the same size, determining the number of the images included in the image groups according to the number of the hard disks in the first hard disk group, and dividing the received image collected by the camera into the corresponding image groups according to the image size and the camera ID;
the storage module is used for storing each picture in the picture group to different hard disks in the first hard disk group, and after all pictures in the picture group are stored to different hard disks in the first hard disk group, checking pictures with the same picture size are created for the picture group and stored in the reserved hard disk; adding index information of a next picture and index information of a last picture in an attribute field of each picture, wherein a last picture comprises index information of a check picture and index information of a penultimate picture, and the check picture comprises index information of a first picture and index information of the last picture;
the index file maintenance module is used for establishing and maintaining a picture group index file corresponding to each picture group, wherein the picture group index file comprises index information of a first picture and a check picture in the picture group.
In a specific embodiment, the storage server further includes:
the picture retrieval module is used for receiving a request of a user for extracting pictures and searching index information of a first picture corresponding to a picture group from a picture group index file; judging whether the picture to be searched is the picture to be searched according to the index information of the first picture, outputting the first picture if the picture to be searched is the picture, otherwise, sequentially searching other pictures in the picture group and judging.
In a specific embodiment, the storage server further includes:
and the picture recovery module is used for searching each picture of the picture group according to the picture group index file when one of the managed hard disks fails, and recovering the picture stored in the failed hard disk according to the searched picture.
In a specific embodiment, the storage server further includes:
the hard disk increase and decrease management module is used for keeping the original picture group unchanged when the number of the hard disks managed by the storage server is increased; and determining the number of pictures included in the new picture group according to the increased number of hard disks, and dividing the received camera acquisition pictures into corresponding new picture groups according to the picture size and the camera ID.
In a specific embodiment, the storage server further includes:
the hard disk increase and decrease management module is used for recovering pictures stored by the hard disks reduced in the picture group when the number of the hard disks managed by the storage server is reduced by one; dividing the restored pictures into slices according to the number of the residual hard disks and generating a check slice, so as to form a slice group, respectively storing all slices and check slices into each residual hard disk, adding index information of the next slice and index information of the last slice into an attribute field of each slice, wherein the last slice comprises index information of the check slice and index information of the last-last slice, and the check slice comprises index information of the first slice and index information of the last slice; the first slicing and checking slicing further comprises an original attribute field and a slicing mark of the restored picture; modifying attribute fields of the front and rear pictures of the restored picture, modifying a start address in index information of a next picture in an attribute field of a next picture into a storage unit address where a first slice of a slice group is located and an offset of the slice relative to the storage unit address, and modifying the start address in index information of a previous picture in the attribute field of the previous picture into a storage unit address where the check slice is located and an offset of the slice relative to the unit address.
The above examples merely represent a few embodiments of the present application, which are described in more detail and are not to be construed as limiting the scope of the invention. It should be noted that it would be apparent to those skilled in the art that various modifications and improvements could be made without departing from the spirit of the present application, which would be within the scope of the present application. Accordingly, the scope of protection of the present application is to be determined by the claims appended hereto.
Claims (10)
1. The collaborative processing method of aviation data is applied to a storage server, and is characterized in that the storage server divides a managed hard disk into a first hard disk group for storing pictures collected by a camera and a reserved hard disk for storing check pictures, and divides a logic block space of each hard disk into continuous storage units with the same size, and the collaborative processing method of aviation data comprises the following steps:
establishing a picture group corresponding to the same camera and the pictures with the same size, determining the number of the pictures included in the picture group according to the number of the hard disks in the first hard disk group, and dividing the received collected pictures of the camera into the corresponding picture groups according to the picture size and the camera ID;
storing each picture in the picture group to different hard disks in the first hard disk group, and after all pictures in the picture group are stored to different hard disks in the first hard disk group, creating check pictures with the same picture size for the picture group and storing the check pictures in a reserved hard disk; adding index information of a next picture and index information of a last picture in an attribute field of each picture, wherein a last picture comprises index information of a check picture and index information of a last-to-last picture, and the check picture comprises index information of a first picture and index information of the last picture;
and establishing and maintaining a picture group index file corresponding to each picture group, wherein the picture group index file comprises index information of a first picture and a check picture in the picture group.
2. The method for collaborative processing of aviation data according to claim 1, wherein the method for collaborative processing of aviation data further comprises:
receiving a request of a user for extracting a picture, and searching index information of a first picture corresponding to a picture group from a picture group index file;
judging whether the picture to be searched is the picture to be searched according to the index information of the first picture, outputting the first picture if the picture to be searched is the picture, otherwise, sequentially searching other pictures in the picture group and judging.
3. The method for collaborative processing of aviation data according to claim 1, wherein the method for collaborative processing of aviation data further comprises:
when one of the managed hard disks fails, each picture of the picture group is searched according to the picture group index file, and the pictures stored in the failed hard disk are restored according to the searched pictures.
4. The method for collaborative processing of aviation data according to claim 1, wherein the method for collaborative processing of aviation data further comprises:
when the number of hard disks managed by the storage server is increased, the original picture group is kept unchanged;
and determining the number of pictures included in the new picture group according to the increased number of hard disks, and dividing the received camera acquisition pictures into corresponding new picture groups according to the picture size and the camera ID.
5. The method for collaborative processing of aviation data according to claim 3, further comprising:
when the number of the hard disks managed by the storage server is reduced by one, restoring the pictures stored by the reduced hard disks in the picture group;
dividing the restored pictures into slices according to the number of the residual hard disks and generating a check slice, so as to form a slice group, respectively storing all slices and check slices into each residual hard disk, adding index information of the next slice and index information of the last slice into an attribute field of each slice, wherein the last slice comprises index information of the check slice and index information of the last-last slice, and the check slice comprises index information of the first slice and index information of the last slice; the first slicing and checking slicing further comprises an original attribute field and a slicing mark of the restored picture;
modifying attribute fields of the front and rear pictures of the restored picture, modifying a start address in index information of a next picture in an attribute field of a next picture into a storage unit address where a first slice of a slice group is located and an offset of the slice relative to the storage unit address, and modifying the start address in index information of a previous picture in the attribute field of the previous picture into a storage unit address where the check slice is located and an offset of the slice relative to the unit address.
6. The utility model provides a collaborative processing system of aviation data, includes storage server and camera, its characterized in that, storage server divide into the first hard disk group that is used for storing the camera collection picture and is used for storing the reservation hard disk of check-up picture with managed hard disk to divide into continuous same size's storage unit with the logical block space of every hard disk, storage server includes:
the image group processing module is used for establishing image groups corresponding to the same camera and the images with the same size, determining the number of the images included in the image groups according to the number of the hard disks in the first hard disk group, and dividing the received image collected by the camera into the corresponding image groups according to the image size and the camera ID;
the storage module is used for storing each picture in the picture group to different hard disks in the first hard disk group, and after all pictures in the picture group are stored to different hard disks in the first hard disk group, checking pictures with the same picture size are created for the picture group and stored in the reserved hard disk; adding index information of a next picture and index information of a last picture in an attribute field of each picture, wherein a last picture comprises index information of a check picture and index information of a last-to-last picture, and the check picture comprises index information of a first picture and index information of the last picture;
the index file maintenance module is used for establishing and maintaining a picture group index file corresponding to each picture group, wherein the picture group index file comprises index information of a first picture and a check picture in the picture group.
7. The system for collaborative processing of aviation data according to claim 6, wherein the storage server further comprises:
the picture retrieval module is used for receiving a request of a user for extracting pictures and searching index information of a first picture corresponding to a picture group from a picture group index file; judging whether the picture to be searched is the picture to be searched according to the index information of the first picture, outputting the first picture if the picture to be searched is the picture, otherwise, sequentially searching other pictures in the picture group and judging.
8. The system for collaborative processing of aviation data according to claim 6, wherein the storage server further comprises:
and the picture recovery module is used for searching each picture of the picture group according to the picture group index file when one of the managed hard disks fails, and recovering the picture stored in the failed hard disk according to the searched picture.
9. The system for collaborative processing of aviation data according to claim 6, wherein the storage server further comprises:
the hard disk increase and decrease management module is used for keeping the original picture group unchanged when the number of the hard disks managed by the storage server is increased; and determining the number of pictures included in the new picture group according to the increased number of hard disks, and dividing the received camera acquisition pictures into corresponding new picture groups according to the picture size and the camera ID.
10. The system for collaborative processing of aviation data according to claim 8, wherein the storage server further comprises:
the hard disk increase and decrease management module is used for recovering pictures stored by the hard disks reduced in the picture group when the number of the hard disks managed by the storage server is reduced by one; dividing the restored pictures into slices according to the number of the residual hard disks and generating a check slice, so as to form a slice group, respectively storing all slices and check slices into each residual hard disk, adding index information of the next slice and index information of the last slice into an attribute field of each slice, wherein the last slice comprises index information of the check slice and index information of the last-last slice, and the check slice comprises index information of the first slice and index information of the last slice; the first slicing and checking slicing further comprises an original attribute field and a slicing mark of the restored picture; modifying attribute fields of the front and rear pictures of the restored picture, modifying a start address in index information of a next picture in an attribute field of a next picture into a storage unit address where a first slice of a slice group is located and an offset of the slice relative to the storage unit address, and modifying the start address in index information of a previous picture in the attribute field of the previous picture into a storage unit address where the check slice is located and an offset of the slice relative to the unit address.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310288970.5A CN115994236B (en) | 2023-03-23 | 2023-03-23 | Collaborative processing method and system for aviation data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310288970.5A CN115994236B (en) | 2023-03-23 | 2023-03-23 | Collaborative processing method and system for aviation data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115994236A CN115994236A (en) | 2023-04-21 |
| CN115994236B true CN115994236B (en) | 2023-08-04 |
Family
ID=85995336
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310288970.5A Active CN115994236B (en) | 2023-03-23 | 2023-03-23 | Collaborative processing method and system for aviation data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115994236B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112463019A (en) * | 2019-09-06 | 2021-03-09 | 华为技术有限公司 | Data reading method and device |
| CN115809027A (en) * | 2023-01-11 | 2023-03-17 | 杭州广科安德生物科技有限公司 | Biological data acquisition and management system, device and method |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN100530125C (en) * | 2007-08-24 | 2009-08-19 | 成都索贝数码科技股份有限公司 | Safety storage method for data |
| US9104560B2 (en) * | 2012-06-13 | 2015-08-11 | Caringo, Inc. | Two level addressing in storage clusters |
| CN111984200B (en) * | 2020-08-28 | 2024-04-12 | 大连大学 | Mass video data storage system |
-
2023
- 2023-03-23 CN CN202310288970.5A patent/CN115994236B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112463019A (en) * | 2019-09-06 | 2021-03-09 | 华为技术有限公司 | Data reading method and device |
| CN115809027A (en) * | 2023-01-11 | 2023-03-17 | 杭州广科安德生物科技有限公司 | Biological data acquisition and management system, device and method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115994236A (en) | 2023-04-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN100562858C (en) | The methods, devices and systems of EMS memory data-base remote disaster tolerance | |
| CN106843750B (en) | Distributed storage system | |
| US7350043B2 (en) | Continuous data protection of block-level volumes | |
| CN104462563A (en) | File storage method and system | |
| US6934725B1 (en) | Management of file extent mapping to hasten mirror breaking in file level mirrored backups | |
| CN109407977B (en) | Big data distributed storage management method and system | |
| US20070162513A1 (en) | Methods and apparatus for point in time data access and recovery | |
| CN109542330B (en) | Data storage method, data query method and device | |
| JP2009507278A (en) | Search and restore data objects | |
| CN104067239A (en) | Systems and methods for data chunk deduplication | |
| CN107092686B (en) | File management method and device based on cloud storage platform | |
| CN111831485B (en) | Data recovery method, device, electronic equipment and medium | |
| CN104461773A (en) | Backup deduplication method of virtual machine | |
| US8543864B2 (en) | Apparatus and method of performing error recovering process in asymmetric clustering file system | |
| CN109491832A (en) | Data disaster tolerance method and website | |
| CN103716384A (en) | Method and device for realizing cloud storage data synchronization in cross-data-center manner | |
| CN109828951A (en) | A kind of method and system of combination Kubernetes and Ceph storage snapshot | |
| CN101937378B (en) | Method for carrying out back-up protection on data of storage equipment and computer system | |
| CN115994236B (en) | Collaborative processing method and system for aviation data | |
| CN104407942A (en) | Off-site storage based Linux operation system backup recovery method | |
| CN115098447A (en) | File recovery method and device, electronic equipment and readable storage medium | |
| CN110737389B (en) | Method and device for storing data | |
| CN103176867A (en) | Fast file differential backup method | |
| CN115809027B (en) | Biological data acquisition and management system, device and method | |
| CN106919478A (en) | A kind of Website page element backup method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |