WO2019128698A1 - 辅助安检方法、装置和系统 - Google Patents

辅助安检方法、装置和系统 Download PDF

Info

Publication number
WO2019128698A1
WO2019128698A1 PCT/CN2018/120291 CN2018120291W WO2019128698A1 WO 2019128698 A1 WO2019128698 A1 WO 2019128698A1 CN 2018120291 W CN2018120291 W CN 2018120291W WO 2019128698 A1 WO2019128698 A1 WO 2019128698A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
standard
dimensional model
scanned image
suspicious
Prior art date
Application number
PCT/CN2018/120291
Other languages
English (en)
French (fr)
Inventor
刘必成
张浩然
徐光明
王琪
朱强强
沃远
Original Assignee
同方威视技术股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 同方威视技术股份有限公司 filed Critical 同方威视技术股份有限公司
Publication of WO2019128698A1 publication Critical patent/WO2019128698A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01V—GEOPHYSICS; GRAVITATIONAL MEASUREMENTS; DETECTING MASSES OR OBJECTS; TAGS
    • G01V5/00—Prospecting or detecting by the use of ionising radiation, e.g. of natural or induced radioactivity
    • G01V5/20—Detecting prohibited goods, e.g. weapons, explosives, hazardous substances, contraband or smuggled objects
    • G01V5/22—Active interrogation, i.e. by irradiating objects or goods using external radiation sources, e.g. using gamma rays or cosmic rays
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—3D [Three Dimensional] image rendering
    • G06T15/005—General purpose rendering architectures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G06T19/006—Mixed reality
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2200/00—Indexing scheme for image data processing or generation, in general
    • G06T2200/04—Indexing scheme for image data processing or generation, in general involving 3D image data

Definitions

  • the present disclosure relates to the field of security inspection, and in particular to an auxiliary security inspection method, apparatus and system.
  • X-ray fluoroscopic images are quite different from natural light images. X-ray fluoroscopic images of different angles of various types of goods are not well known to most people, so the person who is looking at the map must master a large amount of commodity knowledge through experience accumulation or training. The fluoroscopic image under X-ray has a certain pre-judgment in order to effectively carry out the investigation work.
  • an auxiliary security method comprising: acquiring registration information of a test object; acquiring a standard scan image corresponding to the registration information; displaying the standard scan image in an augmented reality AR manner to pass the comparison standard
  • the scanned image and the actual scanned image determine whether the object is a suspicious object; wherein the actual scanned image includes an image of the object to be inspected.
  • the method further includes: acquiring at least one of a three-dimensional model and descriptive information corresponding to the registration information; and at least one of the three-dimensional model and the descriptive information corresponding to the registration information in an AR manner display.
  • the method further includes: acquiring an actual scanned image marked with a suspicious area, wherein if the detected object is a suspicious object, a position of the image of the detected object in the actual scanned image is marked as a suspicious area; The actual scanned image marked with the suspicious area is displayed as AR on the container in which the object is to be placed.
  • the method further comprises: displaying at least one of the standard scan image, the three-dimensional model, and the descriptive information corresponding to the registration information in an AR manner on the container on which the object is to be detected.
  • the method further comprises: acquiring at least one of a standard scan image, a three-dimensional model, and descriptive information of the suspicious object; at least one of a standard scan image, a three-dimensional model, and descriptive information of the suspicious object Displayed in AR mode.
  • the method further comprises: performing image processing on at least one of a standard scan image corresponding to the registration information and a standard scan image of the suspicious object in response to the image processing instruction, and processing the processed standard scan image Displayed in AR mode.
  • the method further includes: performing at least one of a zooming and rotating operation on at least one of the three-dimensional model corresponding to the registration information and the three-dimensional model of the suspicious object in response to the model processing instruction, and The 3D model after the model operation is displayed in the AR manner.
  • the standard scanned image is a multi-angle standard X-ray scan image.
  • an auxiliary security device comprising: an input interface configured to acquire registration information of a test object; a processor configured to acquire a standard scan image corresponding to the registration information; and a display And configured to display a standard scan image of the test object in an augmented reality AR manner to determine whether the test object is a suspicious object by comparing the standard scan image and the actual scan image, wherein the actual scan image includes the test object image.
  • the processor is further configured to acquire at least one of a three-dimensional model and descriptive information corresponding to the registration information; the display is further configured to at least one of a three-dimensional model and descriptive information corresponding to the registration information One is displayed in the form of AR.
  • the input interface is further configured to acquire an actual scanned image marked with a suspicious area, wherein if the detected object is a suspicious object, a position at which the image of the detected object is displayed in the actual scanned image is marked as a suspicious area;
  • the display is also configured to display the actual scanned image marked with the suspicious area in an AR manner on the container in which the object is to be inspected.
  • the display is further configured to display at least one of the standard scan image, the three-dimensional model, and the descriptive information corresponding to the registration information in an AR manner on the container on which the object is to be placed.
  • the processor is further configured to acquire at least one of a standard scan image, a three-dimensional model, and descriptive information of the suspect; the display is further configured to standard scan images, three-dimensional models, and descriptiveness of the suspect At least one of the information is displayed in the form of an AR.
  • the processor is further configured to perform image processing on at least one of a standard scan image corresponding to the registration information and a standard scan image of the suspicious object in response to the image processing instruction; the display is further configured to process The subsequent standard scanned image is displayed in the AR mode.
  • the processor is further configured to perform at least one of a zooming and rotating operation on at least one of a three-dimensional model corresponding to the registration information and a three-dimensional model of the suspicious object in response to the model processing instruction; It is also configured to display the three-dimensional model after the model operation in an AR manner.
  • the standard scanned image is a multi-angle standard X-ray scan image.
  • an auxiliary security system comprising a server, a database and the above-described auxiliary security device; wherein the server is configured to store at least one of a standard scanned image and a three-dimensional model; the database is configured to The address of the standard scanned image and the three-dimensional model in the server, and descriptive information corresponding to the registration information and descriptive information of the suspicious object are stored.
  • an auxiliary security device comprising: a memory; and a processor coupled to the memory, the processor being configured to perform the auxiliary security method as described above based on the instructions stored in the memory.
  • a computer readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the steps of the above-described auxiliary security method.
  • FIG. 1 is a schematic flow chart of some embodiments of an auxiliary security detection method according to the present disclosure.
  • FIG. 2 is a schematic flow chart of other embodiments of the auxiliary security detection method of the present disclosure.
  • FIG. 3 is a schematic flow chart of still another embodiment of the auxiliary security inspection method of the present disclosure.
  • FIG. 4 is a schematic flow chart of still another embodiment of the auxiliary security detection method of the present disclosure.
  • FIG. 5 is a schematic diagram of some application embodiments of the auxiliary security detection method of the present disclosure.
  • FIG. 6 is a schematic structural view of some embodiments of an auxiliary security device according to the present disclosure.
  • Figure 7 is a block diagram showing the structure of some embodiments of the auxiliary security system of the present disclosure.
  • FIG. 8 is a schematic structural view of another embodiment of the auxiliary security device of the present disclosure.
  • FIG. 9 is a schematic structural view of still another embodiment of the auxiliary security device of the present disclosure.
  • FIG. 1 is a schematic flow chart of some embodiments of an auxiliary security detection method according to the present disclosure.
  • the object to be inspected may be a specific item in the container, wherein the container may be, for example, a vehicle, a container, a package, or the like.
  • the name of the goods listed in the customs declaration can be obtained by checking the customs declaration form, and the name of the goods is the registration information of the goods.
  • a standard scan image corresponding to the registration information is acquired.
  • the multi-angle standard X-ray scan image of each cargo can be stored in the memory in advance, and the multi-angle standard X-ray scan image of the cargo can be found in the memory by the name of the cargo. If the customs declaration shows that there is a brand mobile phone in the container being checked, the corresponding multi-angle standard X-ray scan image can be found by the name of the brand mobile phone.
  • step 130 the standard scanned image is displayed in an AR (Augmented Reality) manner to determine whether the object is a suspicious object by comparing the standard scanned image and the actual scanned image, wherein the actual scanned image includes the detected object.
  • AR Augmented Reality
  • the AR technology is applied to the security inspection field.
  • the security personnel wear the AR device, and the AR device can see the multi-angle standard X-ray scan of the cargo in the cargo container at the predetermined position.
  • the image at the same time, the security personnel can also see the current X-ray scan image containing the goods in the container.
  • the multi-angle standard X-ray scan image of the cargo By comparing the multi-angle standard X-ray scan image of the cargo with the current X-ray scan image, it can be judged whether the cargo is a suspicious object. .
  • the standard scan image corresponding to the registration information of the test object is acquired by the registration information of the test object, and is displayed in the manner of AR, and the security personnel determine the check by comparing the standard scan image with the actual scan image. Whether the object is suspicious or not, satisfies the demand of the security personnel for the knowledge of the goods, provides guidance for the security checkers, and improves the accuracy of the security check.
  • the security personnel wear the AR device, and the AR device is, for example, an AR glasses.
  • the security in the embodiment may be a human or an automated device.
  • the AR device acquires registration information of the object to be inspected.
  • the AR device acquires a standard scan image, a three-dimensional model, and descriptive information corresponding to the object registration information.
  • the descriptive information includes the name, quantity, definition, use, characteristic parts on the scanned image, commonly used packaging materials, and the like.
  • the AR device displays the standard scanned image, the three-dimensional model, and the descriptive information in an AR manner at a predetermined location.
  • an identification map sticker may be set at a predetermined position, and the AR device calculates feature points for each frame image captured by the camera, and compares with the feature points of the identification map to identify the predetermined position, and then the standard scan image
  • the three-dimensional model and descriptive information are displayed in a predetermined position in the AR manner.
  • the AR device can also move the display position of the standard scanned image, the three-dimensional model, and the descriptive information in response to the gesture movement instruction, so that the security personnel can view it.
  • the security personnel can compare whether the current scanned image is consistent with the standard scanned image, and view the three-dimensional model and descriptive information to comprehensively determine whether the detected object is a suspicious object.
  • the object to be inspected is displayed as a mobile phone, and the security personnel can see the standard scanned image, the three-dimensional model and the descriptive information of the mobile phone at the predetermined position by wearing the AR device, but display the image as a pistol shape in the current scanned image. Then, it is determined that the object to be inspected does not match the information in the customs declaration, and the object to be inspected may be a suspicious object.
  • the standard scanned image, the three-dimensional model and the descriptive information are displayed in a predetermined position in an AR manner, assisting the security personnel to view the map, satisfying the security personnel's demand for commodity knowledge, and reducing the training cost of the security personnel, and being a security inspector.
  • the AR device may further acquire a standard scan image, a three-dimensional model, and descriptive information of the suspicious object, and then display the standard scan image, the three-dimensional model, and the descriptive information of the suspicious object in an AR manner.
  • the location is predetermined so that the security personnel can compare the standard scanned image of the suspect with the actual scanned image. For example, if the customs declaration shows that the object to be inspected is a mobile phone, and the object to be inspected passes through the security inspection device, after the security inspection device scans the X-ray scan image of the object to be inspected, and the predetermined algorithm identifies that the object to be inspected may be a pistol, the pistol information is obtained. Sended to the AR device, the AR device can obtain the standard scanned image, three-dimensional model and descriptive information of the pistol, and then display it in the AR mode.
  • the security personnel look at the picture, they can see the standard scan image of the pistol and the standard scan image of the mobile phone in AR mode, and see that the current scan image is displayed as a pistol shape image, then the object can be basically determined as a pistol, which further improves Look at the accuracy rate.
  • the AR device further performs image processing on the standard scanned image or the standard scanned image of the suspicious object corresponding to the object registration information in response to the image processing instruction, and processes the processed standard scanned image Displayed in AR mode. For example, performing histogram equalization, logarithmic transformation, and the like on a standard scanned image to enhance the display of the image, for example, mapping a previously narrow grayscale range in the image to a wider grayscale range, thereby making the image contrast larger. It is convenient for security personnel to look at the picture.
  • the AR device further performs a zooming and rotating operation on the three-dimensional model or the three-dimensional model of the suspicious object corresponding to the object registration information in response to the model processing instruction, and performs the model operation.
  • the 3D model is displayed in the form of AR.
  • the AR device stores a correspondence between a user gesture, a cargo, and an operation command, and performs a zooming, rotating, and the like operation on the three-dimensional model of the cargo by recognizing the user gesture and the cargo.
  • FIG. 3 is a schematic flow chart of still another embodiment of the auxiliary security inspection method of the present disclosure.
  • an actual scanned image marked with a suspicious area is acquired, wherein if the object to be inspected is a suspicious object, the position at which the image of the object to be inspected is displayed in the actual scanned image is marked as a suspicious area.
  • the security personnel can determine that the goods are suspicious by comparing the standard scanned image of the goods with the actual scanned image, and then mark the suspicious area in the actual scanned image and send the image to another AR device to open Check the box for the AR equipment to check.
  • the actual scanned image marked with the suspect area is displayed in the AR on the container in which the object is to be placed.
  • the unpacking inspector wears the AR equipment, and the marked suspicious area can be seen at the predetermined position of the cargo box where the goods are located, so that the inspector can accurately locate the suspicious object.
  • the unpacking check and the view at the time of security check are two separate parts.
  • the suspicious area is marked when viewing the picture
  • the unpacking check only one image with a suspicious mark is used, and the inspector only It is not accurate enough to make a rough positioning of the suspicious object in the container by feeling, and the efficiency of the unpacking inspection is lowered, and in the present embodiment, the actual scanning marked with the suspicious area is marked during the unpacking inspection.
  • the image is displayed in the AR on the container where the object is placed, so that the inspector can accurately locate the suspicious object and improve the efficiency of the unpacking inspection.
  • FIG. 4 is a schematic flow chart of still another embodiment of the auxiliary security detection method of the present disclosure.
  • the database and the server may be established in advance, wherein the server stores multi-angle standard X-ray scan images and three-dimensional models of the goods.
  • the security personnel wear the AR device to view the current scanned image.
  • a variety of cargo is contained in a container.
  • the security personnel can see a scanned image containing each cargo in the container.
  • the AR device loads the database and the server through the network, and obtains descriptive information corresponding to the current goods, a multi-angle standard X-ray scan image and a three-dimensional model, and the descriptive information corresponding to the current goods, and the multi-angle standard X.
  • the light scan image and the three-dimensional model are displayed in a specified position in an AR manner.
  • the descriptive information corresponding to the goods, the multi-angle standard X-ray scan image and the three-dimensional model can be displayed in the same position or in different positions.
  • the AR device performs image processing on the multi-angle standard X-ray scan image of the goods or scales and rotates the three-dimensional model of the goods in response to the image processing instruction or the model processing instruction, and displays the image in an AR manner.
  • the processed multi-angle standard X-ray scan image or the 3D model after the model operation is for reference by security personnel.
  • the security personnel perform image processing operations such as histogram equalization or logarithmic transformation on the multi-angle standard X-ray scan image by means of gestures or voices, or zoom and rotate the three-dimensional model of the cargo, and the security personnel refer to the AR display.
  • the image and the 3D model can determine whether the corresponding goods are suspicious.
  • step 440 the security personnel determine whether the shipment is a suspicious object, and if so, proceed to step 450, otherwise, step 410 is performed.
  • the security personnel mark the suspicious area on the viewing software and send the scanned image, the customs declaration information, etc. marked with the suspicious area to another AR device.
  • standard scanned images, 3D models and descriptive information for the goods can be sent to another AR device.
  • the unpacking inspector wears the AR device, and associates the customs declaration information and the scanned image with the container.
  • the security inspector can obtain the customs declaration information and the actual scanned image of the current container to be inspected when viewing the scanned image indoors, so that the customs declaration form is associated with the actual scanned image, and if there is a suspicious area on the actual scanned image,
  • the customs declaration information and the actual scanned image are sent to another AR device, and the unpacking inspector wears the AR device, and can locate the container by means of container number identification, etc., so that the customs declaration information, scanned image and container can be associated. .
  • the actual scanned image marked with the suspect area is displayed in AR on the container in which the goods are located.
  • the standard scanned image, three-dimensional model and descriptive information of the goods can also be displayed in the AR on the container where the goods are located, so that the unpacking inspectors can locate the suspicious objects.
  • the viewer wears the AR device, loads the descriptive information corresponding to the current goods, the standard X-ray scan image and the three-dimensional model through the network loading database and the server, and adopts the AR method.
  • the auxiliary viewers look at the map.
  • the viewer can perform image processing on the standard X-ray image by gesture or voice, such as histogram equalization, logarithmic transformation, etc., to zoom, rotate, etc. the three-dimensional model. It satisfies the demand of the map personnel for the knowledge of the products, reduces the training cost of the mappers, provides the map guide for the mappers, and improves the accuracy of the map and the user experience of the viewers.
  • the unpacking inspector wears the AR equipment, associates the customs declaration information and the scanned image with the container, and displays the suspicious area in the space by the AR method, which is convenient for the inspection personnel to locate the suspicious object and improve the efficiency of the unpacking inspection. .
  • the standard scan image, the three-dimensional model, and the descriptive information of the suspicious object information may also be displayed in the AR on the container where the goods are located, so that the unpacking inspector is more Quickly locate suspicious objects and learn specific information about suspicious objects.
  • FIG. 5 is a schematic diagram of an application embodiment of the auxiliary security detection method of the present disclosure.
  • the AR device is described by taking HoloLens as an example.
  • a database and server are established, wherein the server can use a Tomcat server to store multi-angle standard X-ray scan images and three-dimensional models of the goods.
  • the database can use MySQL to store multi-angle standard X-ray scan images of the goods and the address of the 3D model in the server, as well as some descriptive information about the goods.
  • step 520 it is determined that the network connection of Hololens is normal, and the device does not require an external power source when in use. After booting up, put the focus in the viewport on the application interface in the application interface, then put your hand in the Hololens horizon, hand fist, the index finger thumb open and then the two fingers quickly close together to complete the click operation.
  • Hololens reads the customs declaration corresponding to the current scanned image and obtains the name of the goods listed under the customs declaration.
  • the viewer clicks on the gesture described in step 520 or directly reads the name of the goods to be viewed by voice.
  • the request is processed by the server-side servlet, and the pre-established database is performed according to the name of the goods. Searching and querying the standard X-ray scan image corresponding to the name of the goods and the storage path of the three-dimensional model in the server, and the descriptive information corresponding to the name of the goods.
  • Hololens displays the descriptive information corresponding to the cargo name in an AR manner, and loads and displays the standard X-ray scan image and the three-dimensional model in the server through the network according to the storage path obtained in step 540.
  • the viewer can perform various operations such as zooming in, zooming out, model rotation, image processing, and the like by gesture or voice.
  • various operations such as zooming in, zooming out, model rotation, image processing, and the like by gesture or voice.
  • the viewers say that the histogram is equalized, and after Hololens recognizes, the pixel matrix of the standard X-ray scan image is acquired, thereby performing image processing such as histogram equalization.
  • the viewer can mark the suspicious area on the viewing software and send the customs declaration information, scanned image, and suspicious object location information stored on the server to HoloLens.
  • the server-side socket does not locate a specific AR device client socket, but is in a state of waiting for a connection to monitor the network status in real time.
  • a connection request is made by the client's socket to connect to the server-side socket with the specified address and port number.
  • the server-side socket listens to the client socket connection request, responds to the client socket request, creates a new thread, reads the image information, and sends the image information to the client.
  • the out-of-box inspector wears HoloLens, associates the container to be inspected, the scanned image, and the customs declaration information by gesture, space mapping, box number identification, image recognition, or GPS positioning, and displays the scanned image in an AR manner. And customs declaration information. Based on the suspicious object location information in step 570, the location of the suspicious object is marked in the space.
  • the demand for the product knowledge of the picture drawing person is satisfied, the training cost of the picture drawing personnel is reduced, the picture drawing guide is provided for the picture drawing person, and the user experience of the picture viewing accuracy and the picture viewing person is improved.
  • the container to be inspected, the customs declaration information and the scanned image are associated, the location of the suspect is accurately located, and the efficiency of the unpacking inspection is improved.
  • FIG. 6 is a schematic structural view of some embodiments of an auxiliary security device according to the present disclosure.
  • the auxiliary security device can be an AR device, including an input interface 610, a processor 620, and a display 630, wherein:
  • the input interface 610 is configured to acquire registration information of the object to be inspected.
  • the object to be inspected may be a specific item in the container, wherein the container may be, for example, a vehicle, a container, a package, or the like.
  • the name of the goods listed in the customs declaration can be obtained by checking the customs declaration form, and the name of the goods is the registration information of the goods.
  • the processor 620 is configured to acquire a standard scan image corresponding to the registration information of the object to be inspected.
  • the multi-angle standard X-ray scan image of each cargo can be stored in advance in the memory, and the multi-angle standard X-ray scan image of the cargo can be found in the memory by the name of the cargo. If the customs declaration shows that there is a brand mobile phone in the container being checked, the corresponding multi-angle standard X-ray scan image can be found by the name of the brand mobile phone.
  • the display 630 is configured to display a standard scan image corresponding to the registration information in an augmented reality AR manner to determine whether the object is a suspicious object by comparing the standard scan image and the actual scan image, wherein the actual scan image includes The image of the specimen.
  • the security personnel wear the AR equipment, and the AR equipment can see the multi-angle standard X-ray scan image of the goods in the cargo box at the predetermined position, and the security personnel can also see that the goods are included.
  • the current X-ray scan image of the goods in the box can be judged whether the goods are suspicious by comparing the multi-angle standard X-ray scan image of the goods with the current X-ray scan image.
  • the standard scan image corresponding to the registration information of the test object is acquired by the registration information of the test object, and is displayed in the manner of AR, and the security personnel determine the check by comparing the standard scan image with the actual scan image. Whether the object is suspicious or not, satisfies the demand of the security personnel for the knowledge of the goods, provides guidance for the security checkers, and improves the accuracy of the security check.
  • the processor 620 may be further configured to acquire a standard scan image, a three-dimensional model, and descriptive information corresponding to the object registration information.
  • the descriptive information includes the name, quantity, definition, use, characteristic parts on the scanned image, commonly used packaging materials, and the like.
  • the display 630 is configured to display the standard scanned image, the three-dimensional model, and the descriptive information in an AR manner at a predetermined location.
  • the security personnel can compare the current scanned image with the standard scanned image of the test object, and view the three-dimensional model and descriptive information to comprehensively determine whether the test object is a suspicious object.
  • the object to be inspected is displayed as a mobile phone, and the security personnel can see the standard scanned image, the three-dimensional model and the descriptive information of the mobile phone at the predetermined position by wearing the AR device, but display the image as a pistol shape in the current scanned image. Then, it is determined that the object to be inspected does not match the information in the customs declaration, and the object to be inspected may be a suspicious object.
  • the standard scanned image, the three-dimensional model and the descriptive information are displayed in a predetermined position in an AR manner, assisting the security personnel to view the map, satisfying the security personnel's demand for commodity knowledge, and reducing the training cost of the security personnel, and being a security inspector.
  • the processor 620 may be further configured to acquire a standard scanned image, a three-dimensional model, and descriptive information of the suspect
  • the display 630 is further configured to display a standard scanned image of the suspect, a three-dimensional model, and The descriptive information is displayed in the AR position at a predetermined position, so that the security personnel can compare the standard scanned image and the actual scanned image of the suspicious object to improve the accuracy of viewing.
  • the processor 620 may be further configured to perform image processing on a standard scan image or a standard scan image of the suspicious object corresponding to the object registration information in response to the image processing instruction, and the display 630 is It is configured to display the processed standard scanned image as AR. For example, performing histogram equalization, logarithmic transformation, and the like on a standard scanned image to enhance the display of the image, for example, mapping a previously narrow grayscale range in the image to a wider grayscale range, thereby making the image contrast larger. It is convenient for security personnel to look at the picture.
  • the processor 620 may be further configured to perform a zooming and rotating operation on the three-dimensional model or the three-dimensional model of the suspicious object corresponding to the object registration information in response to the model processing instruction, and the display 630
  • the three-dimensional model configured to perform the model operation is displayed in an AR manner.
  • the AR device stores the correspondence between the user gesture, the goods, and the operation instructions, and the processor 620 performs operations such as zooming, rotating, and the like by recognizing the user gesture and the goods.
  • the input interface 610 is further configured to acquire an actual scanned image marked with a suspicious area, wherein if the detected object is a suspicious object, the position of the image of the detected object in the actual scanned image is Mark as suspicious area.
  • the security personnel can determine that the goods are suspicious by comparing the standard scanned image of the goods with the actual scanned image, and then mark the suspicious area in the actual scanned image and send the image to another AR device to open Check the box for the AR equipment to check.
  • the display 630 is also configured to display the actual scanned image marked with the suspicious area in an AR manner on the container in which the object is to be placed. Among them, in the case of unpacking inspection, the unpacking inspector wears the AR equipment, and the marked suspicious area can be seen at the predetermined position of the cargo box where the goods are located, so that the inspector can accurately locate the suspicious object.
  • the display 630 is further configured to display the standard scanned image, the three-dimensional model and the descriptive information of the object or the suspect in the AR manner on the container where the object is to be placed, so as to facilitate the unpacking inspector to locate the suspicious object.
  • the actual scanned image marked with the suspicious area is displayed on the container where the object is placed in the AR manner, so that the inspector can accurately locate the suspicious object and improve the unpacking inspection. effectiveness.
  • FIG. 7 is a block diagram showing the structure of some embodiments of the auxiliary security system of the present disclosure.
  • the auxiliary security system includes a server 710, a database 720, and an auxiliary security device 730, wherein the auxiliary security device 730 can be an AR device and has been described in detail in the above embodiments.
  • Server 710 is configured to store standard scanned images and three-dimensional models of the goods
  • database 720 is configured to store the standard scanned images of the goods and the addresses of the three-dimensional models in server 710, as well as descriptive information.
  • the goods can be either the goods in the registration information or the goods actually detected.
  • the auxiliary security device 730 can acquire the standard scan image, the three-dimensional model and the descriptive information corresponding to the object registration information through the database 720 and the server 710, and display it in the AR manner, and if the security device has identified that the object may be For a suspicious object, the auxiliary security device 730 can also acquire a standard scan image, a three-dimensional model, and descriptive information of the suspicious object, and display it in an AR manner. In order to enable the security personnel to perform image recognition, improve the accuracy of viewing and the efficiency of unpacking inspection.
  • FIG. 8 is a schematic structural view of another embodiment of the auxiliary security device of the present disclosure.
  • the auxiliary security device includes a memory 810 and a processor 820.
  • the memory 810 can be a magnetic disk, a flash memory, or any other non-volatile storage medium.
  • the memory is configured to store the instructions in the embodiment corresponding to Figures 1-5.
  • the processor 820 is coupled to the memory 810 and can be implemented as one or more integrated circuits, such as a microprocessor or a microcontroller.
  • the processor 820 is configured to execute instructions stored in the memory.
  • the auxiliary security device 900 includes a memory 910 and a processor 920.
  • Processor 920 is coupled to memory 910 via BUS bus 930.
  • the auxiliary security device 900 can also be coupled to the external storage device 950 via the storage interface 940 for invoking external data, and can also be connected to the network or another computer system (not shown) via the network interface 960. It will not be described in detail here.
  • the accuracy of the security check can be improved, and the security personnel can also accurately locate the suspicious object and improve the efficiency of the unpacking check.
  • a computer readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the steps of the method of the embodiment of FIGS. 1-5.
  • a processor may implement the steps of the method of the embodiment of FIGS. 1-5.
  • the present disclosure may be provided as a method, apparatus, or computer program product. Accordingly, the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment, or a combination of software and hardware aspects. Moreover, the present disclosure may take the form of a computer program product embodied on one or more computer-usable non-transitory storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program code. .
  • the computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture comprising the instruction device.
  • the apparatus implements the functions specified in one or more blocks of a flow or a flow and/or block diagram of the flowchart.
  • These computer program instructions can also be loaded onto a computer or other programmable data processing device such that a series of operational steps are performed on a computer or other programmable device to produce computer-implemented processing for execution on a computer or other programmable device.
  • the instructions provide steps for implementing the functions specified in one or more of the flow or in a block or blocks of a flow diagram.

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • High Energy & Nuclear Physics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Life Sciences & Earth Sciences (AREA)
  • Geophysics (AREA)
  • Computer Graphics (AREA)
  • Computer Hardware Design (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Analysing Materials By The Use Of Radiation (AREA)

Abstract

一种辅助安检方法、装置和系统，涉及安检领域。辅助安检方法包括：获取被检物的登记信息(110)；获取登记信息对应的标准扫描图像(120)；将标准扫描图像以增强现实AR的方式显示，以便通过比对标准扫描图像和实际扫描图像确定被检物是否为可疑物(130)，其中，实际扫描图像包括被检物的图像。为安检人员提供了看图指导，提高了安检看图准确率。在开箱检查时，将标记有可疑区域的实际扫描图像以AR的方式显示在被检物所在容器上，使得检查人员能够准确定位可疑物位置，提高了开箱检查的效率。

Description

辅助安检方法、装置和系统
本申请是以CN申请号为201711431607.5，申请日为2017年12月26日的申请为基础，并主张其优先权，该CN申请的公开内容在此作为整体引入本申请中。
技术领域
本公开涉及安检领域，尤其涉及一种辅助安检方法、装置和系统。
背景技术
利用集装箱或车辆夹带走私，例如，偷运毒品、爆炸物甚至大规模杀伤性武器和放射性散布装置等违禁品，已经成为困扰各国政府、干扰国际货物运输正常秩序的国际公害。货物/车辆安全检查是全世界共同关注的课题。X射线透视成像技术是违禁品检查领域最基本的、也是最早得到广泛应用的技术，目前仍然是全世界应用最广泛的集装箱、货物、车辆检查技术。
X射线透视图像与自然光图像差别较大，各类货物的不同角度X光透视图像并不为多数人所熟知，所以看图人员必须通过经验积累或培训掌握大量的商品知识，对各类货物在X光下的透视图像有一定的预判，才能有效地进行查图工作。
发明内容
根据本公开一方面，提出一种辅助安检方法，包括：获取被检物的登记信息；获取与登记信息对应的标准扫描图像；将标准扫描图像以增强现实AR的方式显示，以便通过比对标准扫描图像和实际扫描图像确定被检物是否为可疑物；其中，实际扫描图像包括被检物的图像。
在一些实施例中，该方法还包括：获取与登记信息对应的三维模型和描述性信息中的至少一项；将与登记信息对应的三维模型和描述性信息中的至少一项以AR的方式显示。
在一些实施例中，该方法还包括：获取标记有可疑区域的实际扫描图像，其中，若被检物为可疑物，则实际扫描图像中显示被检物的图像的位置被标记为可疑区域；将标记有可疑区域的实际扫描图像以AR的方式显示在被检物所在容器上。
在一些实施例中，该方法还包括：将与登记信息对应的标准扫描图像、三维模型和描述性信息中的至少一项以AR的方式显示在被检物所在容器上。
在一些实施例中，该方法还包括：获取可疑物的标准扫描图像、三维模型和描述性信息中的至少一项；将可疑物的标准扫描图像、三维模型和描述性信息中的至少一项以AR的方式显示。
在一些实施例中，该方法还包括：响应于图像处理指令，对与登记信息对应的标准扫描图像和可疑物的标准扫描图像中的至少一项进行图像处理，并将处理后的标准扫描图像以AR的方式显示。
在一些实施例中，该方法还包括：响应于模型处理指令，对与登记信息对应的三维模型和可疑物的三维模型中的至少一项进行缩放和旋转操作中的至少一项操作，并将进行模型操作后的三维模型以AR的方式显示。
在一些实施例中，标准扫描图像为多角度标准X光扫描图像。
根据本公开的另一方面，还提出一种辅助安检装置，包括：输入接口，被配置为获取被检物的登记信息；处理器，被配置为获取与登记信息对应的标准扫描图像；和显示器，被配置为将被检物的标准扫描图像以增强现实AR的方式显示，以便通过比对标准扫描图像和实际扫描图像确定被检物是否为可疑物，其中，实际扫描图像包括被检物的图像。
在一些实施例中，处理器还被配置为获取与登记信息对应的三维模型和描述性信息中的至少一项；显示器还被配置为将与登记信息对应的三维模型和描述性信息中的至少一项以AR的方式显示。
在一些实施例中，输入接口还被配置为获取标记有可疑区域的实际扫描图像，其中，若被检物为可疑物，则实际扫描图像中显示被检物图像的位置被标记为可疑区域；显示器还被配置为将标记有可疑区域的实际扫描图像以AR的方式显示在被检物所在容器上。
在一些实施例中，显示器还被配置为将与登记信息对应的标准扫描图像、三维模型和描述性信息中的至少一项以AR的方式显示在被检物所在容器上。
在一些实施例中，处理器还被配置为获取可疑物的标准扫描图像、三维模型和描述性信息中的至少一项；显示器还被配置为将可疑物的标准扫描图像、三维模型和描述性信息中的至少一项以AR的方式显示。
在一些实施例中，处理器还被配置为响应于图像处理指令，对与登记信息对应的标准扫描图像和可疑物的标准扫描图像中的至少一项进行图像处理；显示器还被配置为将处理后的标准扫描图像以AR的方式显示。
在一些实施例中，处理器还被配置为响应于模型处理指令，对与登记信息对应的 三维模型和可疑物的三维模型中的至少一项进行缩放和旋转操作中的至少一项操作；显示器还被配置为将进行模型操作后的三维模型以AR的方式显示。
在一些实施例中，标准扫描图像为多角度标准X光扫描图像。
根据本公开的另一方面，还提出一种辅助安检系统，包括服务器、数据库和上述的辅助安检装置；其中，服务器被配置为存储标准扫描图像和三维模型中的至少一项；数据库被配置为存放标准扫描图像和三维模型在服务器中的地址、以及与登记信息对应的描述性信息和可疑物的描述性信息。
根据本公开的另一方面，还提出一种辅助安检装置，包括：存储器；以及耦接至存储器的处理器，处理器被配置为基于存储在存储器的指令执行如上述的辅助安检方法。
根据本公开的另一方面，还提出一种计算机可读存储介质，其上存储有计算机程序指令，该指令被处理器执行时实现上述的辅助安检方法的步骤。
通过以下参照附图对本公开的示例性实施例的详细描述，本公开的其它特征及其优点将会变得清楚。
附图说明
构成说明书的一部分的附图描述了本公开的实施例，并且连同说明书一起用于解释本公开的原理。
参照附图，根据下面的详细描述，可以更加清楚地理解本公开，其中：
图1为本公开辅助安检方法的一些实施例的流程示意图。
图2为本公开辅助安检方法的另一些实施例的流程示意图。
图3为本公开辅助安检方法的再一些实施例的流程示意图。
图4为本公开辅助安检方法的又一些实施例的流程示意图。
图5为本公开辅助安检方法的一些应用实施例的示意图。
图6为本公开辅助安检装置的一些实施例的结构示意图。
图7为本公开辅助安检系统的一些实施例的结构示意图。
图8为本公开辅助安检装置的另一些实施例的结构示意图。
图9为本公开辅助安检装置的又一些实施例的结构示意图。
具体实施方式
现在将参照附图来详细描述本公开的各种示例性实施例。应注意到：除非另外具体说明，否则在这些实施例中阐述的部件和步骤的相对布置、数字表达式和数值不限制本公开的范围。
同时，应当明白，为了便于描述，附图中所示出的各个部分的尺寸并不是按照实际的比例关系绘制的。
以下对至少一个示例性实施例的描述实际上仅仅是说明性的，决不作为对本公开及其应用或使用的任何限制。
对于相关领域普通技术人员已知的技术、方法和设备可能不作详细讨论，但在适当情况下，所述技术、方法和设备应当被视为授权说明书的一部分。
在这里示出和讨论的所有示例中，任何具体值应被解释为仅仅是示例性的，而不是作为限制。因此，示例性实施例的其它示例可以具有不同的值。
应注意到：相似的标号和字母在下面的附图中表示类似项，因此，一旦某一项在一个附图中被定义，则在随后的附图中不需要对其进行进一步讨论。
为使本公开的目的、技术方案和优点更加清楚明白，以下结合具体实施例，并参照附图，对本公开进一步详细说明。
图1为本公开辅助安检方法的一些实施例的流程示意图。
在步骤110，获取被检物的登记信息。被检物可以为容器里的具体货物，其中，容器例如可以为车辆、集装箱、包裹等。
在对集装箱进行安检时，通过查看报关单可以获取报关单所列货物名称，该货物名称即为货物的登记信息。
在步骤120，获取与登记信息对应的标准扫描图像。其中，可以预先在存储器中存储各货物的多角度标准X光扫描图像，通过货物的名称可以在存储器中查找到该货物的多角度标准X光扫描图像。如报关单显示正在安检的集装箱内有某品牌手机，通过该品牌手机的名称可以找到对应的多角度标准X光扫描图像。
在步骤130，将标准扫描图像以AR(Augmented Reality,增强现实)的方式显示，以便通过比对标准扫描图像和实际扫描图像确定被检物是否为可疑物，其中，实际扫描图像包括被检物的图像。
近年来，AR技术飞速发展，数字虚拟世界与现实世界得以无缝连接，给人类带来了一种全新的视觉体验和交互方式。本公开实施例中将AR技术应用到安检领域，例如，货箱在通过安检设备时，安检人员佩戴AR设备，通过AR设备可以在预定位 置看到该货箱内货物的多角度标准X光扫描图像，同时，安检人员还可以看到包含该货箱内货物的当前X光扫描图像，通过比对货物的多角度标准X光扫描图像与当前X光扫描图像可以判断出该货物是否为可疑物。
在该实施例中，通过被检物的登记信息获取与被检物的登记信息对应的标准扫描图像，并以AR的方式进行显示，安检人员通过比对标准扫描图像和实际扫描图像确定被检物是否为可疑物，满足了安检人员对货物知识的需求，为安检人员提供了看图指导，提高了安检看图准确率。
图2为本公开辅助安检方法的另一些实施例的流程示意图。其中，安检人员佩戴AR设备，AR设备例如为AR眼镜，实施例中的安检者可以为人也可以为自动化设备。
在步骤210，AR设备获取被检物的登记信息。
在步骤220，AR设备获取与被检物登记信息对应的标准扫描图像、三维模型和描述性信息。其中，描述性信息包括被检物的名字、数量、定义、用途、在扫描图像上的特征部位、通常使用的包装材料等。
在步骤230，AR设备将标准扫描图像、三维模型和描述性信息以AR的方式显示在预定位置。其中，可以在预定位置设置一个识别图贴，AR设备对摄像头拍摄的每一帧图像计算特征点，并与识别图贴的特征点进行比对，从而识别出该预定位置，然后将标准扫描图像、三维模型和描述性信息以AR的方式显示在预定位置。另外，AR设备还可以响应于手势移动指令，移动标准扫描图像、三维模型和描述性信息的显示位置，方便安检人员查看。
在步骤240，安检人员可以比对当前扫描图像与标准扫描图像是否一致，并查看三维模型和描述性信息，综合判断该被检物是否为可疑物。例如，报关单上显示被检物为手机，安检人员通过佩戴AR设备，在预定位置看到手机的标准扫描图像、三维模型和描述性信息，但在当前的扫描图像中显示为手枪形状图像，则确定被检物与报关单中的信息不相符，该被检物可能为可疑物。
在该实施例中，将标准扫描图像、三维模型和描述性信息以AR的方式显示在预定位置，辅助安检人员看图，满足安检人员对商品知识的需求，降低安检人员培训成本，为安检人员提供看图指导，提高看图准确率，并提高了安检人员的用户体验。
在本公开的另一些实施例中，AR设备还可以获取可疑物的标准扫描图像、三维模型和描述性信息，然后将可疑物的标准扫描图像、三维模型和描述性信息以AR的方式显示在预定位置，以便安检人员比对可疑物的标准扫描图像和实际扫描图像。例 如，若报关单显示被检物为手机，被检物通过安检设备时，安检设备扫描出被检物的X光扫描图像后，通过预定算法识别出被检物可能为手枪，则将手枪信息发送至AR设备，AR设备可以获取手枪的标准扫描图像、三维模型以及描述性信息，然后以AR的方式进行显示。
即在该实施例中，不仅以AR方式显示手机的标准扫描图像、三维模型以及描述性信息，同时可以显示手枪的标准扫描图像、三维模型以及描述性信息。安检人员看图时，以AR方式看到手枪的标准扫描图像和手机的标准扫描图像，同时看到当前的扫描图像中显示为手枪形状图像，则可以基本确定被检物为手枪，进一步提高了看图准确率。
在本公开的另一些实施例中，AR设备还响应于图像处理指令，对与被检物登记信息对应的标准扫描图像或可疑物的标准扫描图像进行图像处理，并将处理后的标准扫描图像以AR的方式显示。例如，对标准扫描图像进行直方图均衡、对数变换等操作，以对图像进行增强显示，例如，将图像中之前比较窄的灰度范围映射到更宽的灰度范围，使图像对比度变大，方便安检人员看图。
在本公开的另一些实施例中，AR设备还响应于模型处理指令，对与被检物登记信息对应的三维模型或可疑物的三维模型进行缩放和旋转操作等，并将进行模型操作后的三维模型以AR的方式显示。例如，AR设备中保存用户手势、货物、操作指令的对应关系，通过识别用户手势和货物，将货物的三维模型进行缩放、旋转等操作。
图3为本公开辅助安检方法的再一些实施例的流程示意图。
在步骤310，获取标记有可疑区域的实际扫描图像，其中，若被检物为可疑物，则实际扫描图像中显示被检物图像的位置被标记为可疑区域。例如，安检人员通过比对货物的标准扫描图像和实际扫描图像，确定该货物为可疑物，则可以在实际扫描图像中标记出可疑区域，并将图像发送至另一台AR设备，以便在开箱检查时供佩戴AR设备的检查人员查看。
在步骤320，将标记有可疑区域的实际扫描图像以AR的方式显示在被检物所在容器上。其中，在开箱检查时，开箱检查人员佩戴AR设备，可以在货物所在货箱预定位置看到标记的可疑区域，方便检查人员准确定位可疑物位置。
在相关技术中，安检时开箱检查和看图是分离的两个部分，虽然在看图时标记了可疑区域，但在开箱检查时，仅凭一张带可疑标记的图像，检查人员只能靠感觉对可疑物在集装箱内的位置做一个大概的定位，不够准确，而且降低了开箱检查的效率， 而在本实施例中，在开箱检查时，将标记有可疑区域的实际扫描图像以AR的方式显示在被检物所在容器上，使得检查人员能够准确定位可疑物位置，提高了开箱检查的效率。
图4为本公开辅助安检方法的又一些实施例的流程示意图。其中，可以预先建立数据库和服务器，其中，服务器中存放有货物的多角度标准X光扫描图像和三维模型等。数据库中存放该货物的多角度标准X光扫描图像和三维模型在服务器中的地址，以及该货物的一些描述性信息。
在步骤410，安检人员佩戴AR设备查看当前扫描图像。例如，集装箱内装有多种货物，集装箱通过安检设备时，安检人员可以看到包含集装箱内各个货物的扫描图像。
在步骤420，AR设备通过网络载入数据库和服务器中，获取当前货物对应的描述性信息、多角度标准X光扫描图像和三维模型等，并将当前货物对应的描述性信息、多角度标准X光扫描图像和三维模型以AR的方式在指定位置显示。其中，货物对应的描述性信息、多角度标准X光扫描图像和三维模型可以显示在同一个位置，也可以显示在不同位置。
在步骤430，AR设备响应于图像处理指令或模型处理指令，对货物的多角度标准X光扫描图像进行图像处理或对货物的三维模型进行缩放和旋转等操作，并以AR的方式显示进行图像处理后的多角度标准X光扫描图像或进行模型操作后的三维模型，供安检人员参考。例如，安检人员通过手势或语音通过AR设备对多角度标准X光扫描图像进行直方图均衡或对数变换等图像处理操作或对货物的三维模型进行缩放和旋转，安检人员参考以AR方式显示的图像和三维模型，可以判断出对应的货物是否为可疑物。
在步骤440，安检人员判断货物是否为可疑物，若是，则执行步骤450，否则，执行步骤410。
在步骤450，安检人员在看图软件上标记可疑区域，并将标有可疑区域的扫描图像、报关单信息等发送至另一台AR设备。另外，还可以将货物的标准扫描图像、三维模型和描述性信息发送至另一台AR设备。
在步骤460，开箱检查时，开箱检查人员佩戴AR设备，将报关单信息、扫描图像与集装箱关联。其中，安检人员在室内看扫描图时，可以获得当前待检查集装箱的报关单信息和实际扫描图像，这样报关单和实际扫描图像是关联起来的，若发现实际 扫描图像上有可疑区域，则将报关单信息和实际扫描图像发送至另一台AR设备，开箱检查人员佩戴这个AR设备，可以通过集装箱箱号识别等手段，定位到这个集装箱，这样就能够关联报关单信息、扫描图像与集装箱。
在步骤470，将标记有可疑区域的实际扫描图像以AR的方式显示在货物所在集装箱上。另外，还可以将货物的标准扫描图像、三维模型和描述性信息也以AR的方式显示在货物所在集装箱上，方便开箱检查人员定位可疑物。
在该实施例中，在看图部分，看图人员佩戴AR设备，通过网络载入数据库和服务器中加载当前货物对应的描述性信息、标准X光扫描图像和三维模型等，并以AR的方式显示在指定位置，辅助看图人员看图。另外，看图人员可通过手势或语音对标准X光图像进行图像处理，如直方图均衡、对数变换等，对三维模型进行缩放、旋转等操作。满足看图人员对商品知识的需求，降低看图人员培训成本，为看图人员提供看图指导，提高看图准确率和看图人员的用户体验。开箱检查时，开箱检查人员佩戴AR设备，将报关单信息、扫描图像与集装箱关联起来，并以AR的方式在空间中显示出可疑区域，方便查验人员定位可疑物，提高开箱检查效率。
在本公开的另一些实施例中，在开箱检查时，还可以将可疑物信息的标准扫描图像、三维模型和描述性信息以AR的方式显示在货物所在集装箱上，以便开箱检查人员更快的定位可疑物，以及了解可疑物的具体信息。
图5为本公开辅助安检方法的一个应用实施例的示意图。该实施例中AR设备以HoloLens为例进行说明。
在步骤510，建立数据库和服务器，其中，服务器可以使用Tomcat服务器，存放货物的多角度标准X光扫描图像和三维模型等。数据库可以使用MySQL，存放货物的多角度标准X光扫描图像和三维模型在服务器中的地址，以及货物的一些描述性信息。
在步骤520，确定Hololens的网络连接正常，该设备使用时无需外接电源。开机之后，在应用程序界面将视界中的焦点置于应用程序之上，然后将手置于Hololens视界之内，手握拳，食指拇指张开然后二指快速并拢完成点击操作。
在步骤530，进入应用程序后，Hololens读取当前扫描图像对应的报关单，获取该报关单下所列货物名称。
在步骤540，看图人员用步骤520中所描述的手势点击或直接通过语音念出需要查看的货物名称，此时，该请求会被服务器端的Servlet处理，根据货物名称对预先 建立好的数据库进行检索，查询出该货物名称对应的标准X光扫描图像和三维模型在服务器中的存放路径，以及该货物名称对应的描述性信息。
在步骤550，Hololens以AR的方式显示货物名称对应的描述性信息，并且根据步骤540获得的存放路径，通过网络加载并显示服务器中的标准X光扫描图像和三维模型。
在步骤560，当标准X光扫描图像和三维模型加载完毕之后，看图人员可通过手势或语音进行多种操作，如放大、缩小、模型旋转、图像处理等。例如，看图人员说直方图均衡，Hololens识别后，获取标准X光扫描图像的像素矩阵，从而进行直方图均衡等图像处理。
在步骤570，若发现可疑区域，看图人员可在看图软件上标记可疑区域，并将保存在服务器上的报关单信息、扫描图像和可疑物位置信息发送至HoloLens。此处，服务器端套接字并不定位具体的AR设备客户端套接字，而是处于等待连接的状态，实时监控网络状态。由客户端的套接字提出连接请求，连接指定地址与端口号的服务器端套接字。当服务器端套接字监听到客户端套接字的连接请求，响应客户端套接字的请求，建立一个新的线程，读取图像信息，并将图片信息发送给客户端。
在步骤580，开箱检查人员佩戴HoloLens，通过手势、空间映射、箱号识别、图像识别或GPS定位等方式将待检查集装箱、扫描图像以及报关单信息关联，并以AR的方式显示出扫描图像和报关单信息。根据步骤570中的可疑物位置信息，在空间中标记出可疑物位置。
在该实施例中，在看图时，满足看图人员对商品知识的需求，降低看图人员培训成本，为看图人员提供看图指导，提高看图准确率和看图人员的用户体验。在开箱检查时，关联待检查集装箱、报关单信息和扫描图像，准确定位嫌疑物位置，提高开箱检查效率。
图6为本公开辅助安检装置的一些实施例的结构示意图。该辅助安检装置可以为AR设备，包括输入接口610、处理器620和显示器630，其中：
输入接口610被配置为获取被检物的登记信息。被检物可以为容器里的具体货物，其中，容器例如可以为车辆、集装箱、包裹等。
在对集装箱进行安检时，通过查看报关单可以获取报关单所列货物名称，该货物名称即为货物的登记信息。
处理器620被配置为获取与被检物的登记信息对应的标准扫描图像。其中，可以 预先在存储器中存储各货物的多角度标准X光扫描图像，通过货物的名称可以在存储器中查找到该货物的多角度标准X光扫描图像。如报关单显示正在安检的集装箱内有某品牌手机，通过该品牌手机的名称可以找到对应的多角度标准X光扫描图像。
显示器630被配置为将与登记信息对应的的标准扫描图像以增强现实AR的方式显示，以便通过比对标准扫描图像和实际扫描图像确定被检物是否为可疑物，其中，实际扫描图像包括被检物的图像。例如，货箱在通过安检设备时，安检人员佩戴AR设备，通过AR设备可以在预定位置看到该货箱内货物的多角度标准X光扫描图像，同时，安检人员还可以看到包含该货箱内货物的当前X光扫描图像，通过比对货物的多角度标准X光扫描图像与当前X光扫描图像可以判断出该货物是否为可疑物。
在该实施例中，通过被检物的登记信息获取与被检物的登记信息对应的标准扫描图像，并以AR的方式进行显示，安检人员通过比对标准扫描图像和实际扫描图像确定被检物是否为可疑物，满足了安检人员对货物知识的需求，为安检人员提供了看图指导，提高了安检看图准确率。
在本公开的另一些实施例中，处理器620还可以被配置为获取与被检物登记信息对应的标准扫描图像、三维模型和描述性信息。其中，描述性信息包括被检物的名字、数量、定义、用途、在扫描图像上的特征部位、通常使用的包装材料等。
显示器630被配置为将标准扫描图像、三维模型和描述性信息以AR的方式显示在预定位置。安检人员可以比对当前扫描图像与被检物的标准扫描图像是否一致，并查看三维模型和描述性信息，综合判断该被检物是否为可疑物。例如，报关单上显示被检物为手机，安检人员通过佩戴AR设备，在预定位置看到手机的标准扫描图像、三维模型和描述性信息，但在当前的扫描图像中显示为手枪形状图像，则确定被检物与报关单中的信息不相符，该被检物可能为可疑物。
在该实施例中，将标准扫描图像、三维模型和描述性信息以AR的方式显示在预定位置，辅助安检人员看图，满足安检人员对商品知识的需求，降低安检人员培训成本，为安检人员提供看图指导，提高看图准确率，并提高了安检人员的用户体验。
在本公开的另一些实施例中，处理器620还可以被配置为获取可疑物的标准扫描图像、三维模型和描述性信息，显示器630还被配置为将可疑物的标准扫描图像、三维模型和描述性信息以AR的方式显示在预定位置，以便安检人员比对可疑物的标准扫描图像和实际扫描图像，提高看图准确率。
在本公开的另一些实施例中，处理器620还可以被配置为响应于图像处理指令， 对与被检物登记信息对应的标准扫描图像或可疑物的标准扫描图像进行图像处理，显示器630被配置为将处理后的标准扫描图像以AR的方式显示。例如，对标准扫描图像进行直方图均衡、对数变换等操作，以对图像进行增强显示，例如，将图像中之前比较窄的灰度范围映射到更宽的灰度范围，使图像对比度变大，方便安检人员看图。
在本公开的另一些实施例中，处理器620还可以被配置为响应于模型处理指令，对与被检物登记信息对应的三维模型或可疑物的三维模型进行缩放和旋转操作等，显示器630被配置为将进行模型操作后的三维模型以AR的方式显示。例如，AR设备中保存用户手势、货物、操作指令的对应关系，处理器620通过识别用户手势和货物，将货物的三维模型进行缩放、旋转等操作。
在本公开的另一些实施例中，输入接口610还被配置为获取标记有可疑区域的实际扫描图像，其中，若被检物为可疑物，则实际扫描图像中显示被检物图像的位置被标记为可疑区域。例如，安检人员通过比对货物的标准扫描图像和实际扫描图像，确定该货物为可疑物，则可以在实际扫描图像中标记出可疑区域，并将图像发送至另一台AR设备，以便在开箱检查时供佩戴AR设备的检查人员查看。
显示器630还被配置为将标记有可疑区域的实际扫描图像以AR的方式显示在被检物所在容器上。其中，在开箱检查时，开箱检查人员佩戴AR设备，可以在货物所在货箱预定位置看到标记的可疑区域，方便检查人员准确定位可疑物位置。
显示器630还被配置为将被检物或可疑物的标准扫描图像、三维模型和描述性信息以AR的方式显示在被检物所在容器上，方便开箱检查人员定位可疑物。
在本实施例中，在开箱检查时，将标记有可疑区域的实际扫描图像以AR的方式显示在被检物所在容器上，使得检查人员能够准确定位可疑物位置，提高了开箱检查的效率。
图7为本公开辅助安检系统的一些实施例的结构示意图。该辅助安检系统包括服务器710、数据库720和辅助安检装置730，其中，辅助安检装置730可以为AR设备，且已在上述实施例中进行了详细介绍。
服务器710被配置为存储货物的标准扫描图像和三维模型，数据库720被配置为存放货物的标准扫描图像和三维模型在服务器710中的地址、以及描述性信息。其中，货物既可以为登记信息中的货物，也可以为实际检测出的货物。
辅助安检装置730可以通过数据库720、服务器710获取与被检物登记信息对应的标准扫描图像、三维模型和描述性信息，并以AR方式进行显示，同时若安检设备 已经识别出被检物可能为某可疑物，则辅助安检装置730还可以获取可疑物的标准扫描图像、三维模型和描述性信息，并以AR方式进行显示。以便安检人员进行图像识别，提高看图准确率和开箱检查效率。
图8为本公开辅助安检装置的另一些实施例的结构示意图。该辅助安检装置包括存储器810和处理器820。其中：存储器810可以是磁盘、闪存或其它任何非易失性存储介质。存储器被配置为存储图1-5所对应实施例中的指令。处理器820耦接至存储器810，可以作为一个或多个集成电路来实施，例如微处理器或微控制器。该处理器820被配置为执行存储器中存储的指令。
在一些实施例中，还可以如图9所示，该辅助安检装置900包括存储器910和处理器920。处理器920通过BUS总线930耦合至存储器910。该辅助安检装置900还可以通过存储接口940连接至外部存储装置950以便调用外部数据，还可以通过网络接口960连接至网络或者另外一台计算机系统(未标出)。此处不再进行详细介绍。
在该实施例中，通过存储器存储数据指令，再通过处理器处理上述指令，能够提高安检看图准确率，另外，还能够帮助安检人员准确定位可疑物位置，提高开箱检查效率。
在另一些实施例中，一种计算机可读存储介质，其上存储有计算机程序指令，该指令被处理器执行时实现图1-5所对应实施例中的方法的步骤。本领域内的技术人员应明白，本公开的实施例可提供为方法、装置、或计算机程序产品。因此，本公开可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且，本公开可采用在一个或多个其中包含有计算机可用程序代码的计算机可用非瞬时性存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。
本公开是参照根据本公开实施例的方法、设备(系统)和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器，使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特 定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。
这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上，使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。
至此，已经详细描述了本公开。为了避免遮蔽本公开的构思，没有描述本领域所公知的一些细节。本领域技术人员根据上面的描述，完全可以明白如何实施这里公开的技术方案。
虽然已经通过示例对本公开的一些特定实施例进行了详细说明，但是本领域的技术人员应该理解，以上示例仅是为了进行说明，而不是为了限制本公开的范围。本领域的技术人员应该理解，可在不脱离本公开的范围和精神的情况下，对以上实施例进行修改。本公开的范围由所附权利要求来限定。

Claims (19)

  1. 一种辅助安检方法，包括：
    获取被检物的登记信息；
    获取与所述登记信息对应的标准扫描图像；
    将所述标准扫描图像以增强现实AR的方式显示，以便通过比对所述标准扫描图像和实际扫描图像确定所述被检物是否为可疑物，其中，所述实际扫描图像包括所述被检物的图像。
  2. 根据权利要求1所述的辅助安检方法，还包括：
    获取与所述登记信息对应的三维模型和描述性信息中的至少一项；
    将与所述登记信息对应的三维模型和描述性信息中的至少一项以AR的方式显示。
  3. 根据权利要求1所述的辅助安检方法，还包括：
    获取标记有可疑区域的实际扫描图像，其中，若所述被检物为可疑物，则所述实际扫描图像中显示所述被检物的图像的位置被标记为可疑区域；
    将标记有可疑区域的实际扫描图像以AR的方式显示在所述被检物所在容器上。
  4. 根据权利要求2所述的辅助安检方法，还包括：
    将与所述登记信息对应的标准扫描图像、三维模型和描述性信息中的至少一项以AR的方式显示在所述被检物所在容器上。
  5. 根据权利要求1-4任一所述的辅助安检方法，还包括：
    获取所述可疑物的标准扫描图像、三维模型和描述性信息中的至少一项；
    将所述可疑物的标准扫描图像、三维模型和描述性信息中的至少一项以AR的方式显示。
  6. 根据权利要求5所述的辅助安检方法，还包括：
    响应于图像处理指令，对与所述登记信息对应的标准扫描图像和所述可疑物的标 准扫描图像中的至少一项进行图像处理，并将处理后的标准扫描图像以AR的方式显示。
  7. 根据权利要求5所述的辅助安检方法，还包括：
    响应于模型处理指令，对与所述登记信息对应的三维模型和所述可疑物的三维模型中的至少一项进行缩放和旋转操作中的至少一项操作，并将进行模型操作后的三维模型以AR的方式显示。
  8. 根据权利要求1-4任一所述的辅助安检方法，其中，所述标准扫描图像为多角度标准X光扫描图像。
  9. 一种辅助安检装置，包括：
    输入接口，被配置为获取被检物的登记信息；
    处理器，被配置为获取与所述登记信息对应的标准扫描图像；和
    显示器，被配置为将所述标准扫描图像以增强现实AR的方式显示，以便通过比对所述标准扫描图像和实际扫描图像确定所述被检物是否为可疑物，其中，所述实际扫描图像包括所述被检物的图像。
  10. 根据权利要求9所述的辅助安检装置，其中，
    所述处理器还被配置为获取与所述登记信息对应的三维模型和描述性信息中的至少一项；
    所述显示器还被配置为将与所述登记信息对应的三维模型和描述性信息中的至少一项以AR的方式显示。
  11. 根据权利要求9所述的辅助安检装置，其中，
    所述输入接口还被配置为获取标记有可疑区域的实际扫描图像，其中，若所述被检物为可疑物，则所述实际扫描图像中显示所述被检物的图像的位置被标记为可疑区域；
    所述显示器还被配置为将标记有可疑区域的实际扫描图像以AR的方式显示在所述被检物所在容器上。
  12. 根据权利要求10所述的辅助安检装置，其中，
    所述显示器还被配置为将与所述登记信息对应的标准扫描图像、三维模型和描述性信息中的至少一项以AR的方式显示在所述被检物所在容器上。
  13. 根据权利要求9-12任一所述的辅助安检装置，其中，
    所述处理器还被配置为获取所述可疑物的标准扫描图像、三维模型和描述性信息中的至少一项；
    所述显示器还被配置为将所述可疑物的标准扫描图像、三维模型和描述性信息中的至少一项以AR的方式显示。
  14. 根据权利要求13所述的辅助安检装置，其中，
    所述处理器还被配置为响应于图像处理指令，对与所述登记信息对应的标准扫描图像和所述可疑物的标准扫描图像中的至少一项进行图像处理；
    所述显示器还被配置为将处理后的标准扫描图像以AR的方式显示。
  15. 根据权利要求13所述的辅助安检装置，其中，
    所述处理器还被配置为响应于模型处理指令，对与所述登记信息对应的三维模型和所述可疑物的三维模型中的至少一项进行缩放和旋转操作中的至少一项操作；
    所述显示器还被配置为将进行模型操作后的三维模型以AR的方式显示。
  16. 根据权利要求9-12任一所述的辅助安检装置，其中，所述标准扫描图像为多角度标准X光扫描图像。
  17. 一种辅助安检系统，包括服务器、数据库和权利要求9-16任一所述的辅助安检装置；
    其中，所述服务器被配置为存储标准扫描图像和三维模型中的至少一项；
    所述数据库被配置为存放标准扫描图像和三维模型在所述服务器中的地址、以及与所述登记信息对应的描述性信息和所述可疑物的描述性信息。
  18. 一种辅助安检装置，包括：
    存储器；以及
    耦接至所述存储器的处理器，所述处理器被配置为基于存储在所述存储器的指令执行如权利要求1至8任一项所述的辅助安检方法。
  19. 一种计算机可读存储介质，其上存储有计算机程序指令，该指令被处理器执行时实现权利要求1至8任一项所述的辅助安检方法的步骤。
PCT/CN2018/120291 2017-12-26 2018-12-11 辅助安检方法、装置和系统 WO2019128698A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201711431607.5 2017-12-26
CN201711431607.5A CN109959969B (zh) 2017-12-26 2017-12-26 辅助安检方法、装置和系统

Publications (1)

Publication Number Publication Date
WO2019128698A1 true WO2019128698A1 (zh) 2019-07-04

Family

ID=64744689

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/120291 WO2019128698A1 (zh) 2017-12-26 2018-12-11 辅助安检方法、装置和系统

Country Status (4)

Country Link
US (1) US11055869B2 (zh)
EP (1) EP3505972A1 (zh)
CN (1) CN109959969B (zh)
WO (1) WO2019128698A1 (zh)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109959969B (zh) * 2017-12-26 2021-03-12 同方威视技术股份有限公司 辅助安检方法、装置和系统
US10854055B1 (en) * 2019-10-17 2020-12-01 The Travelers Indemnity Company Systems and methods for artificial intelligence (AI) theft prevention and recovery
CN113160615B (zh) * 2021-03-03 2022-11-29 上海凌苇智能科技合伙企业(有限合伙) 一种基于ar技术实现无人机起飞前安全检测方法及系统
CN113391776B (zh) * 2021-05-24 2023-05-09 浙江大华技术股份有限公司 物品信息显示方法、装置、电子装置和存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103745198A (zh) * 2013-12-30 2014-04-23 中云智慧（北京）科技有限公司 行邮辅助识别及对比的方法与装置
CN104750697A (zh) * 2013-12-27 2015-07-01 同方威视技术股份有限公司 基于透视图像内容的检索系统、检索方法以及安全检查设备
CN105372712A (zh) * 2014-09-02 2016-03-02 清华大学 车辆检查方法和系统
CN106443806A (zh) * 2016-09-18 2017-02-22 北京君和信达科技有限公司 辐射成像系统和图像处理方法
US20170147619A1 (en) * 2015-11-24 2017-05-25 International Business Machines Corporation Augmented reality model comparison and deviation detection
US20170346634A1 (en) * 2016-05-27 2017-11-30 Assa Abloy Ab Augmented reality security verification

Family Cites Families (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7783118B2 (en) * 2006-07-13 2010-08-24 Seiko Epson Corporation Method and apparatus for determining motion in images
DE102006047888A1 (de) * 2006-10-10 2008-04-17 Smiths Heimann Gmbh Verfahren zur Verbesserung der Detailerkennbarkeit in Röntgenbildern
EP2634802A1 (en) * 2012-02-29 2013-09-04 BlackBerry Limited Single package imaging and inertial navigation sensors, and methods of manufacturing the same
CN104865279A (zh) * 2014-02-26 2015-08-26 清华大学 一种用于海关监管物项的快速检查设备和方法
US9720934B1 (en) * 2014-03-13 2017-08-01 A9.Com, Inc. Object recognition of feature-sparse or texture-limited subject matter
CN105787495A (zh) 2014-12-17 2016-07-20 同方威视技术股份有限公司 具有车辆参考图像检索及比对功能的车辆检查系统和方法
CN105809655B (zh) * 2014-12-30 2021-06-29 清华大学 车辆检查方法和系统
CN104535594A (zh) * 2015-01-08 2015-04-22 安徽中福光电科技有限公司 基于x射线+dr+cbs的箱包安检方法和智能化x光机
US10261033B2 (en) * 2015-01-20 2019-04-16 National Technology & Engineering Solutions Of Sandia, Llc Material identification system
CN105615831A (zh) * 2015-12-18 2016-06-01 李占和 医学影像全息ar显示系统
CN106503121B (zh) * 2016-10-19 2019-12-06 公安部第三研究所 一种x光安检图像的结构化描述方法及系统
CN206292821U (zh) * 2016-10-31 2017-06-30 紫光智云(江苏)物联网科技有限公司 物品安检判图系统
CN106778514A (zh) * 2016-11-24 2017-05-31 努比亚技术有限公司 一种标识物体的方法及装置
CN107228867A (zh) * 2017-06-21 2017-10-03 同方威视技术股份有限公司 安检图像显示方法、设备和安检系统
CN109959969B (zh) * 2017-12-26 2021-03-12 同方威视技术股份有限公司 辅助安检方法、装置和系统

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104750697A (zh) * 2013-12-27 2015-07-01 同方威视技术股份有限公司 基于透视图像内容的检索系统、检索方法以及安全检查设备
CN103745198A (zh) * 2013-12-30 2014-04-23 中云智慧（北京）科技有限公司 行邮辅助识别及对比的方法与装置
CN105372712A (zh) * 2014-09-02 2016-03-02 清华大学 车辆检查方法和系统
US20170147619A1 (en) * 2015-11-24 2017-05-25 International Business Machines Corporation Augmented reality model comparison and deviation detection
US20170346634A1 (en) * 2016-05-27 2017-11-30 Assa Abloy Ab Augmented reality security verification
CN106443806A (zh) * 2016-09-18 2017-02-22 北京君和信达科技有限公司 辐射成像系统和图像处理方法

Also Published As

Publication number Publication date
CN109959969B (zh) 2021-03-12
US20190197721A1 (en) 2019-06-27
EP3505972A1 (en) 2019-07-03
CN109959969A (zh) 2019-07-02
US11055869B2 (en) 2021-07-06

Similar Documents

Publication Publication Date Title
WO2019128698A1 (zh) 辅助安检方法、装置和系统
US9349076B1 (en) Template-based target object detection in an image
JP2020521959A5 (zh)
CN111340126B (zh) 物品识别方法、装置、计算机设备和存储介质
CN106030610B (zh) 移动设备的实时3d姿势识别和跟踪系统
CN108898127B (zh) 一种基于三维模型匹配的防伪方法及设备
CN107909093B (zh) 一种物品检测的方法及设备
US10963739B2 (en) Learning device, learning method, and learning program
US9996947B2 (en) Monitoring apparatus and monitoring method
WO2020134102A1 (zh) 物品识别方法、装置、售货系统和存储介质
US9285876B2 (en) Transparent display field of view region determination
CN104881673B (zh) 基于信息整合的模式识别的方法和系统
CN115103648A (zh) 器械跟踪机器
US9715701B2 (en) Image-based listing using image of multiple items
US20200184196A1 (en) Volumetric substitution of real world objects
WO2023193482A1 (zh) 显示方法、装置、电子设备和计算机可读存储介质
CN110570513A (zh) 一种展示车损信息的方法和装置
US20210174087A1 (en) System and method for hybrid visual searches and augmented reality
Buls et al. Generation of synthetic training data for object detection in piles
TWI773863B (zh) 自助結帳系統、方法與裝置
KR101845423B1 (ko) 구조물 검사 지원 장치 및 방법
CN114972500A (zh) 查验方法、标注方法、系统、装置、终端、设备及介质
KR20150108556A (ko) 객체 자세 기반 연출 효과 제공 장치 및 방법, 그리고 이를 위한 컴퓨터 프로그램이 기록된 기록매체
CN109978067A (zh) 一种基于卷积神经网络与尺度不变特征变换的商标检索方法以及装置
US9477890B2 (en) Object detection using limited learned attribute ranges

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18895779

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18895779

Country of ref document: EP

Kind code of ref document: A1