CN106682127A - Image searching system and method - Google Patents

Image searching system and method Download PDF

Info

Publication number
CN106682127A
CN106682127A CN201611149728.6A CN201611149728A CN106682127A CN 106682127 A CN106682127 A CN 106682127A CN 201611149728 A CN201611149728 A CN 201611149728A CN 106682127 A CN106682127 A CN 106682127A
Authority
CN
China
Prior art keywords
image
feature
stack features
area
roi
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201611149728.6A
Other languages
Chinese (zh)
Inventor
王季勇
许向阳
曹弯弯
常庆丰
孙乾程
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai United Imaging Healthcare Co Ltd
Original Assignee
Shanghai United Imaging Healthcare Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai United Imaging Healthcare Co Ltd filed Critical Shanghai United Imaging Healthcare Co Ltd
Priority to CN201611149728.6A priority Critical patent/CN106682127A/en
Publication of CN106682127A publication Critical patent/CN106682127A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content

Landscapes

  • Engineering & Computer Science (AREA)
  • Library & Information Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention discloses an image searching system and method. The picture searching method comprises the steps of acquiring a first image, wherein the first image comprises a first area; acquiring a second image, wherein the second image comprises a second area; acquiring a third image, wherein the third image comprises a third area; extracting a first group of features from the first area, a second group of features from the second area and a third group of features from the third area respectively based on a convolutional neutral network algorithm; calculating first data relevant to the first group of the features and the second group of the features, and calculating second data relevant to the first group of the features and the third group of the features; conducting ranking on the second image and the third image according to the first data and the second data. According to the technical scheme, at least one of the speed, stability, and the accuracy of image searching is improved.

Description

Image search system and method
Technical field
The application is related to a kind of image search system and method, is based especially on the feature for extracting image to be searched, and with Characteristics of image in image data base compares, so as to provide the System and method for of the image similar to image to be searched.
Background technology
When similar image search is carried out, weighed and compared generally according to numerous characteristics of image between image, so as to Draw the result with image similarity to be searched.For example, when clinically doctor is diagnosed to Lung neoplasm and passed judgment on, generally according to crowd Many Lung neoplasm similarity balances, and then draw the diagnostic result of tuberosity.In order to obtain the image the most similar to image to be searched, Need by the greatest extent can be more in characteristics of image to be searched and image data base image be compared, finally find out most like figure Picture, provides more objective Comprehensive Evaluation.Over time with the growth of picture number, it is difficult to obtain steady based on the image comparison of human eye Fixed judge a, so that retrieval that can more accurately search for the image to be searched image similar to view data Kuku System and method for.
The content of the invention
The technical scheme of the application provides a kind of image search method and system, solve efficiency present in image retrieval, The problem of stability or accuracy.
A kind of some embodiments of the present application, there is provided image search method.The method includes the step in following operation Or multistep：The first image is obtained, first image includes first area；The second image is obtained, second image includes the secondth area Domain；The 3rd image is obtained, the 3rd image includes the 3rd region；Based on convolutional neural networks algorithm, first area is extracted respectively The first stack features, the second stack features of second area and the 3rd region the 3rd stack features；Calculate with first stack features and The first related data of second stack features, and the second data that first stack features are related to the 3rd stack features；According to this First data and second data, are ranked up to second image and the 3rd image.
In certain embodiments, the method may further include the second parameter of setting, and extract relevant with second area Second feature collection.The second feature collection can include the second stack features.The method further includes to be based on second stack features, The first parameter is set, and extracts first stack features relevant with first area.
In certain embodiments, it is feature based selection algorithm to concentrate from second feature and select the second stack features.One In a little embodiments, this feature selection algorithm is Correlation Based Feature Selection algorithms.
In certain embodiments, the first parameter or the second parameter can include convolutional neural networks level number, convolution kernel At least one in the number of features of size, convolution kernel number and generation.
In certain embodiments, convolutional neural networks algorithm can include multiple structure.
In certain embodiments, the first data are the difference generations of the feature based on the first stack features and the second stack features.Can Choosing, the second data are that the difference of the feature based on the first stack features and the 3rd stack features is produced.
In certain embodiments, the method can include being based on the first data and the second data, be calculated using LambdaMART Method sorts to the second image and the 3rd image.In certain embodiments, the method further include according to the sequence determine this Two images or the 3rd image.
In certain embodiments, the method may further include and use computer-aided diagnosises, fixed in the first image Position first area.In certain embodiments, the method may further include and use computer-aided diagnosises, according to firstth area Domain, in the second image second area is selected.
In certain embodiments, the first image, the second image or the 3rd image are two dimensional image or 3-D view.
In certain embodiments, the first image, the second image or the 3rd image be PET image, CT images, SPECT images, At least one of MRI image, ultrasonoscopy.
According to some embodiments of the present application, there is provided a kind of system of image reconstruction.The system can be including a figure As acquisition device.The image acquiring device can obtain the first image, the second image and the 3rd image, and first image, First area, second area and the 3rd region are positioned respectively in second image and the 3rd image.The system can be further Including an image analysis apparatus.The image analysis apparatus can include characteristic extracting module.This feature extraction module can be carried The first stack features of the first area are taken, the second stack features of the second area and the 3rd stack features in the 3rd region are extracted. The image analysis apparatus may further include characteristics analysis module.This feature analysis module can calculate first stack features and The first related data of second stack features, and the second data that first stack features are related to the 3rd stack features.The figure As analytical equipment may further include order module.The order module can be right based on first data and second data Second image and the 3rd image are ranked up.
In certain embodiments, image acquiring device is further included and uses computer-aided diagnosises, in the first image Positioning first area.In certain embodiments, the image acquiring device is further included and uses computer-aided diagnosises, second Second area is positioned in image.
In certain embodiments, the extraction of the first stack features can be based on the second stack features or the 3rd stack features.
In certain embodiments, this feature selecting module can extract the relevant second feature collection of second area, and this second Feature set may further include second stack features.
In certain embodiments, the system may further include a feature selection module.This feature selecting module can Selection is concentrated to obtain second stack features from the second feature with feature based selection algorithm, this feature selection algorithm can be included Correlation Based Feature Selection algorithms.
In certain embodiments, characteristics analysis module can be based on first stack features and second stack features feature it Difference produces the first data.Optionally, characteristics analysis module can be based on first stack features and the 3rd stack features feature it Difference produces the second data.
In certain embodiments, order module can be based on first data and second data, using LambdaMART Algorithm sorts to the second image and the 3rd image.
A part of bells and whistles of the application can be illustrated in the following description.By to following description and accordingly The inspection of accompanying drawing or the understanding of production or operation to embodiment, a part of bells and whistles of the application is for art technology Personnel are obvious.The characteristic of present disclosure can be by the method for the various aspects to specific embodiments described below, means Practice with combination or using being achieved and reach.
At least one of speed, stability or accuracy when the technical scheme of the application improves picture search.
Description of the drawings
Figure 1A is a kind of schematic diagram of the image search system according to some embodiments of the present application；
Figure 1B is the schematic diagram of the computing device configuration according to some embodiments of the present application；
Fig. 2 is a kind of flow chart of the picture search according to some embodiments of the present application；
Fig. 3 is a kind of schematic diagram of the image acquiring device according to some embodiments of the present application；
Fig. 4 is a kind of image capture flow figure according to some embodiments of the present application；
Fig. 5 is a kind of schematic diagram of the image analysis apparatus according to some embodiments of the present application；
Fig. 6 is a kind of flow chart of the graphical analyses according to some embodiments of the present application；
Fig. 7 is a kind of flow chart of the ROI image feature extraction according to some embodiments of the present application；
Fig. 8 is the stream that a kind of employing convolutional neural networks according to some embodiments of the present application carry out feature extraction Cheng Tu；And
Fig. 9 is a kind of schematic diagram of the storage device according to some embodiments of the present application.
Specific embodiment
It is understandable to enable the above objects, features and advantages of the present invention to become apparent from, below in conjunction with the accompanying drawings to the present invention Specific embodiment be described in detail.Elaborate detail in order to fully understand the present invention in the following description.But It is that the present invention can be implemented with various different from alternate manner described here, those skilled in the art can be without prejudice to this Similar popularization is done in the case of invention intension.Therefore the present invention is not limited by following public specific embodiment.
As shown in the specification and claims, unless context clearly points out exceptional situation, " one ", " one ", The word such as " one kind " and/or " being somebody's turn to do " not refers in particular to odd number, may also comprise plural number.It is, in general, that term " including " is only carried with "comprising" Show including clearly identify the step of and element, and these steps and element do not constitute one it is exclusive enumerate, method or Equipment can also comprising other the step of or element.
Image search system and method described in this specification is referred to based on the feature for extracting image to be searched, and and image Characteristics of image in data base compares, so as to provide the image similar to image to be searched.The image search system can be wrapped Include an image processing engine, storage device, an output device and a control device.
The different embodiments of the application are applicable to multiple fields, including medical science and its derivative science and technology (including medical treatment Apparatus diagnosis, medical imaging etc.) and image processing field (including mathematics, physics, chemistry and Chemical Engineering, biological and biological work Image procossing in the fields such as journey, electronic engineering, communication system, the Internet, Internet of Things, news media).The difference of the application is real An application scenarios is applied including webpage, browser plug-in, client, custom-built system, enterprises analysis system, artificial intelligence machine One or more combination such as people.It is only above specific example to the description of suitable application area, is not considered as unique feasible Embodiment.For clearly for one of skill in the art, in the original substantially for understanding a kind of image search method and system After reason, can be in the case of without departing substantially from this principle, to implementing in said method and systematic difference field form and details Various amendments and change, but these amendments and change still within the scope of above description.For example, at one of the application In embodiment, the image to be searched of image search system 100 can be the species such as medical image, news image.Medical image can Being position emissron tomography (Positron Emission Tomography, PET) image, single photon emission computed tomography Scanning (Single-Photo Emission Comupted Tomography, SPECT) image, CT scan It is (Computed Tomography, CT) image, nuclear magnetic resonance, NMR (Magnetic Resonance Imaging, MRI) image, super Acoustic image or other similar images or combination.Similar replacement or amendment change, still the application protection domain it It is interior.In order to be illustrated more clearly that the technical scheme of the embodiment of the present application, embodiment will be described below needed for be used it is attached Figure is briefly described.It should be evident that drawings in the following description are only some embodiments of the present application, for this area Those of ordinary skill for, on the premise of not paying creative work, can with according to these accompanying drawings by the application application In other similar scenes.Unless apparent from language environment or separately explain, in figure identical label represent identical structure and Operation.
Figure 1A is a kind of schematic diagram of the image search system 100 according to some embodiments of the present application.Such as Fig. 1 institutes State, image search system 100 can include an output device of image processing engine 120, of user interface 110, 130th, a storage device 140 and a control device 150.Fig. 1 described images search system 100 is only to represent the application's Some embodiments, for the ordinary skill in the art, on the premise of not paying creative work, can be according to figure Modification is made as the description of search system 100, is increased and is deleted.For example, two of which device can be combined into an equipment, or The one of device of person can respectively on two or more equipment.In certain embodiments, multiple devices may reside in one In individual computing device (such as computer, mobile phone, wearable computing devices).In certain embodiments, multiple devices can pass through net Network links together.The network can include LAN, wide area network, common network, dedicated network, WLAN, virtual One or more combination such as network, city Metropolitan Area Network (MAN), public switch telephone network.
User interface 110 can with receiving data, and received data is transferred to image search system 100 other Part.In certain embodiments, the setup parameter that user interface 110 can be with receive user to image processing engine 120, for example, Set algorithm of the receive user to image processing engine 120.In certain embodiments, user interface 110 can transmit image and arrive In image acquiring device 121.User interface 110 can include one or more communication terminal.The communication terminal can be handss Machine, PC, wearable device, panel computer, intelligent television etc., or the combination in any of above-mentioned communication terminal.
Image processing engine 120 can obtain the image with image similarity to be searched.Image processing engine 120 can include One image acquiring device 121 and an image analysis apparatus 122.
Image acquiring device 121 can obtain one or more image to be searched and/or reference picture.The figure to be searched Picture and/or reference picture can be obtained from user interface 110, storage device 140 and/or the other parts of image search system 100 Take." image to be searched " refers to the benchmark image in image search procedure, i.e., in image search procedure, image in the application Search system can search out the reference picture similar to benchmark image according to benchmark image.Image to be searched can be defeated by user The image interested for entering, it is also possible to comprising an image that can be identified feature.What " reference picture " referred to is stored in a figure In picture storehouse, and for carrying out the image of similarity-rough set with image to be searched.In certain embodiments, reference picture and/or Image to be searched may reside in one or more image data bases.Described image data base can include reference picture, treat Search image and other images, and the information of image correlation etc..The related information of described image include the time that image generates and The content (such as image be PET image, MRI image, and/or image are image with regard to certain position of patient etc.) of image.Treat Search image and/or reference picture can be the images of any form and content.In certain embodiments, image to be searched and/ Or reference picture can be two dimensional image and/or 3-D view.In certain embodiments, image to be searched and/or reference picture Can be PET image, SPECT images, CT images, MRI image, ultrasonoscopy or other similar images or combination.Obviously Ground, image to be searched and reference picture can also be the image of other guide, for example, the news figure comprising word and/or personage Piece.Image acquiring device 121 can position a certain specific region in image to be searched and/or reference picture.In some enforcements In example, image acquiring device 121 can orient certain using computer-aided diagnosises in image to be searched and/or reference picture One specific region (focus in medical image etc.).In certain embodiments, computer-aided diagnosises can produce a width Or several area-of-interests (Region of Interest, ROI) or volume of interest (Volume of Interest, VOI) Image.The ROI/VOI images are related to specific region.
Image analysis apparatus 122 may determine that the similarity of image to be searched and one or more reference picture.Image point Analysis apparatus 122 can pass through to extract the feature of one or more image to be searched and/or reference picture, and carry out feature comparison, So as to judge the similarity of image to be searched and reference picture.It is individually measurable that " feature " in the application refers to an image Attribute.In certain embodiments, feature can include the pixel of piece image (for example, Lung neoplasm image is multiple Pixel), and the corresponding information of pixel.For example, the size of pixel, quantity, gray value, textural characteristics, morphology is special Levy, gray feature, wavelet character, histogram feature etc. can serve as the feature for being used to compare between image.In some embodiments In, feature can include the statistical nature of piece image, for example, in two dimensional image number of pixels both vertically as well as horizontally be gone up, Or the rectangular histogram of gray value.In certain embodiments, feature can include the feature of object shown by piece image, for example, one The anatomical features of individual biologic-organ.For example, feature can include the display feature of a focus.By taking Lung neoplasm as an example, feature The readability at Lung neoplasm edge can be described, it is smooth or jagged, leaflet is whether there is, cavity is whether there is, is whether there is calcification, is a mao glass Glass Lung neoplasm or solid nodules etc..In certain embodiments, feature can include the feature extracted through feature extraction algorithm, For example, feature for carrying out being obtained after process of convolution to piece image by convolutional neural networks algorithm etc..
Image analysis apparatus 122 can include but is not limited to central processing unit (Central Processing Unit, CPU), specialized application integrated circuit (Application Specific Integrated Circuit, ASIC), special instruction Processor (Application Specific Instruction Set Processor, ASIP), concurrent physical processor (Physics Processing Unit, PPU), digital signal processor (Digital Processing Processor, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), PLD One kind or several in (Programmable Logic Device, PLD), processor, microprocessor, controller, microcontroller etc. The combination planted.
Output device 130 can be received, processes, stores and/or transmitted and come from user interface 110, image acquiring device 121st, the data of the other parts of image analysis apparatus 122, storage device 140 and/or image search system 100.Output device 130 can export the one or more reference picture with image similarity to be searched.In certain embodiments, output device can be with base In the similarity size of reference picture and image to be searched, the one or more of output and image similarity to be searched are selected with reference to figure Picture.Output device 130 can export the related information of to be searched and/or reference picture.For example, output device can be with output image Path that time, the image of generation is stored in data base, image show the related information of content (in going out such as image shows Hold related to certain identification number) etc..Output device 130 can include a display screen (such as lcd screen, LED screen, CRT screens Curtain etc.), a printer (such as ink-jet printer, film printer), or other are used for one or more of device etc. of output Combination.
Storage device 140 can be received, processes and/or stored and come from user interface 110, image acquiring device 121, figure As analytical equipment 122, output device 130, control device 150 and/or come from the number of the other parts of image search system 100 According to.The data can include the combination of one or more of related information of image to be searched, reference picture, feature, image etc.. In certain embodiments, storage device 140 can be the equipment using electric energy mode storage information, such as various memorizeies, with Machine access memorizer (Random Access Memory, RAM), read only memory (Read Only Memory, ROM) etc..With Machine memorizer can include dynamic RAM (DRAM), SRAM (SRAM), IGCT random access memory (T- RAM), the combination of one or more of zero capacitance random access memory (Z-RAM) etc..Read only memory can include CD drive, Hard disk, tape, early stage nonvolatile storage (NVRAM), non-volatile SRAM, flash memory, the electronics formula of erasing can make carbon copies read only memory, The combination of one or more in Erasable Programmable Read Only Memory EPROM, programmable read only memory etc..Storage device 140 can be with It is using the equipment of magnetic energy mode storage information, such as hard disk, floppy disk, tape, core memory, magnetic bubble memory, USB flash disk, sudden strain of a muscle Deposit.Storage device 140 can be using the equipment of optical mode storage information, such as CD or DVD etc..Storage device 140 can To be the equipment using magneto-optic mode storage information, such as magneto-optic disk etc..The access mode of storage device 140 can be deposited at random One or more combination such as storage, serial access storage, read-only storage.Storage device 140 can be impermanent memory storage device, It can also be permanent memory storage device.Above-mentioned storage device is to list some examples, and storage device does not limit to In this.Storage device 140 can be on local, or long-range, or Cloud Server.
Control device 150 can receive, process, store and/or transmit user interface 110, image processing engine 120, defeated Go out the information of device 130, storage device 140 and/or the other parts of image search system 100.Described information can include from User interface 110 user setting (as to reference picture output format set, to image analysis apparatus 122 adopt algorithm Setting etc.), transmit to the information of storage device 140 (as image to be searched storage format and path, carry from image to be searched Storage format for the feature for taking etc.), and transmit to the information of output device 130 (such as whether output one or more reference picture) Deng.
Figure 1B is a kind of schematic diagram of computing device configuration according to some embodiments of the present application.The computing device Can be used in the particular system in the application.Particular system in the present embodiment explains one comprising user using functional block diagram The hardware platform at interface.Computing device can implement one or more assemblies, module, unit, the son list of current descriptive system 100 First (for example, user interface 110, image processing engine 120, control device 150).This computing device can be a general mesh Computer, or a computer for having a specific purpose.Two kinds of computers can be used for realizing the present embodiment In particular system.For convenience's sake, a computing device is only depicted in Figure 1B, but the offer described by the present embodiment The related computing devices function of picture search information needed can be to be implemented in a distributed fashion, by one group of similar platform , the process load of disperse system.
As shown in Figure 1B, computing device is configured may include internal communication bus 160, processor (processor) 162, firmly Disk 164, read only memory (ROM) 166, random access memory (RAM) 168, COM1 170, input output assembly 172. Internal communication bus 160 can be configured to the data communication for realizing computing device inter-module.Processor 162 is used for execute program instructions Complete any function, component, module, unit, the subelement of the image search system 100 described in here application.Processor 162 are made up of one or more processors.In certain embodiments, the species of processor 162 can include but is not limited to micro-control Device processed, RISC (RISC), special IC (ASIC), application-specific instruction set processor (ASIP), Central processing unit (CPU), graphic process unit (GPU), concurrent physical processor (PPU), microprocessor unit, digital signal processor (DSP), field programmable gate array (FPGA), or other be able to carry out calculation procedure instruction circuit or processor or its group Close.
Computing device also is used to store program and data, such as hard disk 164 including the storage device of multi-form, read-only Memorizer (ROM) 166, random access memory (RAM) 168, can be used in computing device process and/or communication use it is various Data file, and the possible programmed instruction performed by processor 162.Storage device can be in image search system 100 Portion's (for example, storage device 140), it is also possible to outside search system 100 (ratio connects such as by user interface 110).
COM1 170 is configurable to realize that computing device (such as uses output device with other parts of image search system 100 130, storage device 140) between data communication.Input output assembly 172 supports computing device with the other assemblies of system 100 (such as User interface 110) between input/output data stream.Computing device can also be sent and be connect by COM1 170 from network By information and data.The form of the output information of image search system 100 can include but is not limited to numeral, character, instruction, pressure The combination of one or more in power, sound, image, system, software, program etc..In certain embodiments, input/output group Part 172 can include but is not limited to the one kind in display, printer, drawing apparatuss, image output equipment, speech output etc. Or several combinations.The information of output can be sent to user, it is also possible to not send.The output information not sent can be stored in Storage device 140, hard disk 164, read only memory (ROM) 166, in random access memory (RAM) 168, it is also possible to delete.
It should be noted that above-mentioned storage device is (such as hard disk 164, read only memory (ROM) 166, random access memory (RAM) 168) and/or processor 162 can be actually existed in system, it is also possible to which corresponding function is completed by cloud computing platform. Wherein, cloud computing platform can include but is not limited to storage-type cloud platform based on data storage, based on processing data Calculation type cloud platform and take into account data storage and process comprehensive cloud computing platform.The cloud that image search system 100 is used Platform can be public cloud, private clound, community cloud or mixed cloud etc..For example, according to actual needs, image search system 100 connects The a part of information received, can be counted by cloud platform, calculated and/or stored.Another part information, can pass through local place Reason equipment and/or storage device are calculated and/or stored.
Fig. 2 is a kind of flow chart of the picture search according to some embodiments of the present application.In 201, can obtain Take image.The image of acquisition can be image to be searched and/or reference picture.The image of acquisition is interested comprising one or more The image that region (ROI) or volume of interest (VOI) are formed.The acquisition of image to be searched and/or reference picture can be by scheming As acquisition device 121 is performed.
In 202, the data of image to be searched can be based on, the image with image similarity to be searched is searched for.The mistake of search Journey can include calculating the similarity of image to be searched and Multi reference images respectively, and according to it is described calculate respectively it is similar Degree is ranked up to Multi reference images.The similarity for calculating image to be searched and a width reference picture can be described by calculating Image to be searched is realized with the similarity of the characteristics of image of the reference picture.In certain embodiments, the process of search can With including the ROI/VOI images calculated respectively in image to be searched in one or more ROI/VOI images and Multi reference images Similarity, and Multi reference images are ranked up according to the similarity for calculating respectively.Extract view data and search Rope similar image can be performed by image acquiring device 121 and image analysis apparatus 122.Described image data can include waiting to search The species (such as PET image, SPECT images, CT images, MRI image) of rope image and/or reference picture, from image to be searched And/or the ROI/VOI images obtained in reference picture, and/or the feature extracted from ROI/VOI images etc..
In 203, the one or more with image similarity to be searched can be determined according to the ranking results of the similarity Reference picture.One or more reference picture with image similarity to be searched mentioned here refers to similarity and meets certain bar The reference picture of part.For example, the reference picture corresponding to the more forward similarity of ranking is higher with the similarity of image to be searched. Can determine in one Search Results 5 before similarity ranking, front 10, or front 20 reference picture.Similar reference picture is really Surely can be performed by output device 130.
It is only above specific example to the description of suitable application area, is not considered as the embodiment of unique feasible. For clearly for one of skill in the art, after the ultimate principle of a kind of image search method and system is understood, Ke Yi In the case of without departing substantially from this principle, to implement the various amendments in said method and systematic difference field form and details and Change, but these amendments and change still within the scope of above description.For example, in 201, can obtain and figure to be searched As related other information, and image is searched for based on the other information.Specifically, can obtain one of image to be searched or Multiple features, or the semantic description related to image to be searched one or more features, follow-up search procedure can be searched The image or information of Suo Hanyou similar characteristics, or search meet the image of the semantic description.
Fig. 3 is a kind of schematic diagram of the image acquiring device 121 according to some embodiments of the present application.Image is obtained Device 121 can include image collection module 301, identification module 302, locating module 303, ROI/VOI image collection module 304, and other parts for realizing image acquiring device function.It should be evident that Fig. 3 described images acquisition device 121 It is only to represent some embodiments of the present application, for the ordinary skill in the art, is not paying creative work On the premise of, modification can be made according to the description of image acquiring device 121, increase and delete.For example, two of which module can To be combined into a module, or one of module can be divided into two or more modules.
Image collection module 301 can be used for obtaining one or more image to be searched and/or reference picture.It is described to wait to search Rope image and/or reference picture can have polytype.In certain embodiments, image to be searched and/or reference picture can be with It is two dimensional image and/or 3-D view.In certain embodiments, image to be searched and/or reference picture can be PET image, The combination of a kind of image or various images in CT images, SPECT images, MRI image, ultrasonoscopy etc..
Identification module 302 can be used for recognizing image to be searched and/or reference picture.Identification module 302 can be recognized and treated The type and content of search image and/or reference picture.In certain embodiments, identification module 302 can recognize figure to be retrieved Picture and/or reference picture are a kind of in two dimensional image, 3-D view.In certain embodiments, identification module 302 can be recognized and treated Retrieval image and/or reference picture are a kind of in PET image, CT images, SPECT images, MRI image, ultrasonoscopy.
Locating module 303 can position one or more specific regions in image to be searched and/or reference picture.It is described Specific region can be comprising area-of-interest (ROI) or volume of interest (VOI).In certain embodiments, locating module 303 can To position institute in the image for identifying using computer-aided diagnosises (Computer-Aided Diagnosis, CAD) State specific region.
ROI/VOI image collection module 304 can obtain image to be searched and/or the ROI/VOI images in reference picture. The ROI/VOI images can be that image to be searched and/or the specific region in reference picture are extracted into formed figure Picture.The ROI/VOI images can be two dimensional image and/or 3-D view.In certain embodiments, it is to be searched for two dimension Image and/or reference picture, the image that ROI/VOI image collection module 304 is obtained can be ROI image；For three-dimensional is treated Search image and/or reference picture, the image that ROI/VOI image collection module 304 is obtained can be VOI images.In some realities In applying example, ROI/VOI images can be the image to be searched and/or reference picture itself for obtaining, or image to be searched and/or ginseng Examine the part in image.Only using lung images medically as example, ROI/VOI images can show whole pulmonary, Or a certain concrete specific region in pulmonary.Specific region can include pulmonary lesionses region or produce region of Lung neoplasm etc..
The acquisition of ROI/VOI images can be completed manually, and for example, medical personnel can specify focal area. The acquisition of ROI/VOI images can be obtained by Computer Automatic Recognition.In certain embodiments, ROI/VOI images can be with Obtained by computer-aided diagnosises.For example, computer identified after focus, and focus can be set as in itself target area, And generate ROI/VOI images.Again for example, computer is identified after focus, can set a region as target area, and root ROI/VOI images are generated according to the target area.The target area can be included focus.The target area can have With the same or analogous profile of focus, or other profiles.The area of the target area can be more essentially identical than lesion area, or It is big by 1%, or 3%, or 5%, or 10%, or 50%, or other arbitrary numerals.Focus mentioned here is right Should there is abnormal region in biological tissue, can show to be different from normal structure " mutation " or " cloudy in the picture Shadow ".
It should be evident that Fig. 3 described images acquisition device 121 is only to represent some embodiments of the present application, for this For the those of ordinary skill in field, on the premise of not paying creative work, can retouching according to image acquiring device 121 State and make modification, increase and delete.For example, in certain embodiments, locating module 303 and ROI/VOI image collection module 304 A module can be combined into.In some more specifically embodiment, locating module 303 can treated by calculating auxiliary diagnosis One or more regions are positioned in search image and/or reference picture, and obtains the ROI/VOI images in the region.
Fig. 4 is a kind of image capture flow figure according to some embodiments of the present application.In 401, image is obtained Process can be comprising acquisition image to be searched and/or reference picture.The acquisition of image to be searched and/or reference picture can be by scheming As acquisition module 301 is performed.Image to be searched and/or reference picture can be from user interfaces 110 and/or storage device 140 etc. Obtain in part.In certain embodiments, image search system 100 can obtain one or more reference from image data base Image.
In 402, image acquisition procedures include identification image to be searched and/or reference picture.Identification process can pass through Identification module 302 is performed.In certain embodiments, image search system 100 can be based in image whether there is three-dimensional data (such as The pixel in three-dimensional coordinate direction etc.) judging that image to be searched and/or reference picture are two dimensional image and/or 3-D view. In certain embodiments, image search system 100 can be based on the related information (storage of type, image such as image of image The information such as the mark in path, image) judge the content of image to be searched and/or reference picture (if whether image is PET figures Whether picture, image are with regard to same patient etc.).
In 403, one or more regions are positioned in image to be searched and/or reference picture.The positioning can be by Locating module 303 is performed.In certain embodiments, can be by computer-aided diagnosises in image to be searched and/or with reference to figure One or more of regions are oriented as in.In certain embodiments, image search system 100 can pass through one or more Model (such as biological structure model, image slices vegetarian refreshments or grey value profile model) treats search image and/or reference pattern enters The region is positioned after row segmentation.
In 404, the related ROI/VOI images in one or more regions are obtained.The acquisition of ROI/VOI images can be by ROI/VOI image collection module 304 is performed.In certain embodiments, the acquisition of ROI/VOI images can be waited to search by being partitioned into Specific region in rope image and/or reference picture is realizing.For example, in a lung images, ROI/VOI images can be The image of part corresponding to focal area.In certain embodiments, image search procedure includes further extracting ROI/VOI images Characterization parameter.In certain embodiments, the characterization parameter includes area, gray average, variance, standard deviation, circularity, ball A kind of combination of the parameters such as shape degree, square, maximum, minima, shape moment descriptor and Fourier descriptor or many kinds of parameters.
Fig. 5 is a kind of schematic diagram of the image analysis apparatus 122 according to some embodiments of the present application.Graphical analyses Device 122 can be comprising parameter module 501, characteristic extracting module 502, feature selection module 503, characteristics analysis module 504, row Sequence module 505 and/or other parts for performing image analysis apparatus function.It should be evident that Fig. 5 described images analysis dress It is only to represent some embodiments of the present application to put 122, for the ordinary skill in the art, is not paying creativeness On the premise of work, modification can be made according to the description of image analysis apparatus 122, increase and delete.For example, two of which mould Block can be combined into a module, or one of module can be divided into two or more modules.
Parameter module 501 can be used for receiving, process and/or storing one or more parameters.The parameter can be with spy Levy extraction module 502, feature selection module 503, characteristics analysis module 504 and/or order module 505 related.The parameter can With comprising the information of characteristic extracting module 502, the information of feature selection module 503, the information of characteristics analysis module 504 and/or The information of order module 505.Information in characteristic extracting module 502, as a example by using convolutional neural networks algorithm, can include The match party of the level number of setting, convolution kernel size, convolution kernel number, the window size of mixed layer, mixed layer and convolutional layer Formula, position of ReLU layers etc..Information in feature selection module 503 can include number of features, type of feature of selection etc.. Information in characteristics analysis module 504 can be compared or analyze comprising the feature for treating search image and/or reference picture Mode etc..Information in order module 505 can include the number of the feature being ranked up, method of sequence etc..Treat and search Rope image, image search system 100 can arrange a kind of parameter (describing for convenience, hereinafter referred to as " the first parameter ").For ginseng Image is examined, image search system 100 can arrange a kind of parameter (describing for convenience, hereinafter referred to as " the second parameter ").Described One parameter and the second parameter can include the group of one or more in the parameter that parameter module 501 is received, processes and/or stored Close.In certain embodiments, the first parameter and the second parameter can be with identical or different.
Characteristic extracting module 502 can be used for extracting the related ROI/VOI images spy of image to be searched and/or reference picture Levy.The ROI/VOI characteristics of image refers to a ROI/VOI image individually gageable attribute.In certain embodiments, institute Stating ROI/VOI characteristics of image can include gradation of image statistical nature, textural characteristics, morphological feature, wavelet character, rectangular histogram The combination that one or more of feature etc..In certain embodiments, the ROI/VOI characteristics of image can be by one or more The image information that model (model of for example, the model of convolutional neural networks algorithm, or other related algorithms etc.) is extracted.It is described Image information may refer to the result related to image calculated by special algorithm.Extracted ROI/VOI characteristics of image can To be included in a ROI/VOI characteristics of image concentration.For example, the ROI/VOI characteristics of image of image to be searched may be embodied in one Individual image ROI/VOI characteristics of image to be searched is concentrated；The ROI/VOI characteristics of image of one or more reference picture may be embodied in One reference image R OI/VOI characteristics of image is concentrated.One ROI/VOI characteristics of image is concentrated and can include one or more image Feature, the model that these characteristics of image can be by same model or different is obtained.
Feature selection module 503 can be used for concentrating selected section or whole ROI/VOI images in ROI/VOI characteristics of image Feature.Feature selection module 503 can carry out feature selection based on one or more feature selecting algorithm.For different types of Image to be searched, feature selection module 503 can adopt identical or different feature selecting algorithm.To feature selecting algorithm Selection can be manually set, or feature selection module 503 is obtained by learning the search history of similar image to be searched Deng.In certain embodiments, the feature selecting algorithm can be Correlation Based Feature Selection (CFS) algorithm.
Characteristics analysis module 504 can analyze the data relevant with ROI/VOI characteristics of image is obtained.The ROI/VOI figures As the relevant data of feature can include the ROI/VOI characteristics of image of image to be searched, reference image R OI/VOI characteristics of image, And/or the difference of feature of image ROI/VOI characteristics of image to be searched and reference image R OI/VOI characteristics of image etc..The feature Difference can respectively be formed with the ROI/VOI characteristics of image of reference picture by the ROI/VOI characteristics of image of image to be searched Characteristic vector difference determining.In certain embodiments, the characteristic vector can be by the pixel in ROI/VOI characteristics of image Point is necessarily operated (for example, arrange, combination etc.) and represented.In certain embodiments, the ROI/VOI images of image to be searched with Reference image R OI/VOI image contains the pixel of equal number.In certain embodiments, the difference of feature can show as two width The difference of the gray value of same pixel point on ROI/VOI images.In certain embodiments, the difference of feature can show as two width ROI/ VOI images calculate the difference obtained between image information through identical algorithms.
Order module 505 can be based on characteristics of image related data and reference picture is ranked up.Order module 505 can To be ranked up to reference picture by LambdaMART algorithms.In certain embodiments, sort higher reference picture with treat The image of search has higher similarity.
It should be evident that Fig. 5 described images analytical equipment 122 is only to represent some embodiments of the present application, for this For the those of ordinary skill in field, on the premise of not paying creative work, can retouching according to image analysis apparatus 122 State and make modification, increase and delete.For example, in certain embodiments, characteristics analysis module 504 and order module 505 can be tied It is combined into a module.
Fig. 6 is a kind of flow chart of the graphical analyses according to some embodiments of the present application.In 601, image is searched Cable system 100 can be loaded into the ROI/VOI images and the second parameter of reference picture.The ROI/VOI images of reference picture and second The acquisition of parameter can be performed by parameter module 501.
In 602, image search system 100 can be based on the second parameter, the spy for extracting the ROI/VOI images of reference picture Levy, form feature set A.The ROI/VOI image characteristics extractions of reference picture can be performed by characteristic extracting module 502.At some In embodiment, the second parameter can be included using the ROI/VOI characteristics of image of convolutional neural networks algorithm extraction reference picture Method.In certain embodiments, the ROI/VOI characteristics of image of reference picture can be stored in storage device 140 and/or image The other parts of search system 100.Image search system 100 can be with the ROI/VOI characteristics of image of direct access reference picture.One In a little embodiments, the second parameter includes the number of features of the ROI/VOI characteristics of image of required extraction, for example, 500.
In 603, image search system 100 can carry out feature selection using feature selecting algorithm to feature set A, obtain Character subset B.Feature selection can be performed by feature selection module 503.In certain embodiments, the feature selecting algorithm can Being Correlation Based Feature Selection (CFS) algorithms.
In 604, image search system 100 can be loaded into the ROI/VOI images and the first parameter of image to be searched.Wait to search The ROI/VOI images of rope image and the acquisition of the first parameter can be performed by parameter module 501.In certain embodiments, first Parameter and the second parameter can be with identical or different.For example, the first parameter and the second parameter can extract image spy containing identical The algorithm levied.Again for example, the first parameter and the second parameter can contain the number of features of different extraction characteristics of image.At some In embodiment, the first parameter is that the corresponding characteristic parameter of feature based subset B is obtained.For example, can include in the first parameter The method for extracting character subset B.Again for example, the number that extracted characteristics of image can be set in the first parameter is equal to spy Levy the number of the characteristics of image included in subset B.
In 605, image search system 100 can extract image to be searched with feature based subset B and the first parameter The feature of ROI/VOI images, forms feature set C.The ROI/VOI image characteristics extractions of image to be searched can be by feature extraction mould Block 502 is performed.In certain embodiments, the first parameter can be included and extract image to be searched using convolutional neural networks algorithm The method of ROI/VOI characteristics of image.
In 606, image search system 100 can store ROI/VOI characteristics of image subset B of reference picture and/or treat The ROI/VOI set of image characteristics C of search image.ROI/VOI characteristics of image subset B of reference picture and/or image to be searched ROI/VOI set of image characteristics C can be stored in the other parts of storage device 140 and/or image search system 100. In some embodiments, ROI/VOI characteristics of image subset B of reference picture and/or the ROI/VOI set of image characteristics of image to be searched C can be stored in storage device 140 in the form of characteristics tree (such as aspect indexing tree set up based on Euclidean distance etc.).
In 607, the characteristic relevant with feature set C can be analyzed and obtained to image search system 100.With feature set The analysis and acquisition of C relevant characteristic can be performed by characteristics analysis module 504.In certain embodiments, feature point Analysis module 504 can compare ROI/VOI characteristics of image of the ROI/VOI set of image characteristics C of image to be searched and reference picture Collection B, obtains the difference of one or more features.In certain embodiments, the ROI/VOI characteristics of image of image to be searched can be compared Whole features and the difference of feature is obtained in ROI/VOI characteristics of image subsets B of collection C and reference picture, or in feature set C It is compared with selected section feature in character subset B, so as to obtain the difference of feature.,
In 608, image search system 100 can be based on the related data of the feature for obtaining, and reference picture is arranged Sequence.The difference of feature feature of the related packet containing single feature, difference of the feature of multiple features etc..In the sequence knot In fruit, the more forward reference picture of ranking is higher with the similarity of image to be searched.The sequence of reference picture can be by sequence Module 505 is performed.Order module 505 can be ranked up by one or more graders to reference picture.In some enforcements In example, the sort method that different images type or different images content are adopted can be with difference.In certain embodiments, sort Module 505 can adopt LambdaMART algorithms, the ROI/VOI images and reference picture based on each group of image to be searched The difference of one or more features between ROI/VOI images, sorts to reference picture.
In 609, image search system 100 can determine one or more reference picture according to ranking results.Here institute It is determined that one or more reference picture meet some requirements with the similarity of image to be searched.For example in the ranking results In, the more forward reference picture of ranking is higher with the similarity of image to be searched.In certain embodiments, in a Search Results Can determine 5 before ranking, front 10, or front 20 reference picture.The determination of reference picture can be performed by output device 130. In certain embodiments, image search system 100 can determine image to be searched, the ROI/VOI images of image to be searched, a width Or the ROI/VOI images of Multi reference images and one or more reference picture etc..
In image search procedure, operation 601,602,603 can occur in the training stage, image search system 100 pairs The ROI/VOI image zooming-out features of width or Multi reference images, form respectively feature set, and using feature selecting algorithm to feature Collection carries out feature selection, and character subset is formed respectively.In certain embodiments, operation 604,605,607,608,609 can occur In test phase, image search system 100 can be directly based upon the ROI/VOI characteristics of image subsets pair of the reference picture for obtaining The characteristic parameter answered, treats the ROI/VOI image zooming-out features of search image, forms feature set.And then analysis reference picture The ROI/VOI set of image characteristics of ROI/VOI characteristics of image subset and image to be searched, obtains characteristic, based on the spy for obtaining Data are levied, the reference picture is ranked up, then based on the ranking results, determine similar reference picture.
It should be evident that Fig. 6 described image analysis process is only to represent some embodiments of the present application, for this area Those of ordinary skill for, on the premise of not paying creative work, can be done according to the description of image analysis apparatus 122 Go out modification, increase and delete.For example, ROI/VOI characteristics of image subset B of reference picture can be previously stored in storage device 140 or image search system 100 in other parts, formed after feature extraction with the ROI/VOI images of image to be searched Feature set C obtains the difference of feature after being compared.
Fig. 7 is a kind of flow chart of the ROI image feature extraction according to some embodiments of the present application.In some realities In applying example, the ROI image feature extraction flow process is to be based on the first parameter and/or the second parameter, and adopts convolutional neural networks Algorithm is achieved the extraction of ROI image feature.
In 701, image search system 100 can obtain a width ROI image.The ROI image can be figure to be searched The ROI image of picture, or the ROI image of reference picture.The ROI image can be the image of any resolution.In some enforcements In example, the ROI image can be 20*20,28*28 etc..
At 702, image search system 100 can adopt first group of convolution kernel using convolutional neural networks to ROI image Multiple convolution is carried out, multiple image conv1 is obtained.In certain embodiments, first group of convolution kernel can be 3*3,5*5, Or the convolution kernel of other sizes.The number of convolution kernel can be manually set, or be adjusted according to the resolution of image.Only As an example, the quantity of convolution kernel can be 10, or 20.Multiple volume collection cores that first group of described convolution kernel is included can be with It is identical, part identical, or differ completely.ROI in a specific embodiment, to a width 28*28 Image, can adopt first group of convolution kernel of 20 5*5 to carry out convolution, obtain the image of 20 width 24*24.
In 703, the one or more image conv1 to obtaining divides sub-block.The sub-block is comprising in image conv1 Part or all of information.In certain embodiments, image search system 100 can be by intercepting one in multiple image conv1 Fixed number purpose pixel forms sub-block.For example, it can be 12*12 sub by the image division of a width 24*24 in above-described embodiment Block, the size of each sub-block is 2*2.
In 704, ready-portioned sub-block in traversal piece image conv1, and to each sub-block assignment, obtain a width new Image.For example, in the above-described embodiments, maximum is extracted in the sub-block of each 2*2 as the pixel value of sub-block, it is possible to obtain The image of one width 12*12.Maximum is extracted respectively to the sub-block in piece image conv1, piece image pool1 can be obtained. Maximum is extracted respectively to the sub-block in multiple image convl, multiple image pool1 can be obtained.For example, in above-described embodiment In, the image of 20 width 24*24 is carried out dividing sub-block and extracts the process of maximum, obtain the image of 20 width 12*12.
In 705, can again be rolled up in the width or multiple image pool1 for obtaining using second group of convolution kernel Product.In certain embodiments, second group of convolution kernel can be 3*3,5*5 or the convolution kernel of other sizes, the number of convolution kernel Mesh can be manually set, or be adjusted according to the resolution of image.Second group of described convolution kernel includes multiple volume collection Core can be identical, and part is identical, or differ completely.
In 706, some or all of image obtained after convolution again can be overlapped, obtain piece image Block.In 707, multiple convolution can be carried out to described image block using the 3rd group of convolution kernel, obtain multiple image conv2. In 708, sub-block can be divided to the width that obtains or multiple image conv2.In 709, piece image conv2 can be traveled through In ready-portioned sub-block, and to each sub-block assignment, obtain the new image of a width.In certain embodiments, can be to each height Block all extracts maximum as a pixel of new images.Maximum is extracted respectively to the sub-block in piece image conv2, can To obtain piece image pool2.Maximum is extracted respectively to the sub-block in multiple image conv2, multiple image can be obtained pool2.In 710, the pixel in the multiple image pool2 for obtaining can be represented the vector being characterized, vector is carried out Neuron is connected to entirely, obtains the feature of certain amount.In the full articulamentum, each lower floor's neuron and last layer institute Some pixels have relation.
It should be evident that ROI image feature extraction flow process is only to represent some embodiments of the present application described in Fig. 7, it is right For one of ordinary skill in the art, on the premise of not paying creative work, can be according to image analysis apparatus 122 Description make modification, increase and delete.For example, 702,705 and 707 steps are operated to be using identical convolution principle, institute State first group of convolution kernel, second group of convolution kernel and the 3rd group of convolution kernel can be the same or different；Operation 703 and operation 708 In the division of sub-block can be the same or different；The selection of 704 and 709 pairs of sub-block values, it would however also be possible to employ take maximum Outside other methods.For example, can take the meansigma methodss of all voxels in sub-block, or based in sub-block more than a certain threshold value Pixel and the numerical value that produces.
Fig. 8 is the stream that a kind of employing convolutional neural networks according to some embodiments of the present application carry out feature extraction Cheng Tu.As illustrated, carrying out feature extraction using convolutional neural networks can include following flow process：(1) it is loaded into the ROI of 28*28 Image pair, the ROI image is to including the ROI image of a ROI image to be searched and a width reference picture；(2) to one Width ROI image carries out convolution with the convolution kernel of 20 5*5, respectively obtains the image of 20 width 24*24；(3) by the figure of every width 24*24 As being divided into 12*12 sub-block, the size per block is 2*2, and to each sub-block maximum is all extracted, and obtains the image of 12*12.It is right The image of 20 width 24*24 is carried out this process, respectively obtains the image of 20 width 12*12；(4) to 20 of per group in (3rd) step The image of width 12*12, the convolution kernel of different 5*5 is respectively adopted carries out convolution, obtains the image of 20 width 8*8；Then these figures As being overlapped, the image of a width 8*8 is obtained；(5) (4th) step is performed 50 times using different convolution kernels, 50 width is obtained The image of 8*8；(4) convolution kernel and in (5) used is 50*20；(6) to the image of 50 width 8*8, each image performs 2*2 Maximum is extracted in block, the image of 50 width 4*4 is obtained；(7) 500 neurons are connected to entirely to 4*4*50 feature, it is defeated Go out 500 features；(8) for the image pair for operating (1) to be input into, each ROI image is carried out operating (2)-(7)；(9) to this The sample set of 500 feature compositions, the feature selecting algorithm CFS algorithm of feature based selecting module 503 carries out feature selection, adopts The characteristic of ROI image is obtained with the characteristic parser in characteristics analysis module 504, using in order module 505 Algorithm (for example, LambdaMART algorithms) the corresponding reference picture of ROI image to be searched is ranked up determines and search Similar reference picture.
What the ROI image of ROI image and image to be searched that the sequence to reference picture can be based on reference picture was formed Loss function.In convolutional neural networks described in Fig. 8, image search system 100 can build Siamese to two width ROI images Network, and further calculate the loss function of Siamese networks.Formula (1) is an exemplary damage of the Siamese networks Lose function：
N is sample number, i.e., image to be searched forms the number of image pair with reference picture.Yi is i-th pair image pair Similarity, represents similar during yi=1, represent dissimilar during yi=0.Di represents the distance between feature of i-th pair image pair,The feature of left side tuberosity is (Li1,Li2,…,Lim), the feature of the right tuberosity is (Ri1,Ri2,…,Rim)。
Fig. 9 is a kind of schematic diagram of the storage device 140 according to some embodiments of the present application.Storage device 140 Can comprising image storage module 901, characteristic storage module 902, relating module 903 and/or other be used for realize storage device The part of function.It should be evident that storage device 140 is only to represent some embodiments of the present application described in Fig. 9, for ability For the those of ordinary skill in domain, on the premise of not paying creative work, can be made according to the description of storage device 140 Change, increase and delete.For example, two of which module can be combined into a module, or one of module can be split For two or more modules.
Image storage module 901 can store image to be searched and/or reference picture.Image to be searched and/or with reference to figure The storage of picture can be carried out in various suitable time points.For example, image to be searched and/or reference picture can occur it is determined that with After the reference picture of image similarity to be searched.Image to be searched and/or reference picture can be with any image search systems 100 The form of support is stored.In certain embodiments, image to be searched and/or reference picture can be with Digital Imaging and Communications in Medicine (DICOM) forms are stored.
Characteristic storage module 902 can store the ROI/VOI characteristics of image of image to be searched and/or reference picture.One In a little embodiments, the ROI/VOI characteristics of image of image to be searched and/or reference picture, and image to be searched and reference picture The difference of ROI/VOI characteristics of image can be stored by characteristic storage module 902.The feature can be stored with a fixed structure In storage device 140.In certain embodiments, characteristic storage module 902 (can be such as based on by arranging characteristics tree to feature The aspect indexing tree of Euclidean distance) and by characteristic storage in storage device 140.
Relating module 903 can be used for image of the associated storage in image storage module 901 and be stored in characteristic storage Characteristics of image in module 903.In certain embodiments, relating module 903 can arrange a contingency table come associated images with Feature.The contingency table can include store name, the store path of image to be searched and/or reference picture, and feature is deposited Storage title, store path etc..
Above basic conception described, it is clear that to those skilled in the art, foregoing invention is disclosed only As an example, and the restriction to the application is not constituted.Although do not clearly state herein, those skilled in the art may Various modifications, improvement and amendment are carried out to the application.Such modification, improvement and amendment are proposed in this application, so such Change, improve, correct the spirit and scope for still falling within the application example embodiment.
Meanwhile, the application describes embodiments herein using particular words.Such as " one embodiment ", " one implements Example ", and/or " some embodiments " mean a certain feature related to the embodiment of the application at least one, structure or feature.Cause This, it should be highlighted that and it is noted that " embodiment " or " enforcement that refer to twice or repeatedly in diverse location in this specification Example " or " alternate embodiment " are not necessarily meant to refer to same embodiment.Additionally, in one or more embodiments of the application Some features, structure or feature can carry out appropriate combination.
Additionally, it will be understood by those skilled in the art that each side of the application can pass through some with patentability Species or situation are illustrated and described, including the combination of any new and useful operation, machine, product or material, or right Their any new and useful improvement.Correspondingly, the various aspects of the application can completely by hardware perform, can be complete Performed, can also be performed by combination of hardware by software (including firmware, resident software, microcode etc.).Hardware above is soft Part is referred to alternatively as " data block ", " module ", " engine ", " unit ", " component " or " system ".Additionally, each side of the application The computer product being located in one or more computer-readable mediums may be shown as, the product includes computer-readable program Coding.
Computer-readable signal media may include a propagation data signal for being contained within computer program code, for example In base band or as a part for carrier wave.The transmitting signal may have many forms, including electromagnetic form, light form Etc. or suitable combining form.Computer-readable signal media can be any in addition to computer-readable recording medium Computer-readable medium, the medium can be by being connected to instruction execution system, device or an equipment to realize communicating, propagate Or transmission is for the program that uses.Program coding in computer-readable signal media can be entered by any suitable medium Row is propagated, including the combination of radio, cable, fiber optic cables, RF or similar mediums or any of above medium.
Computer program code needed for the operation of the application each several part can use any one or more programming language, Including Object-Oriented Programming Language such as Java, Scala, Smal ltalk, Eiffel, JADE, Emerald, C++, C#, VB.NET, Python etc., conventional procedural programming language for example C language, Visual Bas ic, Fortran 2003, Perl, COBOL 2002, PHP, ABAP, dynamic programming language such as Python, Ruby and Groovy, or other programming languages etc..The program Coding can on the user computer run completely or run on the user computer as independent software kit or partly exist Operation part is run in remote computer operation or completely on remote computer or server on subscriber computer.In rear kind In the case of, remote computer can be connected, for example, LAN (LAN) or wide area by any latticed form with subscriber computer Net (WAN), or externally connected computer (such as by the Internet), or in cloud computing environment, or as service using such as Software is serviced (SaaS).
It is the order of herein described processing element and sequence, number-letter additionally, except clearly stating in non-claimed Using or other titles use, be not intended to limit the order of the application flow process and method.Although by each in above-mentioned disclosure Kind of example discusses some it is now recognized that useful inventive embodiments, but it is to be understood that, such details only plays explanation Purpose, appended claims are not limited in the embodiment for disclosing, conversely, claim is intended to, and covering is all to meet the application The amendment of embodiment spirit and scope and equivalent combinations.For example, although system component described above can be set by hardware It is standby to realize, but only can also be achieved by the solution of software, pacify such as on existing server or mobile device The described system of dress.
In the same manner, it is noted that in order to simplify herein disclosed statement, it is real to one or more inventions so as to help Apply the understanding of example, above to the description of the embodiment of the present application in, sometimes by various features merger to one embodiment, accompanying drawing or In descriptions thereof.But, this disclosure method is not meant to be carried in the aspect ratio claim required for the application object And feature it is many.In fact, the feature of embodiment will be less than whole features of the single embodiment of above-mentioned disclosure.
Description composition, the numeral of number of attributes used in some embodiments, it should be appreciated that such for embodiment The numeral of description, has used in some instances qualifier " about ", " approximate " or " generally " to modify.Unless said in addition Bright, " about ", " approximate " or " generally " shows that the numeral allows ± 20% change.Correspondingly, in some embodiments In, the numerical parameter used in description and claims is approximation, approximation feature according to needed for separate embodiment Can change.In certain embodiments, numerical parameter is considered as the significant digit for specifying and is retained using general digit Method.Although the Numerical Range and parameter in the application some embodiments for confirming its scope range is approximation, concrete real In applying example, being set in for such numerical value is reported as precisely as possible in feasible region.
Each patent, patent application, patent application publication thing and the other materials quoted for the application, such as article, book Nationality, description, publication, document, object etc., spy is incorporated herein entire contents as reference.With teachings herein not Except application history file that is consistent or producing conflict, to the conditional file of the application claim widest scope (it is current or Be additional to afterwards in the application) also except.If it should be noted that description in the application attaching material, definition and/ Or the use of term with it is herein described it is interior have it is inconsistent or where conflicting, with the description of the present application, definition and/or term Use be defined.
Finally, it will be understood that embodiment described herein is only to illustrate the principle of the embodiment of the present application.Other Deformation be likely to belong to scope of the present application.Unrestricted accordingly, as example, the alternative configuration of the embodiment of the present application is visual It is consistent with teachings of the present application.Correspondingly, embodiments herein is not limited to the embodiment that the application is clearly introduced and described.

Claims (13)

1. a kind of method, including：
The first image is obtained, described first image includes first area；
The second image is obtained, second image includes second area；
The 3rd image is obtained, the 3rd image includes the 3rd region；
Based on convolutional neural networks algorithm, the first stack features of the first area are extracted；
Based on the convolutional neural networks algorithm, the second stack features of the second area are extracted；
Based on the convolutional neural networks algorithm, the 3rd stack features in the 3rd region are extracted；
Calculate first data related to first stack features and second stack features；
Calculate second data related to first stack features and the 3rd stack features；And
Based on first data and second data, second image and the 3rd image are ranked up.
2. claim 1 methods described, including：
Set the second parameter, extract the second feature collection relevant with the second area, the second feature collection includes described the Two stack features；And
Based on second stack features, first stack features relevant with the first area are extracted.
3. claim 2 methods described, it is that feature based is selected to concentrate from the second feature and select second stack features Algorithm.
4. claim 3 methods described, the feature selecting algorithm is Correlation Based Feature Selection Algorithm.
5. claim 2 methods described, described or described second parameter includes that convolutional neural networks level number, convolution kernel are big At least one in little, convolution kernel number and the number of features of generation.
6. claim 1 methods described, first data are based on first stack features and the feature of second stack features Difference produce.
7. claim 1 methods described, second data are based on first stack features and the feature of the 3rd stack features Difference produce.
8. claim 1 methods described, including based on first data and second data, using LambdaMART algorithms To second image and the 3rd image sequence.
9. claim 1 methods described, including computer-aided diagnosises are used, positions firstth area in described first image Domain.
10. claim 1 methods described, described first image, second image or the 3rd image are PET image, CT At least one of image, SPECT images, MRI image, ultrasonoscopy.
11. 1 systems, including：
One image acquiring device, for obtaining the first image, the second image and the 3rd image, and described first image, First area, second area and the 3rd region are positioned respectively in second image and the 3rd image；And
One image analysis apparatus, including characteristic extracting module, a characteristics analysis module and an order module,
The characteristic extracting module is configured to, and based on convolutional neural networks algorithm, the first of the first area is extracted respectively 3rd stack features of stack features, the second stack features of the second area and the 3rd region,
The characteristics analysis module is configured to
Determine first stack features, first data related to second stack features,
Determine first stack features, second data related to the 3rd stack features, and
The order module is configured to
Second image or the 3rd image are sorted based on first data and second data.
System described in 12. claim 11, described image acquisition device is included and uses computer-aided diagnosises, in first figure The first area is positioned as in.
System described in 13. claim 11, described image acquisition device is included and uses computer-aided diagnosises, in second figure The second area is positioned as in.
CN201611149728.6A 2016-12-13 2016-12-13 Image searching system and method Pending CN106682127A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201611149728.6A CN106682127A (en) 2016-12-13 2016-12-13 Image searching system and method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201611149728.6A CN106682127A (en) 2016-12-13 2016-12-13 Image searching system and method

Publications (1)

Publication Number Publication Date
CN106682127A true CN106682127A (en) 2017-05-17

Family

ID=58868358

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201611149728.6A Pending CN106682127A (en) 2016-12-13 2016-12-13 Image searching system and method

Country Status (1)

Country Link
CN (1) CN106682127A (en)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108133745A (en) * 2017-12-21 2018-06-08 成都真实维度科技有限公司 It is a kind of using medical image as the clinical path partial data correlating method of core
CN108984587A (en) * 2017-05-31 2018-12-11 佳能株式会社 Information processing unit, information processing method, information processing system and storage medium
CN109086705A (en) * 2018-07-23 2018-12-25 北京旷视科技有限公司 Image processing method, device, electronic equipment and storage medium
CN109376741A (en) * 2018-09-10 2019-02-22 平安科技（深圳）有限公司 Recognition methods, device, computer equipment and the storage medium of trademark infringement
CN110598696A (en) * 2019-08-20 2019-12-20 上海联影医疗科技有限公司 Medical image scanning positioning method, medical image scanning method and computer equipment
CN112001044A (en) * 2020-07-31 2020-11-27 宁波浙大联科科技有限公司 Part CAD model similarity estimation method and system
CN112336381A (en) * 2020-11-07 2021-02-09 吉林大学 Echocardiogram end systole/diastole frame automatic identification method based on deep learning
CN113343033A (en) * 2021-06-29 2021-09-03 平安普惠企业管理有限公司 Video searching method and device, computer equipment and storage medium

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101419671A (en) * 2008-11-10 2009-04-29 北方工业大学 Face gender identification method based on fuzzy support vector machine
CN102855352A (en) * 2012-08-17 2013-01-02 西北工业大学 Method for clustering videos by using brain imaging space features and bottom layer vision features
CN105808732A (en) * 2016-03-10 2016-07-27 北京大学 Integration target attribute identification and precise retrieval method based on depth measurement learning
CN105989001A (en) * 2015-01-27 2016-10-05 北京大学 Image searching method and device, and image searching system

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101419671A (en) * 2008-11-10 2009-04-29 北方工业大学 Face gender identification method based on fuzzy support vector machine
CN102855352A (en) * 2012-08-17 2013-01-02 西北工业大学 Method for clustering videos by using brain imaging space features and bottom layer vision features
CN105989001A (en) * 2015-01-27 2016-10-05 北京大学 Image searching method and device, and image searching system
CN105808732A (en) * 2016-03-10 2016-07-27 北京大学 Integration target attribute identification and precise retrieval method based on depth measurement learning

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
董建巍: "基于相关特征选择的乳腺钼靶图像肿块检测", 《中国优秀硕士学位论文全文数据库 信息科技辑》 *

Cited By (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108984587A (en) * 2017-05-31 2018-12-11 佳能株式会社 Information processing unit, information processing method, information processing system and storage medium
CN108984587B (en) * 2017-05-31 2023-04-18 佳能株式会社 Information processing apparatus, information processing method, information processing system, and storage medium
CN108133745A (en) * 2017-12-21 2018-06-08 成都真实维度科技有限公司 It is a kind of using medical image as the clinical path partial data correlating method of core
CN108133745B (en) * 2017-12-21 2020-08-11 成都真实维度科技有限公司 Clinical path complete data correlation method taking medical image as core
CN109086705B (en) * 2018-07-23 2021-11-16 北京旷视科技有限公司 Image processing method, image processing device, electronic equipment and storage medium
CN109086705A (en) * 2018-07-23 2018-12-25 北京旷视科技有限公司 Image processing method, device, electronic equipment and storage medium
CN109376741A (en) * 2018-09-10 2019-02-22 平安科技（深圳）有限公司 Recognition methods, device, computer equipment and the storage medium of trademark infringement
CN110598696A (en) * 2019-08-20 2019-12-20 上海联影医疗科技有限公司 Medical image scanning positioning method, medical image scanning method and computer equipment
CN112001044A (en) * 2020-07-31 2020-11-27 宁波浙大联科科技有限公司 Part CAD model similarity estimation method and system
CN112001044B (en) * 2020-07-31 2024-04-02 宁波智讯联科科技有限公司 Method and system for estimating similarity of CAD model of part
CN112336381A (en) * 2020-11-07 2021-02-09 吉林大学 Echocardiogram end systole/diastole frame automatic identification method based on deep learning
CN112336381B (en) * 2020-11-07 2022-04-22 吉林大学 Echocardiogram end systole/diastole frame automatic identification method based on deep learning
CN113343033A (en) * 2021-06-29 2021-09-03 平安普惠企业管理有限公司 Video searching method and device, computer equipment and storage medium
CN113343033B (en) * 2021-06-29 2022-07-22 平安普惠企业管理有限公司 Video searching method and device, computer equipment and storage medium

Similar Documents

Publication Publication Date Title
CN106682127A (en) Image searching system and method
Wang et al. A multi-view deep convolutional neural networks for lung nodule segmentation
Liu et al. Segmentation of lung nodule in CT images based on mask R-CNN
CN111161275B (en) Method and device for segmenting target object in medical image and electronic equipment
CN109003267B (en) Computer-implemented method and system for automatically detecting target object from 3D image
US10853409B2 (en) Systems and methods for image search
CN109447998B (en) Automatic segmentation method based on PCANet deep learning model
Yi et al. Optimizing and visualizing deep learning for benign/malignant classification in breast tumors
CN109215033A (en) The method and system of image segmentation
CN111932529B (en) Image classification and segmentation method, device and system
CN106611413A (en) Image segmentation method and system
CN110310287A (en) It is neural network based to jeopardize the automatic delineation method of organ, equipment and storage medium
CN114693933A (en) Medical image segmentation device based on generation of confrontation network and multi-scale feature fusion
Sikder et al. Supervised learning-based cancer detection
Song et al. Automatic lung nodule segmentation and intra-nodular heterogeneity image generation
CN111127487B (en) Real-time multi-tissue medical image segmentation method
Ansari et al. Multiple sclerosis lesion segmentation in brain MRI using inception modules embedded in a convolutional neural network
CN113870279A (en) Multi-modal brain tumor image segmentation system and method
CN112150477B (en) Full-automatic segmentation method and device for cerebral image artery
Li et al. A new three-stage curriculum learning approach for deep network based liver tumor segmentation
CN115861716B (en) Glioma classification method and device based on twin neural network and image histology
Wu et al. Semiautomatic segmentation of glioma on mobile devices
Mansour et al. Kidney segmentations using cnn models
WO2022227193A1 (en) Liver region segmentation method and apparatus, and electronic device and storage medium
CN110706209B (en) Method for positioning tumor in brain magnetic resonance image of grid network

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20170517