CN108108437A - The design method that a kind of computer vision sense of hearing smell is mutually mapped with the time - Google Patents

Abstract

A kind of technical field for the design method that computer vision sense of hearing smell was mutually mapped with the time,It is to belong to,Robot,Artificial intelligence,Computer,Image procossing,Olfactory sensor technology,Hearing transducer,The technical fields such as mathematics,Major technique is the image gathered by camera,The data of hearing transducer acquisition and the data of olfactory sensor acquisition,It records simultaneously,Then calculating processing is passed through,The image of the smell generation allowed in picture and the smell smelt and the sound heard are mapped,And when being preserved by computing,It saves simultaneously,This time is also saved simultaneously,And using this time as mapping,So when receiving the smell that has preserved,It can be found in lane database,And pass through the time of record,Find corresponding image and sound simultaneously,When seeing that sub-picture,Corresponding smell and sound are found also by time map,When hearing such sound,Corresponding image and smell can also be found by time map.

Description

The design method that a kind of computer vision sense of hearing smell is mutually mapped with the time

Technical field

A kind of technical field for the design method that computer vision sense of hearing smell was mutually mapped with the time, is to belong to, robot, Technical fields, the major techniques such as artificial intelligence, computer, image procossing, olfactory sensor technology, hearing transducer, mathematics are The data that the data and olfactory sensor that image, the hearing transducer gathered by camera gathers gather, are recorded simultaneously, Then by calculating processing, the image of the smell generation allowed in picture and the smell smelt and the sound heard are mapped, And it when being preserved by computing, saves simultaneously, this time is also saved simultaneously, and using this time to reflect It penetrates, so when receiving the smell that has preserved, can be found in lane database, and pass through the time of record, while find correspondence Image and sound, when seeing that sub-picture, corresponding smell and sound are found also by time map, when hearing such sound Sound can also find corresponding image and smell by time map.

Background technology

Intelligent robot is that most complicated robot and the mankind most thirst for the machine friend that can be manufactured early Friend.However to produce an intelligent robot and be not easy to, it is only to allow the walking motion of the machine simulation mankind, scientists The effort of tens of or even upper a century will have been paid.Nineteen twenty, Czechoslovakia writer Ka Leier Karel apeks were in his section It is unreal it is small be right, according to Robota（Czech's text, original meaning are " forced labour, hard work "）And Robotnik（Polish, original meaning are " worker "）, Create " robot " this word.The domestic appliance of Westinghouse Electrical Corp. manufacture has been put on display in nineteen thirty-nine USA New York World Expo People Elektro.It is controlled by cable, can be walked, and can say 77 words, it might even be possible to it smokes, but from living go back of really doing chores It is not by a long chalk.But it allows people to become more specific to the hope of domestic robot.Nineteen forty-two U.S.'s science fiction great master's Asimov carries Go out " three law of robot ".Although this is the creation in science fiction, later as the research and development principle of academia's acquiescence. Promise Bert wiener is published within 1948《Cybernetics --- on the science for controlling and communicating in animal and machine》, elaborate machine Nerve, the common law of sensor capability of communication and Control function and people in device, take the lead in proposing using computer as core from Dynamic chemical plant.Marvin's Ming Siji proposes his view to intelligence machine in Dartmouth meeting within 1954：Intelligent machine Device " can create the abstract model of ambient enviroment, if encountered problems, can find one's way out from abstract model ".This Definition influences the research direction of later 30 years intelligent robots.American's George's de Waele produces in the world within 1956 The programmable robot of First, and have registered patent.This manipulator can be engaged in different work according to different programs, because This is with versatility and flexibility.Nineteen fifty-nine de Waele jointly produces first with American inventor Joseph's English lattice Burger Platform industrial robot.Then, first hand robot building factory in the world has been set up --- Unimation companies.Due to English lattice Research and development and publicity of the Burger to industrial robot, he is also referred to as " father of industrial robot ".AMF Inc. of U.S. life in 1962 Output " VERSTRAN "（Mean omnipotent carrying）, with Unimation companies production Unimate as become really commercialization Industrial robot, and export to countries in the world, started upsurge of the whole world to robot and robot research.Sensor Application improve the operability of robot.People try to install various sensors in robot, including 1961 The touch sensor that year Ernest ＆ Whitney uses asks Abramovich and Bo Ni 1962 to use pressure in earliest " Dextrous Hand " in the world Force snesor, and mccarthy then starts to add in visual sensing system in robot for 1963, and in 1964 years, MIT is helped to push away Gone out in the world first carry visual sensor, can identify and position the robot system of building blocks.Nineteen sixty-five John Hope Develop Beast robots in Jin Si universities Experiment of Applied Physics room.Beast can by devices such as Sonar system, photoelectric tubes, According to the position of environmental correction oneself.Middle 1960s, Massachusetts Institute Technology, Stanford University, Britain Robot laboratory has been set up successively in Edinburgh University etc..Rise research second generation belt sensor, the machine of " feeling " in the U.S. People, and set out to artificial intelligence.Nineteen sixty-eight Stanford Research Institute of the U.S. announces them and researches and develops successful robot Shakey.It With visual sensor, can be found according to the instruction of people and capture building blocks, but control its computer there are one room so Greatly.Shakey can be the first in the world platform intelligent robot, pull open the prelude of third generation robot research and development.Japanese early rice Field university adds one youth laboratory of rattan to develop the robot that First is walked with both feet.One youth of rattan is added to be devoted for years in research apery Robot is known as " father of anthropomorphic robot ".Japanese expert one is to research and develop the technology of anthropomorphic robot and amusement robot It is good at, later further, expedites the emergence of out the ASIMO of Honda Company and the QRIO of Sony.In the world first time robot and Minicomputer is cooperated, the robot T3 for Cincinnati Milacron companies of the U.S. of being just born.The U.S. Unimation companies release popular industrial machine people PUMA, this indicates Industrial Robot Technology full maturity.PUMA is extremely The present is still operated in factory's First Line.English lattice Burger Zai Tui robot Helpmate, this robot can be patient in hospital It brings meal, drug delivery, send mail.In the same year, he also foretells：" I will allow robot to wash the floor, and cook, and go out to help my carwash, check peace Entirely ".Chinese Famous scholar Zhou Haizhong professors exist《By robot》It is foretold in one text：To middle period 21st century, nanometer robot By the thorough work for changing the mankind and life style.Lego Company of Denmark releases robot（Mind-storms）External member, allows machine Device people manufacture becomes with playing with building blocks, and relatively easy and energy is arbitrary assembled, and robot is made to start to enter into private world.1999 It is precious that Sony corporation of Japan releases dog humanoid robot love（AIBO）, it is sold out, is stepped from this amusement robot as robot at once One of approach into average family.IRobot companies of the U.S. are proposed dust collector robot Roomba within 2002, it can avoid hindering Hinder, Automated Design travelling route, moreover it is possible in not enough power supply, drive towards cradle automatically.Roomba is that sales volume is most in the world at present Greatly, most commercialized domestic robot.Authorised distributor of Beijing area of iRobot companies：Beijing microgrid Zhi Hong Science and Technology Ltd.s. In June, 2006, Microsoft release Microsoft Robotics Studio, and what robot modularized, platform unitized becomes Gesture is more and more apparent, and Bill Gates prophesy, domestic robot will have swept the globe quickly.Therefore it is important for intelligent robot It is exactly vision, smell and the sense of hearing to input information, and present technology is not well mapped vision, smell and the sense of hearing, because This has the present invention.

The content of the invention

Due to the fast development of artificial intelligence and robot, in existing technology, human use's camera can be well Identify some things, while human use's electronics nose energy distinguishes various smells well, the mankind also surpass in sense of hearing resolution capability The mankind are crossed, the ability of the identification of the mankind has been above in one-side technology, in the behavior of the mankind, when the mankind see fox, It will remember its smell, while remember the cry of fox, smell the smell of fox, will remember fox, while can also remember Its cry when the cry for hearing fox, can also remember fox, while remember the smell of fox, and present computer technology is also Have no idea to realize the effect of this association, therefore the present invention exactly solves this technology, realizes this work(mutually mapped Can, the most important thing of the invention is that they are connected by the time, is realized by time map.A kind of computer vision is listened Feel the design method that smell is mutually mapped with the time, it is characterized in that, the design that computer vision sense of hearing smell is mutually mapped with the time Method is made of 5 parts, and 1 is computer vision process part, and 2 be mechanical smell process part, and in terms of 3 are the mechanical sense of hearing, 4 are Correspondence is established, 5 be the method stored using the time as mapping, and computer vision process part mainly uses video camera Picture shooting is got off, picture is handled by various algorithms, then picture extraction feature and storage data into Row compares or is identified by deep learning method, and the method for smell processing is exactly that the gas in air is carried out with electronic nose Identification, these information are converted into data, these data is then allowed to store, auditory processing be exactly to sound signal frequencies into Row processing, is suitble to computer disposal by analog-to-digital conversion, establishes image, smell and the correspondence of the sense of hearing, when camera photographed One things just the feature extraction of this things, while electronic nose also smells the smell that this things is sent, simultaneously also by The different position arrangement of sound transducer, determines the sound that this substance is sent, it is also possible to first pass through the arrangement of sound transducer It hears the sound that things is sent, then further according to the concentration gradient of smell, determines the position of substance, then camera is turned round The image of substance is shot, then carries out feature extraction or first smells the smell that substance is sent, according to the concentration gradient side of determining of smell To, while hear the sound that substance is sent, by the arrangement of sound transducer, determine direction, then to shoot by camera, carry Feature is taken, carries out storage mode using the time as mapping, this is the most important thing, and machinery is allowed to remember when this occurs Thing using time memory as 3 association, possesses uniqueness and real-time, therefore using following design, when smell sensor is received To signal, at first the data with the lane database of storage are matched, and are known that it is what sends gas if matched Taste if unmatched, is positioned by the concentration gradient of smell, by positioning and according to concentration gradient, mechanically moving Vision goes to shoot, and at this moment starts picture Processing Technique, while receives the sound that this things is sent, and sound is handled, This progress computing, it is then determined that whether to store, this phenomenon is stored if necessary, just removes time timer of transferring Time, while the data after picture treated material property and smell treated data and auditory processing are stored respectively Get up, be stored in the space set up for each, they are mapped by the time, if mechanical nose after so Smell has been smelt, just the data that this smell and lane database store have been compared, if finding such data, has just recalled and works as Then the time just remembered looks for corresponding picture and sound according to this time, similarly if computer vision sees things, , if matched, will will recall the time remembered originally with the Data Matching of storage, by the time of memory, pair when Between scan for finding corresponding smell and sound, if hearing sound, will be matched with the data of storage, if matching It is upper, the time remembered originally is just recalled, corresponding picture and smell are then looked for according to this time, it so can be mutual It is corresponding, and it is known that when this occurs.

Description of the drawings

Fig. 1 is the design method schematic diagram that a kind of computer vision sense of hearing smell is mutually mapped with the time, and what a-1 was represented is Odor data storehouse, what a-2 was represented is image data base, and a-3 represents audible data storehouse, and what b-1 was represented is pair of smell and image It should be related to, what b-2 was represented is image and the correspondence of the sense of hearing, and what b-3 was represented is smell and the correspondence of the sense of hearing, c-1 generations Table be odor data storehouse time portion, c-2 represent is odor data storehouse data portion, d-1 represent be image Time portion, what d-2 was represented is the data portion of image, and what e-1 was represented is the time portion of the sense of hearing, and what e-2 was represented is the sense of hearing Data portion, following circle represents many meanings.

Embodiment

After a robot starts, system enters working condition, its mechanical smell, computer vision and the sense of hearing is also into just Normal state, at this moment the olfactory system of robot can ceaselessly receive external smell and be passed to, auditory system ceaselessly receives outside Sound, camera ceaselessly shoots various substances, the design method that computer vision sense of hearing smell is mutually mapped with the time, by 5 Part is formed, and 1 is computer vision process part, and 2 be mechanical smell process part, and in terms of 3 are the mechanical sense of hearing, 4 be to establish to correspond to Relation, 5 be the method stored using the time as mapping, and computer vision process part mainly claps picture using video camera Photograph come, picture is handled by various algorithms, then picture extraction feature and storage data be compared or It being identified by deep learning method, the method for smell processing is exactly that the gas in air is identified with electronic nose, These information are converted into data, these data is then allowed to store, and auditory processing is exactly that sound signal frequencies are handled, Computer disposal is suitble to by analog-to-digital conversion, establishes image, smell and the correspondence of the sense of hearing, when camera photographed a things Just the feature extraction of this things, while electronic nose also smells the smell that this things is sent, simultaneously also by sound sensor The different position arrangement of device, determines the sound that this substance is sent, it is also possible to which things is heard in the arrangement for first passing through sound transducer The sound sent then further according to the concentration gradient of smell, determines the position of substance, then camera is turned round and shoots substance Image, then carry out feature extraction or first smell the smell that substance is sent, direction is determined according to the concentration gradient of smell, simultaneously It hears the sound that substance is sent, by the arrangement of sound transducer, determines direction, then to shoot by camera, extract feature, Storage mode is carried out using the time as mapping, this is the most important thing, machinery is allowed to remember that this is the thing when occurred, with the time Memory possesses uniqueness and real-time as 3 association, therefore using following design, when smell sensor receives signal, It is first matched with the data of the lane database of storage, is known that it is what scents of if matched, if It unmatches, is positioned by the concentration gradient of smell, by positioning and according to concentration gradient, mechanically moving vision goes to clap Take the photograph, at this moment start picture Processing Technique, while receive the sound that this things is sent, sound is handled, this into Row computing it is then determined that whether to store, stores this phenomenon if necessary, the time for the time timer that just goes to transfer, simultaneously Data after picture treated material property and smell treated data and auditory processing are stored respectively, are stored In the space set up for each, they are mapped by the time, if mechanical nose has smelt gas after so Taste is just compared the data that this smell and lane database store, if finding such data, just recalls what is remembered originally Then time looks for corresponding picture and sound according to this time, similarly if computer vision sees things, will and store up The Data Matching deposited if matched, will recall the time remembered originally, by the time of memory, the time is searched Rope finds corresponding smell and sound, if hearing sound, will be matched with the data of storage, if matched, just The time remembered originally is recalled, corresponding picture and smell are then looked for according to this time, can so be corresponded, and And it is known that when this occurs.As shown in drawings, when needing storage, according to the time, store simultaneously, example Such as 01 month 2013 No. 01,12 points 12 minutes, smell the smell of fox, just 01 month 2013 No. 01,12 points of 12 minutes and foxes Odor data be collectively stored in together, while 01 month 2013 No. 01, the characteristics of image of 12 points of 12 minutes and foxes is stored in Together, and at this moment hear the cry of fox, then just 01 month 2013 No. 01,12 points of 12 minutes this times and this sound Data store simultaneously, then establish algorithm, establish correspondence, if be transmitted through come fox smell and database number According to matching, just show this smell, at the same by 01 month 2013 No. 01, the storage of 12 points of 12 minutes removal search characteristics of image is empty Between time portion, so find the matched time, then just fox shown, at the same 01 month 2013 No. 01,12 points 12 this time went audible data library searching data, with regard to the sound of fox can be found, with same reason, and if seeing fox, The characteristics of image of image data library lookup fox can be arrived, the characteristics of image of fox is shown, while with the time 2013 Year 01 month No. 01,12: 12 assign to odor data storehouse into line search, find the data of smell, then show, while 2013 01 month year No. 01,12: 12 this time went audible data library searching data, with regard to that can find the sound of fox, similarly if heard The sound of fox, with 01 month 2013 No. 01,12: 12 this time, find the characteristics of image of fox and the smell of fox, It can thus be realized according to time correspondence mappings relation, and know and when occur.

Claims (1)

1. the design method that a kind of computer vision sense of hearing smell is mutually mapped with the time, it is characterized in that, computer vision sense of hearing smell The design method mutually mapped with the time is made of 5 parts, and 1 is computer vision process part, and 2 be mechanical smell process part, In terms of 3 are the mechanical sense of hearing, 4 be to establish correspondence, and 5 be the method stored using the time as mapping, and computer vision is handled Picture shooting is mainly got off using video camera in part, and picture is handled by various algorithms, then picture is extracted Feature and storage data be compared or be identified by deep learning method, smell processing method be exactly to use electronics The gas in air is identified in nose, these information are converted into data, these data are then allowed to store, auditory processing Exactly sound signal frequencies are handled, computer disposal is suitble to by analog-to-digital conversion, establish image, smell and pair of the sense of hearing It should be related to, when camera photographed a things just the feature extraction of this things, while electronic nose also smells this things hair The smell gone out arranges simultaneously also by the different position of sound transducer, determines the sound that this substance is sent, it is also possible to first lead to The sound that things is sent is heard in the arrangement for crossing sound transducer, then further according to the concentration gradient of smell, determines the position of substance, Then camera is turned round and shoots the image of substance, then carried out feature extraction or first smell the smell that substance is sent, according to gas The concentration gradient of taste determines direction, while hears the sound that substance is sent, and by the arrangement of sound transducer, determines direction, so It is shot afterwards by camera, extracts feature, carry out storage mode using the time as mapping, this is the most important thing, and machinery is allowed to remember This is the thing when occurred, using time memory as 3 association, possesses uniqueness and real-time, therefore using as follows Design, when smell sensor receives signal, at first the data with the lane database of storage are matched, and are known if having matched Road is what scents of, if unmatched, is positioned by the concentration gradient of smell, passes through positioning and root According to concentration gradient, mechanically moving vision goes to shoot, and at this moment starts picture Processing Technique, while receives the sound that this things is sent Sound handles sound, this progress computing, it is then determined that whether to store, stores this phenomenon if necessary, just It goes to transfer time of time timer, while at picture treated material property and smell treated data and the sense of hearing Data after reason store respectively, are stored in the space set up for each, they are mapped by the time, this If mechanical nose has smelt smell after sample, just the data that this smell and lane database store are compared, if found Such data just recall the time remembered originally, then look for corresponding picture and sound according to this time, similarly such as Fruit computer vision sees things, if matched, will will recall the time remembered originally with the Data Matching of storage, By the time of memory, the time is scanned for find corresponding smell and sound, it, will be with the number of storage if hearing sound According to being matched, if matched, the time remembered originally is just recalled, corresponding picture is then looked for according to this time And smell, it can so correspond, and be known that when this occurs.