CN103697900A - Method for early warning on danger through augmented reality by vehicle-mounted emotional robot - Google Patents
Method for early warning on danger through augmented reality by vehicle-mounted emotional robot Download PDFInfo
- Publication number
- CN103697900A CN103697900A CN201310692727.6A CN201310692727A CN103697900A CN 103697900 A CN103697900 A CN 103697900A CN 201310692727 A CN201310692727 A CN 201310692727A CN 103697900 A CN103697900 A CN 103697900A
- Authority
- CN
- China
- Prior art keywords
- emotion
- danger
- machines people
- dangerous
- vehicle machines
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/26—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network
- G01C21/34—Route searching; Route guidance
- G01C21/36—Input/output arrangements for on-board computers
- G01C21/3626—Details of the output of route guidance instructions
- G01C21/3661—Guidance output on an external device, e.g. car radio
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C11/00—Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/16—Anti-collision systems
- G08G1/165—Anti-collision systems for passive traffic, e.g. including static obstacles, trees
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/16—Anti-collision systems
- G08G1/166—Anti-collision systems for active traffic, e.g. moving vehicles, pedestrians, bikes
Landscapes
- Engineering & Computer Science (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Multimedia (AREA)
- Processing Or Creating Images (AREA)
Abstract
The invention provides a vehicle-mounted robot which is formed by combining mobile equipment with a device and can be placed nearby a motorist according to the actual requirement of the motorist, wherein the mobile equipment can be a smart phone which is taken as computing equipment. The vehicle-mounted robot can express the current emotion through the change of expression, can timely send out early warning information in case of various danger situations, and can be switched into an augmented reality mode to display a danger region in an augmenting way. Comprised with a present robot which takes a smart phone only as a computing device or a controller, the vehicle-mounted emotional robot can interact with users to express various danger information in an early warning way by means of expression, voice, images and texts, and action, and can be converted into the augmented reality mode to ensure that the motorist can know the current danger clearly.
Description
Technical field
The present invention relates to the application in mobile unit of artificial intelligence technology, Robotics and smart mobile phone, especially relate to a kind of emotion on-vehicle machines people and by augmented reality, carry out the method for danger early warning.
Background technology
Along with the surge of automobile pollution, the mileage of going on a journey per capita also increases substantially thereupon, and traffic becomes and becomes increasingly complex, and driver has also gradually had a kind of dependence to various Special vehicle-mounted equipment and the App application program based on smart mobile phone.Various automobile assistant driving equipment, Special vehicle-mounted navigator and the map App application program based on smart mobile phone etc. also show very actively on He Houzhuan market, auto-loading market.
Yet no matter existing product is that Special vehicle-mounted equipment or the App application program based on smart mobile phone still exist following problem:
(1) function singleness.This defect is mainly reflected on Special vehicle-mounted equipment, because this kind equipment need to carry out professional hardware design and Software for Design, the construction cycle of product is longer, cost is higher, be usually located at auto-loading market, need to the hand-in-glove of automobile production manufacturer, the function of product depend on early stage design and development, extendability is poor.The comparatively ripe function of Special vehicle-mounted equipment still rests on static path navigation and audiovisual multimedia application aspect at present, and for user, cost performance is on the low side.
(2) complex operation.No matter be Special vehicle-mounted equipment or mobile phone A pp, all comparatively loaded down with trivial details in operation, complicated operation brings inconvenience to driver, and driver cannot be operated expediently according to real-time demand when driving, causes the application in transit of this series products to experience poor.In fact, for driver, most important is exactly application in transit impression, still, and because the inconvenience of operation has seriously hindered user's frequency of usage and experience.
(3) interactivity is poor.At present, mobile unit and mobile phone A pp almost do not carry out mutual function with user, although also there is the Related product that voice technology is implanted in mobile unit or mobile phone A pp to appear on the market, but because reciprocal process still needs manual mode operation and phonetic recognization rate is lower etc. that problem causes interactive voice to be also difficult to break through current bottleneck in the application short time, therefore voice interactive function also becomes " chicken ribs " of this series products.
From the problems referred to above analysis, can find out, if can invent that a user's input cost is low, interactivity good, feature richness, vehicle-mounted product simple to operate, can meet well the demand in transit in vast driver drives vehicle process.
It is worth noting, increasing sensor is integrated on smart mobile phone in recent years, the function of smart mobile phone and calculated performance thereof also become from strength to strength, and smart mobile phone has become a kind of powerful mobile computing device, also more and more around the product of smart mobile phone research and development.CN201210387268.6 be take smart mobile phone and has been invented a kind of supervisory-controlled robot that is used in household safety-protection field as main body, mainly comprises smart mobile phone and robot platform, can realize the functions such as intelligent monitoring, long-range human-computer interactive control.CN201210189512.8 has invented a kind of solution magic square robot based on smart mobile phone, by smart mobile phone, gather magic square colouring information, complete out of order magic square reduction logical operation, and by Bluetooth communication, send to the control module in robot to control in real time solution, by performance element, complete the reduction to three rank magic squares.CN201210337637.0 has invented a kind of sweeping robot that adopts smart mobile phone navigation, by the sensor on smart mobile phone, obtain external information, avoided directly on sweeping robot, installing additional navigator fix and sensor hardware device, implemented simply, sweeping efficiency is high.CN201010543931.8 has invented a kind of robot man-machine interactive system based on iPhone smart mobile phone, by smart mobile phone, build portable general man-machine interactive system, thereby just can realize anywhere or anytime the tele-robotic straighforward operation of convenient and efficient without extras.CN201310160936.6 has invented a kind of emotion on-vehicle machines people of information-driven, this on-vehicle machines people expresses various information in succinct mode anthropomorphic and/or plan thing expression, be equipped with corresponding sound, allow user " just listen at a glance, " that just all are clear.In realization, user need obtain robot body device and App is installed on smart mobile phone and just can have emotion on-vehicle machines people, for customization and personalized information service is provided.
From foregoing invention, can find out, robot based on smart mobile phone will change traditional pattern of passing through the sealing autonomous system framework exploitation of developer based on oneself, R&D costs and the technical threshold of robot product are reduced, thereby promote robot product can serve vast common user.
But, existing invention lays particular emphasis on using smart mobile phone as control end, substitute the hardware calculation element using in conventional machines people performance history, be equivalent to reduce the cost of development of hardware, but its function aspects is too many improvement not, still belong to " hardware " and replace level, " intelligence " degree is lower.
The object of the invention is the actual demand from driver, with mobile device, if smart mobile phone is computing equipment, by a kind of device and described mobile device, carry out combination, construct a kind of on-vehicle machines people who is positioned over driver side, described on-vehicle machines people can express current emotion by the variation of expression, various danger situations is sent to early warning information in time simultaneously, and can switch to augmented reality pattern, strengthen and show hazardous location.Invent different from the existing robot calculating with opertaing device that only smart mobile phone served as, the present invention can carry out alternately with user by the emotion on-vehicle machines people of design, with { expression, sound, picture and text, action } all kinds of dangerous information are carried out to early warning expression, and can switch to augmented reality pattern, make driver can clearly know the danger of current existence.
Summary of the invention
The problem that the embodiment of the present invention will solve is to provide a kind of emotion on-vehicle machines people and by augmented reality, carries out the method for danger early warning, the problems such as existing vehicle-mounted assistant equipments on board function singleness, interactivity are poor to solve, complicated operation, finally, based on technical scheme of the present invention, arbitrary driver or traveler all can obtain danger early warning information in transit by emotion on-vehicle machines people of the present invention.For example, user can know that whether vehicle front road conditions are dangerous by observing emotion on-vehicle machines people's expression state, when there is intense strain in emotion on-vehicle machines people, show that the place ahead road conditions exist potential danger, emotion on-vehicle machines people can warn driver by the mode of expression+sound+action+risk markings, and driver also can switch to augmented reality pattern by clickable icon.
To achieve these goals, the present invention has provided a kind of emotion on-vehicle machines people and by augmented reality, has carried out the method for danger early warning, it is characterized in that, comprises the following steps:
(1) emotion on-vehicle machines people works in normal mode, by the image information of video acquisition module Real-time Obtaining vehicle front, and the image information obtaining is processed;
(2) emotion on-vehicle machines people is divided into three classes by the result of image information, without danger, potential danger and highly dangerous;
(3) if the result of described image information is that emotion on-vehicle machines people keeps current normal mode without dangerous;
(4) if the result of described image information is potential danger, emotion on-vehicle machines people is responded by intense strain immediately, with alerting drivers; Described emotion on-vehicle machines people's intense strain is integrated and is expressed by expression, sound, action and dangerous alert flag; Described dangerous alert flag is word and/or picture;
(5) the dangerous alert flag that taps or slide, emotion on-vehicle machines people switches to augmented reality pattern, strengthens and shows potential danger situation, after described dangerous disappearance, automatically switches to normal mode;
(6) if the result of described image information is highly dangerous, emotion on-vehicle machines people is by expression and the frightened mood of acoustic expression, and be directly switch into augmented reality pattern, and strengthen and show highly dangerous situation, after described dangerous disappearance, automatically switch to normal mode.
Described emotion on-vehicle machines people comprises mobile device and robot body, described mobile device includes but not limited to communication module, video acquisition module, phonetic entry output module, expression respond module, message processing module and data transmission module, sensor assembly, described sensor assembly includes but not limited to GPS, electronic compass, gyroscope, shock sensor, ambient light sensor, acceleration transducer, gravity sensor, range sensor, described robot body includes but not limited to the stationary installation of fixing described mobile device, carry out the data transmission device of exchanges data with described mobile device, robot body actuator, receive the fill order that robot body actuator sends, and according to fill order drive machines human body rotation or before changing mobile device, face upward or the drive unit of the angle of swinging back, make robot body carry out the telecontrol equipment of 360 degree rotations, make robot body with automobile instrument panel top platen or front windshield or hang on the coupling arrangement that air-conditioner air outlet suspension member is connected, for robot body and mobile device provide the electric supply installation of power supply,
Described emotion on-vehicle machines people expresses mood by expression, sound, action, word and/or picture; Described mood includes but not limited to calmness, happiness, excitement, anxiety, fear, indignation;
Described emotion on-vehicle machines people mode of operation is divided into normal mode of operation and augmented reality pattern; Described normal mode of operation, shows expression element on mobile device screen; Described augmented reality pattern, the actual image of vehicle front that on mobile device screen, display video acquisition module obtains and be added on the supplementary in described actual image, realizes real enhancing;
Described emotion on-vehicle machines people carries out data interaction by wired or wireless communication mode and automobile CAN-bus and Cloud Server.
Beneficial effect of the present invention is:
(1) the present invention takes the principle of simplifying, and in succinct mode anthropomorphic and/or that intend thing expression, various information is expressed, and is equipped with corresponding sound simultaneously, makes user " just listen at a glance, " that just all are clear;
(2) user only need download App, just mobile phone can be become to emotion on-vehicle machines people, judges the danger situation of vehicle front by affiliated emotion on-vehicle machines people's expression shape change;
(3) emotion on-vehicle machines people adopts augmented reality, and the danger situation that can exist vehicle front carries out identification, and hazardous location is strengthened to demonstration, is convenient to driver and knows fast the danger that may exist.
Accompanying drawing explanation
Fig. 1-1st, one of structural representation of one embodiment of the invention emotion on-vehicle machines people;
Fig. 1-2 be one embodiment of the invention emotion on-vehicle machines people structural representation two;
Fig. 2 is that one embodiment of the invention emotion on-vehicle machines people is in augmented reality mode of operation schematic diagram;
Fig. 3 is one embodiment of the invention emotion on-vehicle machines people danger early warning workflow;
Fig. 4 is that one embodiment of the invention emotion on-vehicle machines people differentiates dangerous frame of video flow process;
Fig. 5 is emotion expression service process after one embodiment of the invention emotion on-vehicle machines people perception danger;
Fig. 6 is the element that one embodiment of the invention emotion on-vehicle machines people intense strain comprises;
Fig. 7 is that one embodiment of the invention emotion on-vehicle machines people warns process to potential danger.
Embodiment
The preferred embodiments of the present invention provide a kind of emotion on-vehicle machines people by augmented reality, to carry out the method for danger early warning.This emotion on-vehicle machines people is assembled jointly by mobile device and robot body parts, mobile device is fixed on robot body, and carry out information transmission by data line and robot body, mobile device can send control command to robot body according to the variation of external information simultaneously, drive machines human body is rotated or mobile device is faced upward or swings back before carrying out on vertical plane, and current information is responded with sound, picture and expression by the App being installed on mobile device.
Described emotion on-vehicle machines people comprises mobile device and robot body, described mobile device includes but not limited to communication module, video acquisition module, phonetic entry output module, expression respond module, message processing module and data transmission module, sensor assembly, described sensor assembly includes but not limited to GPS, electronic compass, gyroscope, shock sensor, ambient light sensor, acceleration transducer, gravity sensor, range sensor, described robot body includes but not limited to the stationary installation of fixing described mobile device, carry out the data transmission device of exchanges data with described mobile device, robot body actuator, receive the fill order that robot body actuator sends, and according to fill order drive machines human body rotation or before changing mobile device, face upward or the drive unit of the angle of swinging back, make robot body carry out the telecontrol equipment of 360 degree rotations, make robot body with automobile instrument panel top platen or front windshield or hang on the coupling arrangement that air-conditioner air outlet suspension member is connected, for robot body and mobile device provide the electric supply installation of power supply,
Described emotion on-vehicle machines people expresses mood by expression, sound, action, word and/or picture; Described mood includes but not limited to calmness, happiness, excitement, anxiety, fear, indignation;
Described emotion on-vehicle machines people mode of operation is divided into normal mode of operation and augmented reality pattern; Described normal mode of operation, shows expression element on mobile device screen; Described augmented reality pattern, the actual image of vehicle front that on mobile device screen, display video acquisition module obtains and be added on the supplementary in described actual image, realizes real enhancing;
Described emotion on-vehicle machines people carries out data interaction by wired or wireless communication mode and automobile CAN-bus and Cloud Server.Described emotion on-vehicle machines people can be undertaken alternately, obtaining vehicle-related information by OBD interface and automobile CAN-bus; Described emotion on-vehicle machines people and Cloud Server are undertaken alternately, uploading and/or downloading data by communication, send and/or receive order.
Fig. 1-1st, one of structural representation of one embodiment of the invention emotion on-vehicle machines people, consists of mobile device and robot body.The 101st, the mobile device of operating system is installed, described operating system includes but not limited to Android or iOS or Windows etc.; Described mobile device includes but not limited to communication module, video acquisition module, phonetic entry output module, expression respond module, message processing module and data transmission module etc.101 mobile devices are connected with 103 robot bodies, described robot body comprises the stationary installation of fixing described mobile device, carry out the data transmission device of exchanges data with described mobile device, robot body actuator, receive the fill order that robot body actuator sends, and according to fill order drive machines human body rotation or before changing mobile device, face upward or the drive unit of the angle of swinging back, make robot body carry out the telecontrol equipment of 360 degree rotations, the coupling arrangement that robot body is connected with automobile instrument panel top platen, for robot body and mobile device provide the electric supply installation of power supply.
The 102nd, be installed on the App Application Program Interface example on mobile device 101, this App application program mainly presents by expression mode anthropomorphic and/or plan thing, and 102 shown expressions can be carried out real-time variation response according to the information receiving.Described variation comprises the change of predetermined various expressions, also comprises current expression interface is switched to other application program interface styles such as picture, word, form, webpage, video, map.
103 is robot body, can be connected with automobile instrument panel top table top or hangs on on-board air conditioner mouth place or stick on front windshield or be built in the table top of automobile instrument panel top.
When emotion on-vehicle machines people is during in normal mode of operation, one of its expression can be as shown in 102, and along with the various information receiving is different and change thereupon.
Fig. 1-2 be one embodiment of the invention emotion on-vehicle machines people structural representation two.When emotion on-vehicle machines people experiences the place ahead when dangerous, it becomes tense situation, described tense situation be embodied in nervous expression, nervous sound, tonic action and picture and word caution.Particularly, 101 mobile devices can send order-driven robot body 103 and rotate on surface level, and/or 101 mobile device at vertical plane, go forward to face upward or swing back to express tonic action, as successively respectively rotated clockwise and counterclockwise 5 degree a given time inner machine human body, before mobile device, face upward 3 degree and layback 3 degree simultaneously; Nervous expression can be carried out Expression by similar 102 expression mode, sends nervous mood sound simultaneously or expresses statement nervous and/or that fear; Picture and word caution can by 104 on 102 expression interfaces Overlapping display.
Fig. 2 is that one embodiment of the invention emotion on-vehicle machines people is in augmented reality mode of operation schematic diagram.When the manual inspection of emotion on-vehicle machines, to measure vehicle front dangerous, and undertaken after danger warning by 104 picture and word caution marks, can 104 switch to augmented reality and show mode of operation by clicking or sliding.Under augmented reality mode of operation, mobile device passes through video acquisition module Real-time Obtaining vehicle front image information, and shows on screen, sees shown in 201; 202 hazardous location label informations are added in the hazardous location detecting simultaneously, highlight dangerous object and position.
Specific embodiment, suppose that current emotion on-vehicle machines people is in normal mode of operation, its expression state is as shown in 102, when the place ahead road conditions being detected when dangerous, emotion on-vehicle machines people sends intense strain, by anxiety express one's feelings, nervous sound, tonic action and picture and word caution express.When perception emotion on-vehicle machines people expresses intense strain, can be by tapping or the picture and word caution mark 104 that slides switches to augmented reality mode of operation, and shown by 201 and 202.
Fig. 3 is one embodiment of the invention emotion on-vehicle machines people danger early warning workflow.Emotion on-vehicle machines people carries out a method for danger early warning by augmented reality, comprise the following steps:
301 is initial phase, is about to mobile device and is connected with robot body, and on-vehicle machines people completes initialization procedure, and starts App application program, enters the normal mode of operation state of 302 acquiescences;
302 is acquiescence normal mode of operation, and under this mode of operation, emotion on-vehicle machines people can be responded the various information of obtaining in the variation by expression+sound+action;
303 is that emotion on-vehicle machines people passes through the image information of video acquisition module Real-time Obtaining vehicle front, and the image information obtaining is processed;
304 are divided into three classes for emotion on-vehicle machines people by the result of image information, without danger, potential danger and highly dangerous;
305 confirm module for differentiating, if without danger, go to 306, otherwise go to 307;
306 is that emotion on-vehicle machines people keeps current normal mode of operation, goes to 303 if the result of described image information is dangerous for nothing;
Whether 307 continue existing potential danger to judge;
If 308 for the result of described image information is potential danger, emotion on-vehicle machines people is responded by intense strain immediately, with alerting drivers; Described emotion on-vehicle machines people's intense strain is integrated and is expressed by expression, sound, action and dangerous alert flag; Described dangerous alert flag is word and/or picture;
309 for tapping or slide dangerous alert flag, and emotion on-vehicle machines people switches to augmented reality pattern, strengthens and show potential danger situation;
310 for automatically switching to normal mode of operation after described dangerous disappearance, goes to 303;
If 311 for the result of described image information is highly dangerous, emotion on-vehicle machines people passes through expression and the frightened mood of acoustic expression, and is directly switch into augmented reality pattern, strengthens and shows highly dangerous situation;
312 for automatically switching to normal mode of operation after described dangerous disappearance, goes to 303.
Fig. 4 is that one embodiment of the invention emotion on-vehicle machines people differentiates dangerous frame of video flow process.It is further comprising the steps of that emotion on-vehicle machines people processes image information:
401 process real-time video flowing for on-vehicle machines people, dangerous frame of video whether in identification video flowing; If there is no dangerous frame of video, is considered as current vehicle front without danger;
402 is if dangerous frame of video is considered as current vehicle front dangerous, and mark is carried out in the hazardous location in described dangerous frame of video, turns 403;
403 according to the locus of the hazardous location of described mark, judges that the danger that hazardous location exists is general dangerous probability SD
lprobability SD with highly dangerous
h;
404 is to judge the distance between the hazardous location of mark in video acquisition module and dangerous frame of video by image processing algorithm, is called risk distance;
405 for judging that according to described risk distance current danger is as general dangerous probability DD
lprobability DD with highly dangerous
h;
406 for to calculate the danger classes of current existence according to formula 3-1:
D
rank=αSD
L+βSD
H+γDD
L+θDD
H 3-1
In formula 3-1, D
rankfor danger classes, α, β, γ, the θ parameter for demarcating, SD
lfor hazardous location is general dangerous probability, SD
hfor hazardous location be highly dangerous probability, DD
lfor risk distance is general dangerous probability, DD
hfor the risk distance probability that is highly dangerous;
407 for the danger classes of calculating in 406 and the threshold value λ of setting are compared, if D
rank< λ, current danger is potential danger, if D
rank>=λ, current danger is highly dangerous.
Fig. 5 is the emotion expression service process after one embodiment of the invention emotion on-vehicle machines people perception danger.Emotion on-vehicle machines people comprises the following steps by intense strain alerting drivers potential danger:
Emotion on-vehicle machines people knows that the place ahead exists after potential danger, by 501, further hazard types is carried out to identification, described hazard types includes but not limited to that leading vehicle distance is too near with speeding, the place ahead has pedestrian, the place ahead to have that there is barrier in bicycle, the place ahead, the place ahead visible rays is weak, front light is strong;
502 pick out after concrete hazard types for emotion on-vehicle machines people, in the mode of word and/or picture, generate or call the warning against danger sign matching with described hazard types;
503 show described warning against danger sign for emotion on-vehicle machines people on mobile device screen, trigger intense strain event simultaneously;
504 is that after emotion on-vehicle machines people intense strain triggers, emotion on-vehicle machines people's expression changes tense situation into, sends nervous sound simultaneously and follows nervous action; The action of described anxiety refers to that described robot body rotates on surface level and/or mobile device is faced upward and/or backward wing before doing on vertical plane;
505 is that emotion on-vehicle machines people finally realizes the warning to potential danger by intense strain.
Fig. 6 is the element that one embodiment of the invention emotion on-vehicle machines people intense strain comprises.601 is intense strain, comprising 602 nervous expressions, 603 nervous sound, 604 picture and word cautions and 605 tonic actions.Described 602 nervous expressions can be expressed by static or dynamic all kinds of expression element; Described 603 nervous sound can or be expressed nervous language and express by nervous mood sound; Described 604 picture and word cautions can be shown by the mode of static images or dyhamic video frame or word or figure; Described 605 tonic actions can be expressed by the rotation of robot body and mobile device.
Strengthen demonstration potential danger situation or highly dangerous situation further comprising the steps of:
(1) emotion on-vehicle machines people switches to augmented reality pattern by normal mode, the image information that on mobile device screen, display video acquisition module obtains in real time;
(2) by regular or irregular and with coloured lines, mark is carried out in the hazardous location occurring in frame of video;
(3) by image processing algorithm, calculate the distance in video acquisition module and actual danger region, and adopt the distance with coloured line or face mark video acquisition module and described hazardous location in the image frame obtaining;
(4) distance in the car speed obtaining according to mobile device sensor and described video acquisition module and actual danger region, calculates vehicle and arrives at the required time of described hazardous location, and described temporal information is strengthened to demonstration in the image frame obtaining.
Fig. 7 is that one embodiment of the invention emotion on-vehicle machines people warns process to potential danger.A method for dangerous frame whether in emotion on-vehicle machines people identification video flowing, comprises the following steps:
701 for to process dissimilar danger situation picture, extracts feature, and by every class danger situation characteristic storage in background server or Cloud Server;
702 for emotion on-vehicle machines people obtains the frame in video flowing with the sampling interval of setting, and the described frame picture obtaining is analyzed to extraction feature, then with described server in every class danger situation feature of storing compare;
If 703 the match is successful, by 704, return to warning against danger, and the hazard types that exists of judgement, turn 705;
705 according to hazard types and the described risk distances of described judgement, and whether emotion on-vehicle machines people determines switching working mode, further by 706 warnings that realize potential danger;
If 703 couplings are unsuccessful, emotion on-vehicle machines people is considered as the place ahead without danger, in next sampling interval, continues to obtain the frame in video flowing, turns 702.
The present invention describes by specific embodiment, it will be appreciated by those skilled in the art that, without departing from the present invention, can also carry out various conversion and be equal to alternative the present invention.In addition, for particular condition or concrete condition, can make various modifications to the present invention, and not depart from the scope of the present invention.Therefore, the present invention is not limited to disclosed specific embodiment, and should comprise the whole embodiments that fall within the scope of the claims in the present invention.
Claims (6)
1. emotion on-vehicle machines people carries out a method for danger early warning by augmented reality, it is characterized in that, comprises the following steps:
(1) emotion on-vehicle machines people works in normal mode, by the image information of video acquisition module Real-time Obtaining vehicle front, and the image information obtaining is processed;
(2) emotion on-vehicle machines people is divided into three classes by the result of image information, without danger, potential danger and highly dangerous;
(3) if the result of described image information is that emotion on-vehicle machines people keeps current normal mode without dangerous;
(4) if the result of described image information is potential danger, emotion on-vehicle machines people is responded by intense strain immediately, with alerting drivers; Described emotion on-vehicle machines people's intense strain is integrated and is expressed by expression, sound, action and dangerous alert flag; Described dangerous alert flag is word and/or picture;
(5) the dangerous alert flag that taps or slide, emotion on-vehicle machines people switches to augmented reality pattern, strengthens and shows potential danger situation, after described dangerous disappearance, automatically switches to normal mode;
(6) if the result of described image information is highly dangerous, emotion on-vehicle machines people is by expression and the frightened mood of acoustic expression, and be directly switch into augmented reality pattern, and strengthen and show highly dangerous situation, after described dangerous disappearance, automatically switch to normal mode.
2. as claimed in claim 1, a kind of emotion on-vehicle machines people carries out the method for danger early warning by augmented reality, it is characterized in that:
Described emotion on-vehicle machines people comprises mobile device and robot body, described mobile device includes but not limited to communication module, video acquisition module, phonetic entry output module, expression respond module, message processing module and data transmission module, sensor assembly, described sensor assembly includes but not limited to GPS, electronic compass, gyroscope, shock sensor, ambient light sensor, acceleration transducer, gravity sensor, range sensor, described robot body includes but not limited to the stationary installation of fixing described mobile device, carry out the data transmission device of exchanges data with described mobile device, robot body actuator, receive the fill order that robot body actuator sends, and according to fill order drive machines human body rotation or before changing mobile device, face upward or the drive unit of the angle of swinging back, make robot body carry out the telecontrol equipment of 360 degree rotations, make robot body with automobile instrument panel top platen or front windshield or hang on the coupling arrangement that air-conditioner air outlet suspension member is connected, for robot body and mobile device provide the electric supply installation of power supply,
Described emotion on-vehicle machines people expresses mood by expression, sound, action, word and/or picture; Described mood includes but not limited to calmness, happiness, excitement, anxiety, fear, indignation;
Described emotion on-vehicle machines people mode of operation is divided into normal mode of operation and augmented reality pattern; Described normal mode of operation, shows expression element on mobile device screen; Described augmented reality pattern, the actual image of vehicle front that on mobile device screen, display video acquisition module obtains and be added on the supplementary in described actual image, realizes real enhancing;
Described emotion on-vehicle machines people carries out data interaction by wired or wireless communication mode and automobile CAN-bus and Cloud Server.
3. as claimed in claim 1, a kind of emotion on-vehicle machines people carries out the method for danger early warning by augmented reality, it is characterized in that, it is further comprising the steps of that described emotion on-vehicle machines people processes image information:
(1) real-time video flowing is processed to dangerous frame of video whether in identification video flowing;
(2) if there is no dangerous frame of video, is considered as current vehicle front without danger; If dangerous frame of video, is considered as current vehicle front dangerous, and mark is carried out in the hazardous location in described dangerous frame of video, turn (3);
(3), according to the locus of the hazardous location of described mark, judge that the danger that hazardous location exists is general dangerous probability SD
lprobability SD with highly dangerous
h;
(4) by image processing algorithm, judge the distance between the hazardous location of mark in video acquisition module and dangerous frame of video, be called risk distance;
(5) according to described risk distance, judge that current danger is as general dangerous probability DD
lprobability DD with highly dangerous
h;
(6) according to formula 3-1, calculate the danger classes of current existence:
D
rank=αSD
L+βSD
H+γDD
L+θDD
H 3-1
In formula 3-1, D
rankfor danger classes, α, β, γ, the θ parameter for demarcating, SD
lfor hazardous location is general dangerous probability, SD
hfor hazardous location be highly dangerous probability, DD
lfor risk distance is general dangerous probability, DD
hfor the risk distance probability that is highly dangerous;
(7) danger classes of calculating in (6) and the threshold value λ of setting are compared, if D
rank< λ, current danger is potential danger, if D
rank>=λ, current danger is highly dangerous.
4. as claimed in claim 1, a kind of emotion on-vehicle machines people carries out the method for danger early warning by augmented reality, it is characterized in that, described emotion on-vehicle machines people is further comprising the steps of by intense strain alerting drivers potential danger:
(1) emotion on-vehicle machines people knows that the place ahead exists after potential danger, further hazard types is carried out to identification, described hazard types includes but not limited to that leading vehicle distance is too near with speeding, the place ahead has pedestrian, the place ahead to have that there is barrier in bicycle, the place ahead, the place ahead visible rays is weak, front light is strong;
(2) emotion on-vehicle machines people picks out after concrete hazard types, in the mode of word and/or picture, generates or call the warning against danger sign matching with described hazard types;
(3) emotion on-vehicle machines people shows described warning against danger sign on mobile device screen, triggers intense strain event simultaneously;
(4) after described intense strain triggers, emotion on-vehicle machines people's expression changes tense situation into, sends nervous sound simultaneously and follows nervous action; The action of described anxiety refers to that described robot body rotates on surface level and/or mobile device is faced upward and/or backward wing before doing on vertical plane.
5. as claimed in claim 1, a kind of emotion on-vehicle machines people carries out the method for danger early warning by augmented reality, it is characterized in that, described enhancing demonstration potential danger situation or highly dangerous situation are further comprising the steps of:
(1) emotion on-vehicle machines people switches to augmented reality pattern by normal mode, the image information that on mobile device screen, display video acquisition module obtains in real time;
(2) by regular or irregular and with coloured lines, mark is carried out in the hazardous location occurring in frame of video;
(3) by image processing algorithm, calculate the distance in video acquisition module and actual danger region, and adopt the distance with coloured line or face mark video acquisition module and described hazardous location in the image frame obtaining;
(4) distance in the car speed obtaining according to mobile device sensor and described video acquisition module and actual danger region, calculates vehicle and arrives at the required time of described hazardous location, and described temporal information is strengthened to demonstration in the image frame obtaining.
6. a method for dangerous frame whether in emotion on-vehicle machines people identification video flowing, is characterized in that, comprises the following steps:
(1) dissimilar danger situation picture is processed, is extracted feature, and by every class danger situation characteristic storage in background server or Cloud Server;
(2) emotion on-vehicle machines people obtains the frame in video flowing with the sampling interval of setting, and the described frame picture obtaining is analyzed to extraction feature, then with described server in every class danger situation feature of storing compare;
(3) if the match is successful, return to warning against danger, and the hazard types of judgement existence, turn (4);
(4), according to hazard types and the described risk distance of described judgement, whether emotion on-vehicle machines people determines switching working mode;
(5) if mate unsuccessfully, emotion on-vehicle machines people is considered as the place ahead without danger, in next sampling interval, continues to obtain the frame in video flowing, turns (2).
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310692727.6A CN103697900A (en) | 2013-12-10 | 2013-12-10 | Method for early warning on danger through augmented reality by vehicle-mounted emotional robot |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310692727.6A CN103697900A (en) | 2013-12-10 | 2013-12-10 | Method for early warning on danger through augmented reality by vehicle-mounted emotional robot |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN103697900A true CN103697900A (en) | 2014-04-02 |
Family
ID=50359512
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310692727.6A Pending CN103697900A (en) | 2013-12-10 | 2013-12-10 | Method for early warning on danger through augmented reality by vehicle-mounted emotional robot |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103697900A (en) |
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104002818A (en) * | 2014-05-27 | 2014-08-27 | 昆山市智汽电子科技有限公司 | Interaction and automation integrated vehicle-mounted system |
| CN104317298A (en) * | 2014-11-04 | 2015-01-28 | 郭海锋 | Emotional interaction type mobile phone robot |
| CN106127829A (en) * | 2016-06-28 | 2016-11-16 | 广东欧珀移动通信有限公司 | The processing method of a kind of augmented reality, device and terminal |
| CN107223235A (en) * | 2016-12-14 | 2017-09-29 | 深圳前海达闼云端智能科技有限公司 | Auxiliary display method, device and display system |
| CN107807947A (en) * | 2016-09-09 | 2018-03-16 | 索尼公司 | The system and method for providing recommendation on an electronic device based on emotional state detection |
| CN110114740A (en) * | 2016-12-27 | 2019-08-09 | 本田技研工业株式会社 | Information provider unit and information providing method |
| US10497161B1 (en) | 2018-06-08 | 2019-12-03 | Curious Company, LLC | Information display by overlay on an object |
| CN110871813A (en) * | 2018-08-31 | 2020-03-10 | 比亚迪股份有限公司 | Control method and device of virtual robot, vehicle, equipment and storage medium |
| US10636216B2 (en) | 2018-09-06 | 2020-04-28 | Curious Company, LLC | Virtual manipulation of hidden objects |
| US10650600B2 (en) | 2018-07-10 | 2020-05-12 | Curious Company, LLC | Virtual path display |
| CN111598974A (en) * | 2014-06-03 | 2020-08-28 | 苹果公司 | Method and system for presenting digital information related to real objects |
| US10818088B2 (en) | 2018-07-10 | 2020-10-27 | Curious Company, LLC | Virtual barrier objects |
| US10872584B2 (en) | 2019-03-14 | 2020-12-22 | Curious Company, LLC | Providing positional information using beacon devices |
| US10970935B2 (en) | 2018-12-21 | 2021-04-06 | Curious Company, LLC | Body pose message system |
| US10991162B2 (en) | 2018-12-04 | 2021-04-27 | Curious Company, LLC | Integrating a user of a head-mounted display into a process |
| CN113386785A (en) * | 2019-07-03 | 2021-09-14 | 北京百度网讯科技有限公司 | Method and apparatus for displaying augmented reality alert information |
| CN115427910A (en) * | 2020-12-23 | 2022-12-02 | 松下知识产权经营株式会社 | Robot control method, robot, and program |
Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2003196793A (en) * | 2001-12-25 | 2003-07-11 | Mitsubishi Electric Corp | Mobile information communication system |
| US20050057689A1 (en) * | 2003-09-16 | 2005-03-17 | Honda Motor Co., Ltd. | Image distribution system |
| CN1922056A (en) * | 2004-02-26 | 2007-02-28 | 日本精机株式会社 | Information providing device for vehicle |
| CN101016053A (en) * | 2007-01-25 | 2007-08-15 | 吉林大学 | Warning method and system for preventing collision for vehicle on high standard highway |
| CN101318491A (en) * | 2008-05-14 | 2008-12-10 | 合肥工业大学 | Built-in integrated visual sensation auxiliary driving safety system |
| CN101391589A (en) * | 2008-10-30 | 2009-03-25 | 上海大学 | Vehicle intelligent alarming method and device |
| CN102303605A (en) * | 2011-06-30 | 2012-01-04 | 中国汽车技术研究中心 | Multi-sensor information fusion-based collision and departure pre-warning device and method |
| CN102490673A (en) * | 2011-12-13 | 2012-06-13 | 中科院微电子研究所昆山分所 | Vehicle active safety control system based on internet of vehicles and control method of vehicle active safety control system |
| CN102582621A (en) * | 2012-02-23 | 2012-07-18 | 朱庆平 | Intelligent vehicle-mounted GPS (global positioning system) navigation anti-collision warning system |
| US20120191269A1 (en) * | 2011-01-21 | 2012-07-26 | Mitre Corporation | Teleoperation of Unmanned Ground Vehicle |
| CN102642510A (en) * | 2011-02-17 | 2012-08-22 | 汽车零部件研究及发展中心有限公司 | Image-based vehicle anti-collision early warning method |
| CN102781728A (en) * | 2010-03-09 | 2012-11-14 | 本田技研工业株式会社 | Vehicle-mounted device capable of operating in cooperation with portable device |
| CN102910130A (en) * | 2012-10-24 | 2013-02-06 | 浙江工业大学 | Actually-enhanced driver-assisted early warning system |
| CN103164975A (en) * | 2011-12-12 | 2013-06-19 | 株式会社电装 | Warning system, vehicular apparatus, and server |
| CN103324100A (en) * | 2013-05-02 | 2013-09-25 | 郭海锋 | Emotion vehicle-mounted robot driven by information |
-
2013
- 2013-12-10 CN CN201310692727.6A patent/CN103697900A/en active Pending
Patent Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2003196793A (en) * | 2001-12-25 | 2003-07-11 | Mitsubishi Electric Corp | Mobile information communication system |
| US20050057689A1 (en) * | 2003-09-16 | 2005-03-17 | Honda Motor Co., Ltd. | Image distribution system |
| CN1922056A (en) * | 2004-02-26 | 2007-02-28 | 日本精机株式会社 | Information providing device for vehicle |
| CN101016053A (en) * | 2007-01-25 | 2007-08-15 | 吉林大学 | Warning method and system for preventing collision for vehicle on high standard highway |
| CN101318491A (en) * | 2008-05-14 | 2008-12-10 | 合肥工业大学 | Built-in integrated visual sensation auxiliary driving safety system |
| CN101391589A (en) * | 2008-10-30 | 2009-03-25 | 上海大学 | Vehicle intelligent alarming method and device |
| CN102781728A (en) * | 2010-03-09 | 2012-11-14 | 本田技研工业株式会社 | Vehicle-mounted device capable of operating in cooperation with portable device |
| US20120191269A1 (en) * | 2011-01-21 | 2012-07-26 | Mitre Corporation | Teleoperation of Unmanned Ground Vehicle |
| CN102642510A (en) * | 2011-02-17 | 2012-08-22 | 汽车零部件研究及发展中心有限公司 | Image-based vehicle anti-collision early warning method |
| CN102303605A (en) * | 2011-06-30 | 2012-01-04 | 中国汽车技术研究中心 | Multi-sensor information fusion-based collision and departure pre-warning device and method |
| CN103164975A (en) * | 2011-12-12 | 2013-06-19 | 株式会社电装 | Warning system, vehicular apparatus, and server |
| CN102490673A (en) * | 2011-12-13 | 2012-06-13 | 中科院微电子研究所昆山分所 | Vehicle active safety control system based on internet of vehicles and control method of vehicle active safety control system |
| CN102582621A (en) * | 2012-02-23 | 2012-07-18 | 朱庆平 | Intelligent vehicle-mounted GPS (global positioning system) navigation anti-collision warning system |
| CN102910130A (en) * | 2012-10-24 | 2013-02-06 | 浙江工业大学 | Actually-enhanced driver-assisted early warning system |
| CN103324100A (en) * | 2013-05-02 | 2013-09-25 | 郭海锋 | Emotion vehicle-mounted robot driven by information |
Cited By (32)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104002818A (en) * | 2014-05-27 | 2014-08-27 | 昆山市智汽电子科技有限公司 | Interaction and automation integrated vehicle-mounted system |
| CN111598974A (en) * | 2014-06-03 | 2020-08-28 | 苹果公司 | Method and system for presenting digital information related to real objects |
| CN111598974B (en) * | 2014-06-03 | 2023-12-22 | 苹果公司 | Method and system for presenting digital information related to a real object |
| CN104317298A (en) * | 2014-11-04 | 2015-01-28 | 郭海锋 | Emotional interaction type mobile phone robot |
| CN106127829A (en) * | 2016-06-28 | 2016-11-16 | 广东欧珀移动通信有限公司 | The processing method of a kind of augmented reality, device and terminal |
| CN107807947A (en) * | 2016-09-09 | 2018-03-16 | 索尼公司 | The system and method for providing recommendation on an electronic device based on emotional state detection |
| CN107223235A (en) * | 2016-12-14 | 2017-09-29 | 深圳前海达闼云端智能科技有限公司 | Auxiliary display method, device and display system |
| CN107223235B (en) * | 2016-12-14 | 2022-02-25 | 达闼机器人有限公司 | Auxiliary display method, device and display system |
| CN110114740A (en) * | 2016-12-27 | 2019-08-09 | 本田技研工业株式会社 | Information provider unit and information providing method |
| CN110114740B (en) * | 2016-12-27 | 2022-10-18 | 本田技研工业株式会社 | Information providing device and information providing method |
| US10497161B1 (en) | 2018-06-08 | 2019-12-03 | Curious Company, LLC | Information display by overlay on an object |
| US11282248B2 (en) | 2018-06-08 | 2022-03-22 | Curious Company, LLC | Information display by overlay on an object |
| US10650600B2 (en) | 2018-07-10 | 2020-05-12 | Curious Company, LLC | Virtual path display |
| US10818088B2 (en) | 2018-07-10 | 2020-10-27 | Curious Company, LLC | Virtual barrier objects |
| CN110871813A (en) * | 2018-08-31 | 2020-03-10 | 比亚迪股份有限公司 | Control method and device of virtual robot, vehicle, equipment and storage medium |
| US10636216B2 (en) | 2018-09-06 | 2020-04-28 | Curious Company, LLC | Virtual manipulation of hidden objects |
| US10861239B2 (en) | 2018-09-06 | 2020-12-08 | Curious Company, LLC | Presentation of information associated with hidden objects |
| US10902678B2 (en) | 2018-09-06 | 2021-01-26 | Curious Company, LLC | Display of hidden information |
| US10636197B2 (en) | 2018-09-06 | 2020-04-28 | Curious Company, LLC | Dynamic display of hidden information |
| US11238666B2 (en) | 2018-09-06 | 2022-02-01 | Curious Company, LLC | Display of an occluded object in a hybrid-reality system |
| US10803668B2 (en) | 2018-09-06 | 2020-10-13 | Curious Company, LLC | Controlling presentation of hidden information |
| US11995772B2 (en) | 2018-12-04 | 2024-05-28 | Curious Company Llc | Directional instructions in an hybrid-reality system |
| US10991162B2 (en) | 2018-12-04 | 2021-04-27 | Curious Company, LLC | Integrating a user of a head-mounted display into a process |
| US11055913B2 (en) | 2018-12-04 | 2021-07-06 | Curious Company, LLC | Directional instructions in an hybrid reality system |
| US10970935B2 (en) | 2018-12-21 | 2021-04-06 | Curious Company, LLC | Body pose message system |
| US10872584B2 (en) | 2019-03-14 | 2020-12-22 | Curious Company, LLC | Providing positional information using beacon devices |
| US10901218B2 (en) | 2019-03-14 | 2021-01-26 | Curious Company, LLC | Hybrid reality system including beacons |
| US10955674B2 (en) | 2019-03-14 | 2021-03-23 | Curious Company, LLC | Energy-harvesting beacon device |
| CN113386785B (en) * | 2019-07-03 | 2022-11-22 | 北京百度网讯科技有限公司 | Method and device for displaying augmented reality warning information |
| CN113386785A (en) * | 2019-07-03 | 2021-09-14 | 北京百度网讯科技有限公司 | Method and apparatus for displaying augmented reality alert information |
| CN115427910A (en) * | 2020-12-23 | 2022-12-02 | 松下知识产权经营株式会社 | Robot control method, robot, and program |
| CN115427910B (en) * | 2020-12-23 | 2024-03-15 | 松下知识产权经营株式会社 | Robot control method, robot, and program |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103697900A (en) | Method for early warning on danger through augmented reality by vehicle-mounted emotional robot | |
| CN103786061B (en) | Vehicular robot device and system | |
| US10423844B2 (en) | Personalized augmented reality vehicular assistance for color blindness condition | |
| CN111739344B (en) | Early warning method and device and electronic equipment | |
| CN102141797B (en) | Airport terminal service robot and control method thereof | |
| CN104627078A (en) | Automobile drive simulation system based on flexible transparent OLED and control method thereof | |
| WO2017038166A1 (en) | Information processing device, information processing method, and program | |
| JP2018027776A (en) | Individualized adaptation of driver action prediction models | |
| CN110104036B (en) | Tramcar simulation operation system and tramcar simulation operation method | |
| CN104900081A (en) | Vehicle-mounted head-up display equipment and information processing method and device thereof | |
| CN203681429U (en) | Display control device of vehicle-mounted system and the vehicle-mounted system | |
| CN112634611A (en) | Method, device, equipment and storage medium for identifying road conditions | |
| CN105223844B (en) | The master system that a kind of simulation UI interacts with parking assisting system | |
| US11462113B2 (en) | Vehicle automation-connected pedestrian risk assessment device | |
| US11604908B2 (en) | Hardware in loop testing and generation of latency profiles for use in simulation | |
| CN106143308A (en) | Lane Departure Warning System based on intelligent back vision mirror | |
| CN106251698A (en) | For the method communicated between autonomous vehicle with passenger and control unit | |
| CN112601693A (en) | Solution for monitoring and planning the movement of a vehicle | |
| CN113160691A (en) | Unmanned automatic driving vehicle for education and training | |
| CN109808600A (en) | The method for visualizing and device of the perception information of automatic driving vehicle | |
| CN110871810A (en) | Vehicle, vehicle equipment and driving information prompting method based on driving mode | |
| CN103552517A (en) | Driver gesture control language | |
| CN206524032U (en) | From dynamic auxiliary driving actual training device | |
| CN216388538U (en) | Unmanned automatic driving vehicle for education and training | |
| CN108133609A (en) | A kind of vehicle-state display system based on ADAS systems |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20140402 |