JP4159049B2 - Human support system, human support method, and human support program - Google Patents

Description

  The present invention relates to a human support system, a human support method, and a human being, which are alienated from information technology living in an urban space where an information infrastructure is established, and support the elderly and disabled persons who have not sufficiently obtained the benefits. Regarding support programs.

  In recent years, with the rapid development of information communication technology, information infrastructure has been developed. However, while the development of a powerful and highly scalable information infrastructure that can handle enormous amounts of information has progressed and the number of people who benefit from it increases in daily life, it is represented by elderly people and people with functional disabilities in hospitals and homes. However, the current situation is that the disparity with people called information weak who have been marginalized from information technology and have not been able to fully benefit from it has been relatively advanced.

  In addition, it is guaranteed that the retention of portable information terminals will function effectively in the event of a disaster than the example of people who were rescued early by the retention of mobile phones at the time of the new terrorist attacks in the United States. On the other hand, the 1995 Hanshin-Awaji Earthquake reveals that there is a weak point that if a communication station such as a mobile phone is destroyed, it will not function at all. In addition, due to the complexity of the above-described functional operations, a problem for information weak persons, such as low personal information terminal holding ratios, is raised. In the midst of serious progress toward an aging society with a declining birthrate, a warning will sound that such vulnerable people will increase more and more. In such a social background, it is urgent to build a human-centered urban space that has a natural interface that is free of information barriers and that supports people spatially.

  On the other hand, the advanced safety car promotion study group of the Ministry of Transport has proposed a preventive safety technique for enhancing the safety of a car by using new technologies such as electronics technology (Non-Patent Document 1). This preventive safety technology detects the driver's dangerous condition during normal driving, the dangerous situation of the driving environment, etc. with various sensors and alerts the driver, reducing the driver's burden, etc. It is designed for illustration purposes. However, even if the danger can be notified immediately before the dangerous situation, the situation before the dangerous situation cannot be notified. Further, it is known that when a danger is suddenly displayed immediately before a dangerous situation, the display immediately before that enters the field of view, so that the peripheral visual field becomes extremely difficult to see.

In this manner, for example, the presence of pedestrians and vehicles can be detected and presented in an urban space, but the information becomes excessive, which causes confusion and hinders the driver.
Secretariat for the Promotion of Advanced Safety Vehicles Promotion Ministry of Transportation

  It is urgent to build a human-centered urban space with an information barrier-free natural interface that supports people spatially. On the other hand, when driving a car, to make the car highly intelligent and greatly improve safety. Preventive safety technology has been proposed. However, even though the danger can be notified immediately before the dangerous situation, the situation before the dangerous situation cannot be notified, so there is a danger that the driver may fall into the dangerous situation.

  In this manner, for example, the presence of pedestrians and vehicles can be detected and presented in an urban space, but the information becomes excessive, which causes confusion and hinders the driver.

  The present invention has been made to solve the above-described problems, and an object of the present invention is to provide a human support system, a human support method, and a human support program that support driving of pedestrians and drivers. It is in.

  According to an aspect of the present invention, a camera installed in the course of a vehicle, a vehicle position information calculation unit that calculates the position information of the vehicle by performing image processing on the route image information captured by the camera, and imaged by the camera In order to inform the pedestrian of the presence of the vehicle based on the pedestrian position information calculation unit that calculates the pedestrian position information by performing image processing on the obtained route image information, the vehicle position information, and the pedestrian position information. A human support management unit that generates walking support information or generates driving support information for notifying the vehicle of the presence of a pedestrian, and communication control means that transmits the walking support information or the driving support information. A human support system characterized by the above is provided.

  According to another aspect of the present invention, the course image information captured by a camera installed in the course of the vehicle is subjected to image processing to calculate the position information of the vehicle, and the course image information captured by the camera is imaged. Processing and calculating position information of the pedestrian, and generating walking support information for notifying the pedestrian of the presence of the vehicle based on the vehicle position information and the pedestrian position information; There is provided a human support method characterized by generating driving support information for notifying existence and transmitting the walking support information or the driving support information.

  According to still another aspect of the present invention, a computer uses a vehicle position information calculation unit that calculates the position information of the vehicle by performing image processing on the route image information captured by a camera installed in the route of the vehicle, Based on the pedestrian position information calculation unit that calculates the pedestrian position information by image processing the route image information captured by the camera, the vehicle position information, and the pedestrian position information, the presence of the vehicle to the pedestrian As a communication control means for generating walking support information for notifying, or generating driving support information for notifying the vehicle of the presence of a pedestrian, and transmitting the walking support information or the driving support information A human support program characterized by functioning is provided.

  According to the present invention, it is possible to appropriately support driving of a pedestrian or a pedestrian.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a diagram showing an example of a network system to which a human support system according to an embodiment of the present invention is applied.

As shown in FIG. 1, a pedestrian 1 and a vehicle 2 are traveling on roads 41 and 42 in directions indicated by arrows, respectively, and show a state of approaching an intersection 43. Two intersection installation cameras 31 and 32 are installed at the intersection 43, and the acquired image is transmitted to the server 3 by wireless communication. A display unit 22, a control unit 23, and a vehicle-mounted camera 24 are mounted on the vehicle 2. Moreover, the visual field imaging camera 21 is being fixed to the driver | operator's head of the vehicle 2, and a driver | operator's visual field image is acquired. The field-of-view image and the in-vehicle camera image captured by the vehicle-mounted camera 24 are transmitted to the server 3. In addition, the control unit 23 calculates vehicle speed information V _v from the motor, the handle, or the like of the vehicle 2 and transmits it to the server 3. Of course, when the vehicle 2 itself is calculated vehicle speed, the vehicle speed information V _v may simply be transmitted to the server 3. The server 3 generates driving support information for notifying the presence of a pedestrian and the presence of a vehicle based on various data received from the cameras 31 and 32 and the vehicle 2, transmits the driving support information to the vehicle 2, and causes the display unit 22 to display the driving support information. . The walking support information indicating the presence of the vehicle 2 is transmitted to the portable information terminal 11 carried by the pedestrian 1 and displayed on the display unit 12 of the portable information terminal 11.

  FIG. 2 is a diagram showing an example of a detailed configuration of the vehicle 2, the server 3, and the intersection installation cameras 31 and 32 constituting the human support system.

  As shown in FIG. 2, the intersection installation camera 31 is installed at point A, and includes two cameras 31a and 31b. The camera 31a is a camera that images the road 42 side from the point A, and the camera 31b is a camera that images the road 41 side from the point A.

  The intersection installation camera 32 is installed at a point B, and includes two cameras 32a and 32b. The camera 32a is a camera that images the road 42 side from the point B, and the camera 32b is a camera that images the road 41 side from the point B.

  The server 3 includes a CPU, a storage device 39 accessible by the CPU, and communication control means 38. When the CPU executes a program read from the storage device 39, the CPU executes a vehicle position information calculation unit 33, a pedestrian position information calculation unit 34, a face direction information calculation unit 35, an intention recognition unit 36, and a human support. It functions as the management unit 37. Further, the calculation result in the CPU is transmitted to the external terminal or the like via the communication control means 38, and the data from the external terminal or the like is received by the communication control means 38.

  Although not shown in FIG. 2, both the portable information terminal 11 and the vehicle 2 are provided with a communication control unit, and can transmit and receive data to and from the server 3 by wireless communication.

  Each image data captured by the cameras 31a, 31b, 32a, and 32b is transmitted to the server 3 by wireless communication. The communication control means 38 of the server 3 receives the image data from the cameras 31a, 31b, 32a and 32b and outputs it to the vehicle position information calculation unit 33 or the pedestrian position information calculation unit 34.

Vehicle position information calculation unit 33, a point A motor vehicle image data obtained from the camera 31a, calculates the vehicle position information V _L on the basis of the point B car image data obtained from the camera 31c, intended recognition unit 36 and human The data is output to the support management unit 37.

Pedestrian position information calculator 34 calculates a point A pedestrian image data obtained from the camera 31b, the pedestrian position information P _L based on the point B pedestrian image data obtained from camera 31d, Human Support The data is output to the management unit 37.

In-vehicle camera image data obtained by the vehicle-mounted camera 24 provided in the vehicle 2 and visual field image data obtained by the visual field imaging camera 21 fixed to the driver's head are transmitted to the server 3 by wireless communication. The The communication control unit 38 of the server 3 receives the image data from the cameras 21 and 24 and outputs it to the face direction information calculation unit 35. Face direction information calculator 35 outputs the intended recognition unit 36 vehicle camera image data (driver captured image data) and calculates the face direction information theta _D indicating the direction of the face of the driver based on the field image data . Incidentally, the face direction information calculator 35 may calculate the face direction information theta _D by image processing based solely on the driver captured image data, and calculate the face direction information theta _D based only on the field image data Good.

On the other hand, the speed information V _v of the vehicle 2 from the control unit 23 is transmitted to the server 3 by wireless communication. Communication control means of the server 3 38 outputs the speed information V _v from the control unit 23 to the intention recognizing section 36.

Intention recognizing section 36, the speed information V _v obtained from the control unit 23, the face direction information theta _D obtained from the face direction information calculator 35, the vehicle position information V obtained from the vehicle position information calculator 33 Based on _L , the driver's intention recognition result is calculated and output to the human support manager 37.

Human Support management unit 37, a vehicle position information V _L from the vehicle position information calculator 33, and the pedestrian position information P _L from the pedestrian position information calculator 34, to the intended recognition result from the intention recognizing section 36 Based on this, the walking support information for supporting the walking of the pedestrian is generated, and the driving support information for supporting the driving of the vehicle is generated.

  The walking support information is transmitted to the portable information terminal 11 carried by the pedestrian 1 via the communication control means 38 and displayed on the display unit 12 of the portable information terminal 11. Further, the driving support information is transmitted to the display unit 22 of the vehicle 2 via the communication control means 38 and displayed. Note that these walking support information and driving support information are shown as an example in which the walking support information and the driving support information are transmitted without going through the communication control means 38 in FIG. 2, but actually, they are sent through the communication control means 38.

  The server 3 includes a storage device 39. The storage device 39 stores data necessary for various data processing as described above, and calculation results such as membership functions and association matrices.

  Next, the pedestrian position information calculation process in the pedestrian position information calculation unit 34 will be described with reference to FIG.

As shown in FIG. 3, first, the object position extracting means 341 that has acquired the pedestrian image data from the points A and B2 extracts the position of the object to be noted in the pedestrian image data. The object to be noted is set in a color portion different from the objects other than the pedestrian, such as a skin color portion of the pedestrian. Of all the positions I ₁ ~I _n specified by the image data, the position of the pedestrian skin color portion of the extracted object and _{P 1 ~P m (m ≦ n} ).

The center-of-gravity position calculation unit 342 calculates the center-of-gravity position P _c (center-of-gravity position information) based on the object positions P _{1 to} P _m obtained by the object position extraction unit 341. Thereby, the position of the pedestrian at a certain time t is uniquely specified.

Feature point extraction unit 343 calculates a feature point C on the basis of the gravity center position P _C obtained in the center of gravity position calculating means 342. The feature point indicates the position of the center of gravity of a typical pedestrian for every fixed time, that is, for a fixed time. When the center of gravity of the obtained time t at the center of gravity position calculating means 342 and P _Ct, center-of-gravity position P _Ct is the time series data that varies over time. FIG. 4 is a diagram showing the barycentric position _PCt as the time series data. FIG. 4A shows the centroid position P _{Cxt in} the x direction, and FIG. 4B shows the centroid position P _Cyt in the y direction. These barycentric positions P _Cxt and P _Cyt are both time series data that takes a maximum value or a minimum value during 200 × _{1/60 s} . For example, the sampling time is set to 200 × _{1/60 s,} and out, for example, a maximum value and wherein point P _L. Of course, the minimum value and other values may be characteristic point P _L a of the extremes. In the case of the example of FIG. 4, the position P _{Lx in} the x direction of the feature point P _L is 500 (× 0.6 × 10 ⁻³ m), and the position P _{Ly in the} y direction is 700 (× 0.6 × 10 ⁻³ m). ) Of course, the calculation of the feature point P _L is not limited to the extraction technique of this extreme value. The obtained feature point P _L is output as pedestrian position information P _L. The feature point P _L is because it is intended to be calculated for each elapsed sampling time, is outputted to the human support management section 37 every sampling time.

It is also possible to output the center-of-gravity position P _ct is time-series data as a pedestrian position information P _L without calculating the feature point P _L.

Similar to the pedestrian position information calculation of the pedestrian position information calculation unit 34 described above, the vehicle position information calculation unit 33 extracts the object positions V _{1 to} V _k (k ≦ n), calculates the gravity center position V _ct , and features. It performs calculation of the point C, and outputs a vehicle intended recognition unit 36 and human support management section 37 the resulting feature point V _L as the vehicle position information V _L. The object positions V _{1 to} V _k are extracted by the same method as the object positions P _{1 to} P _k extracted by the object position extracting unit 341. Further, calculation of the position of the center of gravity _{V ct} is by the same method as calculating the center of gravity _{P ct} in the center-of-gravity position calculating means 342. The calculation of the feature point V _L is performed by the same method as the calculation of the feature point P _L in the feature point extraction unit 343.

Next, the intention recognition process in the intention recognition unit 36 will be described. Note that the processing in the intention recognizing unit 36 described below executes intention recognition processing based on vehicle position information V _L , speed information V _v , and face direction information θ _D inputted by reading out prestored data, and generates an intention recognition result. To do.

When attempting to take a certain driving action, it is generally confirmed that the driver operates in a certain order. Detecting the operation of the face of the driving operation, it recognizes its face direction information theta _D, the driver's intention from the vehicle speed information V _v and the vehicle position information V _L. Driving intention is classified into, for example, right turn, left turn, and straight ahead.

In the present embodiment, a knowledge representation of a concept fuzzy set (Conceptual Fuzzy Sets: hereinafter referred to as CFS) is used as a technique for constructing knowledge about a human intention that is an ambiguous concept. CFS has the following two properties.
a) The real-world numerical data and human-thinking symbols can be fused.
b) It is possible to fuse bottom-up processing from a physical quantity to a higher-order concept and top-down processing from a higher-order concept to a physical quantity.

Hereinafter, an intention recognition model for recognizing three basic driving action intentions of straight driving, right turn, and left turn of driving intention from face direction information θ _D , vehicle speed information V _v, and vehicle position information V _L will be described.

  FIG. 5 is a conceptual diagram of the intention recognition model. For example, the intention recognition model is represented by a lower layer, a middle layer 53, and an upper layer 54 indicated by a membership function 51 and a fuzzy label 52. The lower layer fuzzy label 52 (fuzzy set) includes an angle label 521, a position label 522, and a velocity label 523. The angle label 521 includes three types of labels: negative 521a “N”, zero 521b “Z”, and positive 521c “P”. The position label 522 is composed of three types of labels: negative 522a “N”, zero 522b “Z”, and positive 522c “P”. The speed label 523 includes three types of labels: negative 523a “N”, zero 523b “Z”, and positive 523c “P”.

  These nine types of fuzzy labels are represented by membership functions 51, respectively. Reference numerals 511a to 511c denote functions corresponding to three types of labels of the negative label 521a “N”, zero 521b “Z”, and positive 521c “P” of the angle label 521, respectively. Reference numerals 512a to 512c respectively denote functions corresponding to three types of labels of the position label 522: negative 522a “N”, zero 522b “Z”, and positive 522c “P”. Reference numerals 513a to 513c denote functions corresponding to three types of labels of negative 523a “N”, zero 523b “Z”, and positive 523c “P” of the speed label 523, respectively.

  The middle layer 53 represents a case node which is a case used in the learning process. For example, since three basic driving behaviors are learned for N cases, a total of 3N patterns are learned. It is desirable that the N cases are not limited to the case of one driver, but, for example, the case of separate N drivers is adopted. Thereby, for example, it can be registered as data reflecting the habit of driving for N people, that is, individuality. Since this case node is data reflecting individuality in this way, it should also be referred to as individuality data. Driving habits vary from person to person. Therefore, by using a case node reflecting this individuality in driving support and referring to the data of the case node with the most similar wrinkles, it is possible to provide driving support that properly identifies the person's wrinkles. Furthermore, it is desirable to register a single driver's behavior separately for a plurality of cases. Even the driving behavior of the same person varies depending on the person's mood, for example, when he / she is irritated or relaxed. Therefore, a plurality of driving actions of the same person are registered according to the state of the driver when driving. This makes it possible to provide more detailed driving assistance including the person's mood.

  In this way, instead of averaging the cases and comparing them with the average value, the individuality data reflecting each individuality is left as it is, and the individuality data and the driving behavior are compared, thereby achieving high accuracy. Driving assistance is possible.

  As for the case nodes, three types of 531 to 533 are given to the basic driving behaviors of left turn, straight ahead, and right turn, and each of these case nodes 531 to 533 is assigned to the nodes A, 521a to 521c associated with the labels 521a to 521c. B, C, and the nodes A, B, C and the nodes 541, 542, and 543 of the upper layer 54 are represented by a coupling relationship.

  As for the position, three types of 534 to 536 are given to the basic driving behaviors of left turn, straight ahead, and right turn, and each of these case nodes 534 to 536 are nodes A, B, and C associated with labels 522a to 522c. These nodes A, B, and C are expressed by a connection relationship between the nodes 541, 542, and 543 of the upper layer 54.

  As for speed, three types of 537 to 539 are given to the basic driving behaviors of left turn, straight ahead, and right turn. Each of these case nodes 537 to 539 is associated with nodes A, B, and C associated with labels 523a to 523c. These nodes A, B, and C are expressed by a connection relationship between the nodes 541, 542, and 543 of the upper layer 54.

  The reason why the case nodes of the middle layer 53 are arranged by dividing into angles, positions, and speeds like 531 to 533, 534 to 536, and 537 to 539 is that the intention of driving behavior is not expressed as a fixed pattern, This is because it is composed of a combination of various patterns of angle, position and speed.

  The upper layer 54 is a layer indicating driving intention, and is composed of three types of nodes: a left turn 541, a straight advance 542, and a right turn 543.

  When angle, position, and velocity feature values are input, the node having the closest feature in the middle layer 53 is most strongly activated. When the degree of activation is obtained as an activity value distribution, it can be seen which case is close for each pattern.

  However, the driving behavior cannot be specified only by the activity value distribution appearing in the middle layer 53. This problem is solved by introducing a context. The upper layer 54 is a layer representing driving behavior, and is coupled to all the case nodes 531 to 539 of the middle layer 53.

  The learning data is given to the lower layer indicated by the fuzzy label 52 and the membership function 51. As a result, the middle layer 53 obtains the active value distribution of the feature values and the case nodes 531 to 539. Further, on the basis of the activity value distribution of the middle layer 53, the upper layer 54 is given a driving intention of “right turn”, “straight forward”, and “left turn” as a teacher. Based on these, a connection relationship between the middle layer 53 and the upper layer 54 is obtained by Heb learning. In this intention recognition model, driving behavior is stored as a fuzzy set from an example, and when an input is given, echo action is repeated between the lower layer and the middle layer 53, and between the middle layer 53 and the upper layer 54. The recognition result is obtained by the convergence of the activity value distribution to a state consistent with the context. Here, the context is a standard pattern obtained from each driving action example, and a combination of feature values of angles, positions, and speeds similar to the pattern is promoted, while feature values that do not match the pattern are The combination is suppressed.

Next, the concept of CFS (concept fuzzy set) will be described with reference to FIG.
The concept fuzzy 61 not only has fuzzy knowledge expressed in a distributed manner on an associative memory, but also has a mechanism for performing fuzzy knowledge processing using an associative memory recall process. Further, since the concept fuzzy 61 is realized by an associative memory, it is possible to obtain merits such as handling of situation dependency and complicated processing by multi-layer coupling.

  The fuzzy labels 62 and 63 of the fuzzy set express the name of the concept, and the form of the fuzzy set expresses the meaning of the concept. According to the meaning usage theory, the meaning of a word is expressed by other words, but its meaning changes depending on the time of use. Therefore, if the attention range at the time of use is expressed by an active value associated with a word, the meaning of an ambiguous word can be expressed by a fuzzy set formed by another word. This distributed knowledge representation is called a concept fuzzy 61.

  Since the distribution of the activity values changes depending on various situations, it is possible to express a change in meaning depending on the situation. The concept fuzzy 61 is not limited to a logical expression, and can represent a knowledge. Therefore, what is logically impossible to express explicitly can be expressed explicitly.

  The activation value of the concept fuzzy 61, that is, the membership value is controlled by the associative memory. Nodes represent concepts, and links correspond to the strength of relationships between concepts. In FIG. 6, nodes are represented by fuzzy labels 62 and 63, and links are represented by arrows. For example, when the fuzzy label 62 in FIG. 6 is activated as a concept node to be expressed semantically, an activation value is propagated through a network constructed in advance and an echo operation occurs. This reverberation stops when the system energy is at a minimum, and as a result, the conceptual node on the side that expresses the meaning is recalled in a form that is activated with a certain grade. The distribution of activity values appearing here is a fuzzy set.

The concept fuzzy 61 is functionally constructed using a neural network learning rule. That is, according to the concept expression example, the link between nodes is corrected using the Hebb learning rule of the following formula (1).

Here, m _ij denotes the matrix element of the associative matrix M of i rows and j row below, a _i is the respective nodes of the layer L _A, b _j corresponds to each node of layer L _B.

  In the concept fuzzy set, other concepts that are not necessary to explain the concept are not activated. Therefore, structural learning including node selection is performed even from redundant explanation elements. Moreover, in the concept fuzzy set, the general concept is formed by sequentially combining the individual concepts formed separately, as in the case where a person usually forms a concept. That is, a plurality of conceptual fuzzy sets can be synthesized according to the following equation (2).

M = Norm (M ₁ + M ₂ +... + M _n ) (2)
M _{1 to} M _{n denote} individual concept fuzzy, and M denotes a synthesized concept fuzzy set. Norm indicates normalization. In a general neural network, an explicit knowledge expression is not performed on the learning result, but an explicit expression is obtained by the method of the present embodiment.

  Next, fuzzy associative reasoning that is the basis of the fuzzy associative memory system used in the intention recognition unit 36 will be described.

  Fuzzy associative reasoning is a network that uses multiple bi-directional associative memories (BAMs), which is a type of associative memory neural network, and infers by echoing that repeats the propagation of active values. is there. Here, the bidirectional associative memory, which is the basis of associative reasoning, will be described first, and then the fuzzy associative memory system in which this is systemized will be described.

FIG. 7 is a diagram for explaining the concept of bidirectional associative memory (BAM). As shown in FIG. 7, the layer _{L A} and layer _{L B} is has a network of the layer _L plurality of nodes to _{A a 1, a 2, ...} , there is _{a n,} a plurality in layers _{L B} Nodes b ₁ , b ₂ ,..., B _p exist. The connection weight between nodes is expressed by an association matrix MεR ^{n × p} . Pattern A∈R ⁿ for this association matrix M Layer _{L A} and layer _{L B,} when to be noted the pair B∈R ^p, given A'∈R ⁿ containing noise to the layer _{L A} as also, by the echo operation between layers L _a and layer L _B, the pattern B is recalled. Conversely, the pattern A will occur can give a pattern B'∈R ^p containing noise to the layer L _B. Association in this bidirectional associative memory is performed by the following equations (3.1) and (3.2).

Here, a _i (t) and b _j (t) are active value vectors of the respective nodes in step t and take values belonging to the section [0, 1]. It may be a value of {0, 1}. φ represents each node A (t) = (a ₁ (t), a ₂ (t),..., a _n (t)), B (t) = (b ₁ (t), b ₂ (t),. , B _p (t)), for example, a sigmoid function is used. The association matrix M is obtained by the following equations (4.1) and (4.2) when the pattern pairs to be recorded are (A ₁ , B ₁ ),..., (A _m , B _m ). .

Here, β _i is an associative coefficient. In the Kosko method, when an associative matrix M is obtained, a pattern pair obtained by bipolar conversion of a pattern pair of {0, 1} is used. However, if the association matrix M obtained by this Kosko technique is used, the network may cause a saturation state, and adjustment is necessary. In order to solve this problem, a method for normalizing the associative matrix M has been proposed. Therefore, in the fuzzy associative memory system used in this embodiment, a normalized associative matrix obtained by this normalization method is used.

There are restrictions on the pattern pairs that can be memorized by bidirectional associative memory (BAM). One is that the orthogonality between pattern pairs is strong and the Hamming distance between patterns and pattern complements must be sufficiently separated within each layer. Another is that the continuity assumption needs to hold. The continuity assumption is expressed by the following equation (5).

  Here, H is a Hamming distance. That is, it can be said that it is difficult to record unless the pattern to be recorded in each layer is as far as the Hamming distance.

  Next, a method for constructing a fuzzy associative memory system will be described.

FIG. 8 is a schematic diagram expressing fuzzy rules using an associative memory network. In setting the layers, the layer representing the antecedent part of the fuzzy rule is the If (X) layer, and the layer representing the consequent part is the Then (Y) layer. A node in the If (X) layer represents a fuzzy set (membership function) of the antecedent part, and a node in the Then (Y) layer represents a fuzzy set (membership function) of the consequent part or an input / output function. . Further, since the correlation between the antecedent part and the consequent part exceeds the limit that BAM can store, one node sets a Rule (R) layer representing one rule. Therefore, the fuzzy rule can be expressed by configuring a BAM between the If (X) layer and the Rule (R) layer (associative matrix M _xr ). Furthermore, in the Rule (R) layer, in order to prevent the explosion of ambiguity, a coordinator combination (M _rr ) having a function of suppressing activation of other nodes is set. Coordinator coupling means coupling to a node with a positive weight at an arbitrary node in the same layer and coupling to another node with a negative weight.

Next, the concept of construction on the memory will be described by taking the network configuration of FIG. 8 as an example. FIG. 9 shows knowledge necessary for constructing an associative memory network. First, in order to obtain an association matrix between each layer, the correlation of fuzzy rules is replaced with a value belonging to the interval [0, 1]. In some cases, a binary value of 0 or 1 indicated by {0, 1} may be substituted. The result is as shown in the following formula (6).

  In this equation (6), as an example, the correlation is represented by binary values of 0 and 1 represented by {0, 1}, but it is needless to say that the present invention is not limited to this.

  Using this calculation result as a pattern pair to be recorded, an associative matrix between each layer is obtained by Expression (4) -1 and Expression (4) -2. However, the Kosko method obtained using a pattern pair obtained by bipolar conversion is not used.

As described above, the fuzzy associative memory system used in the present embodiment uses the normalized associative matrix at the time of inference. Thus, where an associative matrix obtained saves obtains the normalized associative matrix M _e by the following equation (7).

M _e = a (M + B) (7)
Here, each parameter is as follows.

Here, M, M _e , BεR ^{m × n} , and m _ij are M (i, j) elements.

  Here, when a plurality of rule sets are defined, the above processing is repeated by the number of rule sets to propose a plurality of networks. Then, information relating to the connection between the rule sets is read from the text file, and the respective networks are connected. At the time of inference, the weight (association matrix) of the connection between defined rule sets is also normalized. From here, the network constructed by the fuzzy rule R is referred to as a network R.

Next, the operation process of the fuzzy associative memory system will be described.
Inference is performed by repeating the propagation of active values while maintaining synchronization throughout the network. First, a value obtained by evaluating the input with the membership function is input to the If (X) layer. The association matrix _{M xr,} M _rr, the _{M ry} formula (3.1), by applying the formula (3.2), is repeated as many active value propagation speed associative steps set. For example, the output out _j of each node OUT layers connected by associative matrix _{M r} multiple INr layers is calculated by equation (11).

The proposed initial associative reasoning does not integrate reasoning results and can cause excessive ambiguity. In order to solve this, the active values of the Then (Y) layer are normalized using equations (12.1) to (12.3).

Here, n is the number of nodes in the Then (Y) layer, γf _i is the activation value of node i, and γ _i is the contribution.

Thereby, the contribution degree γ _i is obtained. An inference result is obtained by interpolating the output value of the membership function of the consequent part or the input / output function with this contribution, and MAX functioning (summing) the obtained function (value).

  Next, intention recognition determination processing by the intention recognition unit 36 based on the actually constructed fuzzy associative memory system will be described with reference to the data flow diagrams of FIGS. 12 and 13.

  First, before executing the intention recognition determination process, a membership function used for the intention recognition determination process and N individuality data are registered in the storage device 39 in advance.

More specifically, the speed membership function mu _vS, mu _vB, face direction membership functions mu _.theta.S, mu _.theta.B, stored in the storage device 39 located membership function _{mu LS,} a mu _LB as input membership functions. Also, as the N personality data, associative matrix ^{_{M xr1, M T xr1 ~M xrN}} , M T xrN, normalized matrix _{M rr,} associative matrix ^{_{M ry1, M T ry1 ~M ryN}} , the _{M T RYn} as personality data It is stored in the storage device 39. Further, the driving intention membership functions μ _LS , μ _LB , μ _SS , μ _SB , μ _RS , μ _{RB to} μ _LS , μ _LB , μ _SS , μ _SB , μ _RS , μ _RB are stored as output membership functions. 39.

  N individuality data can be registered, for example, by the following processing.

For driving behavior for N persons, vehicle speed information V _v , face direction information θ _D and vehicle position information V _L are calculated, and the calculated values are set as input nodes. Then, the left turn, straight ahead, and right turn, which are the results of the driving action, are set as output nodes. An intermediate node is set as a buffer for these input nodes and output nodes. Then, by applying Expression (3.1), Expression (3.2), Expression (4.1), and Expression (4.2) to the input node and the intermediate node, the associative matrix M _xr1, M ^T _xr1 ~M _xrN, to calculate the _{^M T} ^xrN. On the other hand, by applying the expressions (3.1), (3.2), (4.1), and (4.2) to the intermediate node and the output node, the associative matrix M _{ry1, M} ^T _ry1 ~M ryN, to calculate the _{^M T} ^ryN.
As described above, N individuality data are obtained as an association matrix.

Next, intention recognition determination processing is performed as follows using the registration data obtained as described above.
First, the intention recognition unit 36, based on the input vehicle velocity V _v , face direction θ _p , and position V _L , respectively, velocity membership functions μ _vs , μ _vB , face direction membership functions μ _θs , μ _θB , The function values (velocity grade μ _vs (V _v ), μ _vB (V _v ), face direction grade μ _θs (θ _p ), μ _θB (θ _p ), position grade μ _{LS are obtained from the} position membership functions μ _LS and μ _LB. (V _L ), μ _LB (V _L )) are calculated.

Next, matrix calculation in the positive direction using associative memory is performed.
Specifically, first, μ _vs (V _v ), μ _vB (V _v ), face direction grades μ _θs (θ _p ), μ _θB (θ _p ), position grades μ _LS (V _L ), μ _LB ( V _L) an associative memory input _{IN vs, IN vB, IN ps} , IN pB, IN Ls, and _{IN LB.} Then, to these associative memory input, performs N number of matrix calculation using the matrix _M xr1 _{~M XRN} corresponding to the N personality data, respectively obtained an associative memory intermediate output _R 1 to R _N. The associative memory intermediate outputs R _{1 to} R _N are normalized by the matrix M _rr every time they are calculated. Further, the obtained associative memory intermediate outputs R _{1 to} R _N are further _subjected to N matrix calculations using the matrices M _{ry1 to} M _ryN , respectively, and associative memory outputs OUT _L1 , OUT _L expressed as left turn, straight _advance , and right turn _S1 , OUT _{R1 to} OUT _LN , OUT _SN , and OUT _RN are obtained. The associative memory outputs OUT _{L1 to} OUT _LN are output values corresponding to the driving intention “left turn”, the associative memory outputs OUT _{s1 to} OUT _sN are output values corresponding to the driving intention “straight ahead”, and the associative memory outputs OUT _{R1 to} OUT _RN. Is an output value corresponding to the driving intention “right turn”.

Next, reverse matrix calculation using an associative memory is performed.
The resulting associative memory output _{OUT L1, OUT S1, OUT R1} ~OUT LN, OUT SN, performs N number of matrix calculation using an associative matrix _{^M T} ry1 ^~M _T ^ryN based on _{OUT RN,} associative intermediate output R obtain ₁ to R _N. Each of the associative memory intermediate outputs R _{1 to} R _N is obtained for a plurality of nodes. Then, N matrix calculations are performed on the obtained associative memory intermediate outputs R _{1 to} R _N using the matrices M ^T _{xr1 to} M ^T _xrN , and associative memory inputs IN _vs , IN _vB , IN _ps , IN _pB , IN _Ls , IN _LB are obtained.

A reverberation operation is performed in which forward and backward calculations using these associative memories are repeated a predetermined number of times, and associative memory outputs OUT _L1 , OUT _S1 , OUT _{R1 to} OUT _LN , OUT _SN , Let OUT _RN converge. Then, the converged associative memory outputs OUT _L1 , OUT _S1 , OUT _{R1 to} OUT _LN , OUT _SN , OUT _RN are normalized to normalize associative memory outputs OUT ^* _{L 1} , OUT ^* _{S 1} , OUT ^* _{R 1} to OUT ^* _LN , OUT ^* _SN and OUT ^* _RN are obtained. Driving intention grades μ _LS1 (OUT ^* _L1 ), μ _LB1 (OUT ^* ) using the driving intention membership functions μ _LS , μ _LB , μ _SS , μ _SB , μ _RS , μ _RB for the obtained normalized associative memory output ^{_{L1), μ SS1 (OUT *}} S1), μ SB1 (OUT * S1), μ RS1 (OUT * R1), μ RB1 (OUT * R1) ~μ LSN (OUT * LN), μ LBN (OUT * LN) , _ΜSSN (OUT ^* _SN ), _μSBN (OUT ^* _SN ), _μRSN (OUT ^* _RN ), and _μRBN (OUT ^* _RN ) are obtained.

  Based on the driving intention grade for the obtained N individuality data, the determination means 36a determines the driving intention and obtains the intention recognition result (left turn, straight advance, right turn). The determination of the determination means 36a is based on a predetermined determination condition (for example, the one having the largest value of the driving intention grade, that is, the one recalled most strongly) is selected from the N driving intention grades. An intention (for example, a right turn) that is previously associated with the determined driving intention grade is output as an intention recognition result.

  The determination conditions of the determination unit 36a are not limited to those described above. For example, when the difference between the largest driving intention grade and the next largest driving intention grade is small, the driving intention grade with the largest driving intention grade and the driving intention corresponding to the next largest driving intention grade are averaged. May be.

  Next, a process of generating walking support information and driving support information by the human support management unit 37 based on the driving intention obtained in the driving intention recognition determination process will be described.

In the above example, the intention recognition result is indicated by the right turn, straight ahead, and left turn. As shown in FIG. 14, the human support manager 37 calculates a progress prediction region based on the intention recognition result and the vehicle position information V _L. This progress prediction region is obtained as a set of positions, for example. Then, it is determined whether based on this advanced prediction region and the pedestrian position information P _L, pedestrian progression prediction area expected to proceed from its intended recognition result is located. This is readily determinable by like whether contains pedestrian position information P _L in the traveling prediction region is a set of example location. When a pedestrian is located in the progress prediction area, the human support manager 37 generates walking support information including danger information for notifying that there is a danger that the vehicle will travel. When the pedestrian is not located in the progress prediction area, the human support management unit 37 generates walking support information that does not include danger information. The generated walking support information is transmitted to the portable information terminal 11 via the communication control means 38 and displayed on the display unit 12.

  In addition, when the pedestrian is located in the progress prediction area, the human support management unit 37 generates driving support information including danger information for notifying that there is a risk that the pedestrian enters the progress area. To do. When a pedestrian is not located in the progress prediction area, the human assistance manager 37 generates driving assistance information that does not include danger information. The generated driving support information is transmitted to the vehicle 2 via the communication control means 38 and displayed on the display unit 22.

Incidentally, the walking assistance information and driving support information, it is desirable to produce a stepwise danger information in accordance with the pedestrian position information P _L progression prediction region. For the production of this stepwise risk information, whether specifically reaches the pedestrian position indicated by the pedestrian position information P _L For example many seconds after the vehicle position is grasped by the vehicle position information V _L velocity V _v of the vehicle may be calculated using the. Then, the time until arrival is compared with a predetermined arrival time threshold value T _w (w = 1, 2, 3...), And the arrival time threshold value T _w or less is equal to or more than the arrival time threshold value T _w−1. in the case, it is determined that the risk _{D w.} That is, the degree of risk is determined based on whether or not T _w−1 ≦ | V _L −P _L | / V _v ≦ T _w is satisfied. If it is established, it is determined that there is a corresponding risk, and if it is not established, it is determined that there is no corresponding risk.

  A specific example of this danger information generation will be described below.

For example, when the arrival time thresholds T ₁ = 2, T ₂ = 4, and T ₃ = 6, when | V _L −P _L | / V _v ≦ 2, the risk level D ₁ , 2 ≦ | V _L −P _L | When / V _v ≦ 4, the risk level D _{2 is determined} , and when 4 ≦ | V _L −P _L | / V _v ≦ 6, the risk level D ₃ is determined. Incidentally, if the resulting velocity _{P v} of the pedestrian, the determination formula _{T w-1 ≦ | V L} -P L | / | V v -P v | may be the ≦ _{T w.} For example, the human support manager 37 may generate the danger information by generating display data that can be visually distinguished according to the degree of danger. For example, risk D ₃ display blue to has not imminent danger when data, the yellow display data to come slightly imminent danger in the case of risk D _2, in the case of risk D ₁ May generate red display data because the danger is imminent. Further, not only visually distinguishable display data but also audio data that stimulates hearing may be combined, or only audio data may be used. In this case, the risk levels D _{1 to} D ₃ are warnings in the foreground by combining the display data and the voice data, and in the case of the risk levels D ₄ , D ₅ ,. You may make it tell only by.

  As described above, the driving support information and the walking support information are generated based on the driving intention obtained by using the fuzzy associative reasoning. Therefore, for example, as shown in Non-Patent Document 1, it is possible to perform danger display in which danger is detected in advance, compared to the case where danger is displayed based only on the immediately preceding information such as the vehicle position and pedestrian position as a result of driving behavior. . That is, in the case of the non-patent document 1, the warning information is warned only when the danger is imminent in order to prevent the support information from being excessive, or various dangers are warned after being prepared to be excessive. There was only. In this case, it is also necessary to give a warning that is completely out of the driving intention. On the other hand, in the case of the present embodiment, the warning is limited to the range predicted from the driving intention, so that there is no excess of information, and the danger can be warned in advance rather than immediately before.

  Hereinafter, operational effects and application examples of the human support system of the present embodiment will be described in detail.

A cognitive scientific view of the danger labeling mechanism.
As a conventional hardware system, there is a system for the purpose of improving accuracy of hardware materials such as a sensor / camera for object detection. Such a system can accurately display the most dangerous events, but in some cognitive science experiments, particularly in the elderly, sudden visual hard movements can enter the field of view, resulting in extreme peripheral vision. The characteristic of becoming difficult to see was noticeable. Therefore, it is necessary to prevent the surrounding visual field from being narrowed by informing the driver of the danger in advance.

Soft-oriented driving support system
In order to solve the problems as described above, in the present embodiment, the following soft-oriented driving support is performed. Soft-oriented driving assistance is a system that is flexible in consideration of human weaknesses in cognitive science. Even if a warning is given to the driver immediately before the danger, the driver's field of view is stuck to the display, and the surroundings are difficult to see. As an example of the proposed display, as described above, the color of the image changes from blue, yellow, and red according to the degree of danger, and a mechanism for generating a voice warning is also generated each time a voice is presented. The blue image gives the driver a visual effect that remains in the background. Each time the color changes from yellow to red, a visual effect is given to the driver in the foreground. By informing the driver of the danger in advance, it is possible to prevent the surrounding field of view from being narrowed.

Human-assisted driving mechanism using intention
By the pedestrian and vehicle detection mechanism, it is possible to obtain direction information of the faces of the pedestrian and the driver. When a pedestrian is present at a position where the driver is not paying attention from the direction information of the driver's face, the pedestrian information is presented to the driver. Inferring the driver's intention does not result in excessive information, so it does not cause confusion for the driver. The pedestrian information is displayed in the HTML format. If the information changes suddenly depending on the speed of the pedestrian, the driver is confused. Information obtained from the server can be updated by a user setting. In addition, the presence of a vehicle can be presented to a pedestrian using a portable information terminal.

As an example, FIG. 10 shows a case where a pedestrian and a vehicle are presented 2, 4, 6 seconds before entering the intersection when the pedestrian and the vehicle head for the intersection on an L-shaped road with poor visibility. FIG. 10A shows a state 2 seconds ago, FIG. 10B shows a state 4 seconds ago, and FIG. 10C shows a state 6 seconds ago. In the example, for example FIG. 10 (a), the driver, the driving support information including the risk information risk D ₁ is displayed in red, the walking assistance information to pedestrians, including risk information risk D ₁ Is displayed in red. In the example of FIG. 10 (b), the driver, driving and walking assistance information respectively pedestrian including risk information risk D ₂ is displayed in yellow. In the example of FIG. 10 (c), the driver, driving and walking assistance information respectively pedestrian including risk information risk D ₃ is displayed in blue.

Welfare and disaster prevention cyber city
In the space surrounding humans, intelligent agents that support humans have various sensors and intelligence in a distributed manner (distributed sensory intelligence), and each intelligent agent cooperates autonomously, so that it is appropriate for humans. Need to show support and guide. Such a space is called a distributed sensory intelligence space. Distributed sensory intelligence consists of a camera and a CPU. In this embodiment, the distributed sensory intelligence space is expanded to the urban space where people live, and it is a disaster-resistant, welfare / disaster prevention cyber cyber with a focus on everything from daily life to emergency support in the event of a disaster. Realization of urban construction is possible.

  As described above, while the development of the information infrastructure is progressing, the social situation is declining birthrate and aging, and the IT disparity with the information weak is progressing accordingly. A major factor that hinders the use of infrastructure by vulnerable information is the difficulty in operating information equipment. Personal information terminals (PDA) and human vehicles, etc., which are handled by vulnerable individuals, recognize human emotions and intentions from human movements and provide human support so that Sony's AIBO (registered trademark) can show various expressions through human movements. If these can be done, and in the event of a disaster / emergency, they switch to disaster prevention safety functions and protect human safety, users will be able to use these crises in their daily lives. Real peace of mind. FIG. 11 is a schematic diagram of a welfare disaster prevention cyber city. As shown in FIG. 11, a plurality of cameras 31 and 32 are arranged near an intersection or road in the environment around the station. Walking support calculated based on images captured by these cameras 31 and 32 when a plurality of vehicles 2 are parked and the vehicle 2 enters the intersection or a pedestrian 1 comes out from the vicinity of the station. Information and driving support information are transmitted to the pedestrian 1 and the vehicle 2.

  As described above, in this embodiment, an intelligent interface that supports a weak person living in an urban space with an information infrastructure is assumed. Welfare disaster prevention that spreads throughout the city by a natural dialogue mechanism using PDA so that personal information terminals such as PDAs can communicate daily welfare information to vulnerable people and their supporters, and emergency information in the event of a disaster It is possible to construct a disaster prevention cyber city based on PDA, and a safe human-linked mobility system through cooperation with cyber city using welfare disaster prevention PDA.

  In other words, the distributed sensory intelligence installed at intersections and vehicles transmits and receives data by performing socket communication with a wireless LAN. Pedestrians and vehicles are detected by distributed sensory intelligence installed at intersections. By presenting the presence of a pedestrian to the driver and presenting the presence of a vehicle to the pedestrian, it is possible to provide support and guidance to a human.

  As described above, according to the present embodiment, it is possible to issue a danger warning in the direction in which the driver is not gazing from the direction information of the driver's face, and present only necessary information without being excessive in information. can do.

  The present invention is not limited to the above embodiment.

  In the above-described embodiment, the example in which the human support information is generated based on the intention recognition result obtained by the intention recognition unit 36 is shown, but the present invention is not limited to this. Human support information may be generated based only on vehicle position information and pedestrian position information.

Moreover, although the human assistance management part 37 simplified the intention recognition result and showed the case where walking assistance information and driving assistance information were produced | generated based on the left turn, straight advance, and right turn, it is not limited to this. For example, the human support management unit 37 may generate danger information using fuzzy associative reasoning, similar to the intention recognition unit 36. In this case, for example, the input node is set to the driving intention grade, and the output node is associated with the risk level. Then, an associative memory in which the driving intention and the degree of risk are associated in advance is generated in the same manner as the driving intention and the vehicle position V _L , the vehicle speed V _v and the face direction θ _D in the intention recognition unit 36. And whenever a driving intention grade is obtained, the human assistance management part 37 produces | generates a danger level. Thereby, even when the intention recognition result is ambiguous that does not clearly converge to the registration result such as left turn, straight advance, and right turn, an appropriate degree of risk can be obtained.

  Further, the determination of the risk level in the human support manager 37 is only an example. When other environmental information such as the pedestrian's intention to walk, the number of pedestrians, and road information can be obtained, the support accuracy is improved by making a determination including the environmental information. When fuzzy associative reasoning is applied to the human support management unit 378, it is possible to easily improve support accuracy by setting each piece of environment information as an input node in advance. When processing in this way, for example, when road information cannot be obtained or the pedestrian's intention to walk, even if all the environmental information is not obtained, the advantage that the degree of risk can be determined only from the obtained information Have Also, when more environmental information can be obtained, the accuracy of driving and walking support can be easily improved by simply setting new environmental information in the input node and obtaining an association matrix. Thus, the more accurate the environmental information is entered, the better the support accuracy can be.

  Moreover, although the intention recognition result showed only the left turn, the straight ahead, and the right turn, it is not limited to this. For example, more representative driving behaviors such as stopping, changing oblique lines, and parking may be registered. In this way, by registering more driving actions, fine driving support can be achieved.

  Furthermore, in the said embodiment, although demonstrated with the example of a vehicle and a pedestrian, it can apply to the driving assistance of vehicles by substituting a pedestrian for a vehicle, By substituting a vehicle for a pedestrian, It can also be applied to walking support between pedestrians. That is, it goes without saying that the present invention can be applied to movement support for all moving objects. Furthermore, the type of vehicle is not limited to the automobile shown in the specific example. Bicycles and motorcycles are also included in the vehicle.

  Also, the pedestrian position is calculated for each vehicle position using two cameras installed at point A and point B, but when there are multiple moving objects such as vehicles and pedestrians in the imaging region, What is necessary is just to select and calculate the position about either of a some mobile body.

  As described above, the present invention is a technology of a human support system that supports elderly people, disabled persons, etc. who have been marginalized from information technology living in an urban space with an information infrastructure and who have not fully obtained the benefits. Effective in the field.

The figure which shows an example of the network system with which the human assistance system which concerns on one Embodiment of this invention is applied. The figure which shows an example of the detailed structure of the vehicle which comprises the human assistance system which concerns on the embodiment, a server, and an intersection installation camera. The figure for demonstrating the pedestrian location information calculation process in the pedestrian location information calculation part which concerns on the same embodiment. It shows the center-of-gravity position P _Ct as time series data according to the same embodiment. The conceptual diagram of the intention recognition model which concerns on the same embodiment. The figure for demonstrating the concept of CFS (concept fuzzy set) which concerns on the same embodiment. The figure for demonstrating the concept of the bidirectional associative memory (BAM) which concerns on the same embodiment. The schematic diagram which expressed the fuzzy rule which concerns on the embodiment using the associative memory network. The figure which shows the knowledge required when constructing | assembling the network of the associative memory which concerns on the embodiment. The figure which shows typically the case where the pedestrian and vehicle which concern on the embodiment show to the driver 2, 4, and 6 seconds before entering the intersection when heading for the intersection. Schematic of the welfare disaster prevention cyber city according to the embodiment. The figure which shows the data flow of the intention recognition determination process in the intention recognition part which concerns on the same embodiment. The figure which shows the data flow of the intention recognition determination process in the intention recognition part which concerns on the same embodiment. The figure which shows the data flow of the driving assistance information in the human assistance management part which concerns on the embodiment, and walking assistance information generation.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Pedestrian, 2 ... Vehicle, 3 ... Server, 21 ... Visual field imaging camera, 22 ... Display part, 23 ... Control part, 24 ... Vehicle mounted camera, 31, 32 ... Intersection installation camera, 33 ... Vehicle position information calculation 34: Pedestrian position information calculation unit, 35 ... Face direction information calculation unit, 36 ... Intention recognition unit, 37 ... Human support management unit, 38 ... Communication control means, 41, 42 ... Road, 43 ... Intersection,

Claims (15)

A camera installed in the course of the vehicle,
A vehicle position information calculation unit that calculates the position information of the vehicle by performing image processing on the route image information captured by the camera;
A pedestrian position information calculation unit that performs image processing on the route image information captured by the camera and calculates position information of the pedestrian;
A person who generates walking support information for informing the pedestrian of the presence of the vehicle based on the vehicle position information and the pedestrian position information, or generates driving support information for informing the vehicle of the presence of the pedestrian. Support management department,
Communication control means for transmitting the walking support information or the driving support information;
A vehicle-mounted camera that is mounted on the vehicle and images a driver of the vehicle;
A field-of-view imaging camera fixed to the driver of the vehicle and capturing an image of the field of view of the driver;
A face direction information calculation unit that calculates the direction of the driver's face based on the driver captured image obtained by the vehicle-mounted camera and the field of view image obtained by the field of view image capturing camera;
The human support system characterized by comprising. Furthermore, an intention recognition unit that recognizes the intention of the driver of the vehicle and outputs an intention recognition result is provided.
The human support management unit, the human support system according to claim 1, characterized in that to generate the walking assistance information or driving support information based on the intention recognizing section intended recognition result obtained in.   The human support system according to claim 1, further comprising a display device that is carried by a pedestrian and receives and displays the walking support information.   The human support system according to claim 1, further comprising a display device mounted on a vehicle for receiving and displaying the driving support information. The communication control means receives the speed of the vehicle and the direction of the face of the driver of the vehicle,
The intention recognition unit recognizes a driver's intention based on the vehicle speed received by the communication control unit, the face direction, and the vehicle position information calculated by the vehicle position information calculation unit. The human support system according to claim 1, wherein: A camera installed in the course of the vehicle,
A vehicle position information calculation unit that calculates the position information of the vehicle by performing image processing on the route image information captured by the camera;
A pedestrian position information calculation unit that performs image processing on the route image information captured by the camera and calculates position information of the pedestrian;
A person who generates walking support information for informing the pedestrian of the presence of the vehicle based on the vehicle position information and the pedestrian position information, or generates driving support information for informing the vehicle of the presence of the pedestrian. Support management department,
Communication control means for transmitting the walking support information or the driving support information and receiving the speed of the vehicle and the direction of the face of the driver of the vehicle ;
A storage device that stores a plurality of associative matrices for each driver, each of which associates an input parameter including the speed of the vehicle, the direction of the face, and the position information of the vehicle, and an output parameter indicating the driving intention of the driver; ,
The vehicle speed information received by the communication control means, the face direction, and the vehicle position information calculated by the vehicle position information calculation unit are echoed based on an association matrix read from the storage device. An intention recognition unit for recognizing the driver's driving intention based on the state transition accompanying the movement;
The human support system characterized by comprising. Before Symbol Human Support management unit, the human support system according to claim 6, characterized in that to generate the walking assistance information or driving support information based on the intent recognition result obtained by the intention recognizing section. The human support system according to claim 6 , further comprising a display device that is carried by a pedestrian and receives and displays the walking support information. The human support system according to claim 6 , further comprising a display device mounted on a vehicle for receiving and displaying the driving support information. The human association system according to claim 6 , wherein the association matrix is stored for a plurality of different states of one driver. The intention recognition unit includes a determination unit that determines the driving intention most strongly recalled among the driving intentions calculated for the association matrix for each of the plurality of drivers as the driving intention of the driver. The human support system according to claim 6 . A step of image processing the path image information captured by a camera mounted in the path of the vehicle calculates the position information of the vehicle,
Calculating a position information of the pedestrian image processing route image information captured by the camera,
Step based on said vehicle position information and the pedestrian position information, generates the walking assistance information for notifying existence of the vehicle pedestrian, or generates the driving support information for notifying the presence of the pedestrian on the vehicle And
Transmitting the walking support information or the driving support information;
Imaging a vehicle driver with a vehicle-mounted camera mounted on the vehicle;
Capturing an image of the driver's field of view with a field of view imaging camera fixed to the driver of the vehicle;
Calculating a driver's face direction based on a driver captured image obtained by the vehicle-mounted camera and a visual field image obtained by the visual field image capturing camera;
A human support method comprising the steps of : Computer
A vehicle position information calculation unit that calculates the position information of the vehicle by performing image processing on the route image information captured by a camera installed in the route of the vehicle;
A pedestrian position information calculation unit that performs image processing on the route image information captured by the camera and calculates position information of the pedestrian;
A person who generates walking support information for informing the pedestrian of the presence of the vehicle based on the vehicle position information and the pedestrian position information, or generates driving support information for informing the vehicle of the presence of the pedestrian. A support management department;
Communication control means for transmitting the walking support information or the driving support information;
Obtained by capturing an image of a driver captured by a vehicle-mounted camera mounted on the vehicle and an image of the driver's field of view with a field-of-view image capturing camera fixed to the driver of the vehicle. A human support program that functions as a face direction information calculation unit that calculates a driver's face direction based on the field of view image . A step of image processing the path image information captured by a camera mounted in the path of the vehicle calculates the position information of the vehicle,
Calculating the pedestrian's position information by image-processing the course image information captured by the camera;
Step based on said vehicle position information and the pedestrian position information, generates the walking assistance information for notifying existence of the vehicle pedestrian, or generates the driving support information for notifying the presence of the pedestrian on the vehicle And
Transmitting the walking support information or the driving support information;
Receiving the speed of the vehicle and the direction of the face of the driver of the vehicle;
For the received vehicle speed, the face direction, and the calculated vehicle position information, an input parameter including the vehicle speed, the face direction, and the vehicle position information, and a driver The step of recognizing the driver's driving intention by the state transition associated with the reverberating action based on the association matrix read out from the storage device that stores a plurality of associative matrices for each driver in association with the output parameter indicating the driving intention of the driver When,
A human support method comprising the steps of: Computer
A vehicle position information calculation unit that calculates the position information of the vehicle by performing image processing on the route image information captured by a camera installed in the route of the vehicle;
A pedestrian position information calculation unit that performs image processing on the route image information captured by the camera and calculates position information of the pedestrian;
A person who generates walking support information for informing the pedestrian of the presence of the vehicle based on the vehicle position information and the pedestrian position information, or generates driving support information for informing the vehicle of the presence of the pedestrian. Support management department,
Communication control means for transmitting the walking support information or the driving support information and receiving the speed of the vehicle and the direction of the face of the driver of the vehicle ;
A storage device that stores a plurality of associative matrices for each driver, each of which associates an input parameter including the speed of the vehicle, the direction of the face, and the position information of the vehicle, and an output parameter indicating the driving intention of the driver; ,
The vehicle speed information received by the communication control means, the face direction, and the vehicle position information calculated by the vehicle position information calculation unit are echoed based on an association matrix read from the storage device. A human support program that functions as an intention recognizing unit that recognizes a driver's intention based on a state transition associated with movement .

Images