JP2005078377A - Traveling object detecting device and method, and robot device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a traveling object detecting device for accurately detecting only a target traveling object by improving detection precision and sensitivity without increasing any calculation load, and to provide a method therefor and a robot device. <P>SOLUTION: A robot device 1 starts traveling object detection processing (step S1) when judging that the robot device itself does not move, divides a preceding image inputted as a luminance image and a present image adjacent to the preceding image on a time axis into blocks constituted of a plurality of pixels, and calculates the difference square sum of every block. When the different square sum is not less than a fixed value (step S2), the robot device 1 operates labeling to those blocks (step S3). Then, areas where the labeled traveling object candidate blocks adjoin are connected, and when its size is not less than a predetermined value (step S4), the area whose size is the largest is selected as the target of detection (step S5). Lastly, whether the targeted block connection area is the traveling object is judged (step S6). <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a moving object detection apparatus, a moving object detection method, and a robot apparatus equipped with the moving object detection apparatus, and more particularly to a moving object detection apparatus and a moving object detection method that are designed to detect only a target object accurately and at high speed. And a robot apparatus equipped with the same.

  A mechanical device that performs an action similar to that of a human (living body) using an electrical or magnetic action is called a “robot device”. Robot devices began to spread in Japan from the end of the 1960s, but most of them were industrial robot devices (manipulators, transfer robot devices, etc.) for the purpose of automating and unmanned production work in factories ( Industrial Robot).

  Recently, practical robot devices that support life as a human partner, that is, support human activities in various situations in daily life such as the living environment, have been developed. Such industrial robot devices, unlike industrial robot devices, have the ability to learn how to adapt themselves to people with different personalities or to various environments in various aspects of the human living environment. Yes. For example, a “pet-type” robot that mimics the body mechanism and movement of a quadruped animal such as a dog or cat, or the body mechanism or movement of an animal that walks upright on two legs. Legged mobile robotic devices such as “humanoid” or “humanoid” robotic devices have already been put into practical use.

  Since these legged mobile robot devices can perform various operations with an emphasis on entertainment properties as compared with industrial robot devices, they may be referred to as entertainment robot devices.

  The legged mobile robot device has an appearance shape that is as close as possible to the appearance of animals and humans, and is designed to perform movements that are as close as possible to the movements of animals and humans. For example, the above-described four-legged “pet-type” robot apparatus has an external shape similar to a dog or a cat raised in a general home, and is “struck” or “boiled” from a user (owner). Act autonomously according to the approach and surrounding environment. For example, as an autonomous behavior, the behavior such as “barking” and “sleeping” is performed in the same manner as an actual animal.

  By the way, some legged mobile robot devices determine surrounding conditions based on an image captured by a CCD (Charge Coupled Device) camera and act autonomously based on the determination result. According to such a legged mobile robot device, for example, a moving object existing in the vicinity can be detected and the moving object can be tracked.

  Here, various methods have been proposed and put to practical use as methods for detecting moving objects in an image by image processing. These methods are roughly classified into a first method for obtaining a pixel difference between frame images, a second method for obtaining an optical flow at an arbitrary point on an image using a spatiotemporal filter, and a small region between frames. And a third method for obtaining the movement of the region. These methods are based on the premise that a camera for capturing an image is fixed and the detected motion is an external motion.

  However, in the case where the robot apparatus itself operates like the above-described legged mobile robot apparatus, the mounted CCD camera moves greatly and complicatedly with its own operation. For this reason, if the moving object in the image is detected by the above-described first to third methods, the entire image moves greatly between frames, so that even if the moving object is actually imaged, the movement is detected. Was difficult.

  Therefore, the present applicant has proposed a technique for accurately detecting a moving object existing in the surroundings regardless of the presence or absence of its own movement in Patent Document 1 below. According to the technique described in Patent Document 1, an image region having a high correlation with the central portion of the previous frame image is extracted from the current frame image, and the movement of the entire image is detected. Regardless of the presence or absence of motion, it is possible to accurately detect moving objects in the image.

JP 2002-251615 A

  However, in the technique described in Patent Document 1, in order to extract an image region having a high correlation with the central portion of the previous frame image, each template divided region obtained by dividing the central portion of the previous frame image and the current frame image as predetermined pixels. There is a problem in that it is necessary to calculate the matching score sequentially while shifting in units of one pixel in the horizontal direction and the vertical direction between the search image divided areas divided while being overlapped by the same amount. On the other hand, as described above, in an autonomous legged mobile robot device or the like, it is difficult to distinguish between its own movement and the movement of a moving object. Misdetections that would cause moving objects occur frequently.

  In addition, a moving object is usually detected for tracking the moving object. For example, when there are a plurality of moving objects, the moving object is not suitable as a tracking target. Therefore, even a moving object does not need to be detected by the robot apparatus. There is a case. Especially in entertainment robots and the like, the resources are limited, so detection errors and detection of moving objects that cannot be tracked are prevented, while the calculation load is reduced as much as possible, and only moving objects to be tracked are detected at high speed. It is desirable.

  The present invention has been proposed in view of such a conventional situation, and can improve the accuracy and sensitivity of detection without increasing the calculation load, and can accurately detect only a target moving object. It is an object of the present invention to provide a moving object detection apparatus, a moving object detection method, and a robot apparatus.

  In order to achieve the above-described object, the moving object detection device according to the present invention obtains a block difference value obtained for each block composed of a plurality of pixels by calculating a difference between adjacent images taken on the time axis taken by the imaging unit. A block difference value calculating means for calculating, and a moving object detecting means for detecting, as a moving object area, a block having a block difference value equal to or greater than a predetermined threshold as a moving object candidate block, and a block connection area adjacent to the moving object candidate block as a moving object area. It is characterized by having.

  In the present invention, instead of obtaining a difference for each pixel between adjacent frame images and detecting a moving object, the resolution is reduced as a block unit in which a plurality of pixels are combined, and a difference value is calculated for each block. Therefore, even if the threshold value for determining a moving object candidate block is lowered, detection of noise or the like can be suppressed as compared with processing in pixel units, and the calculation amount can be reduced. Can be reduced.

  Further, the moving object detection means can determine whether or not the number of blocks included in the block connection area is equal to or greater than a predetermined threshold value, and a block connection area that is equal to or greater than the predetermined threshold value can be determined as the moving object area. When the detected block connection area is small, the detection accuracy of the target moving object area is further improved by canceling as a small area or noise for tracking as a moving object.

  Furthermore, when there are a plurality of the block connection areas, the moving object detection unit can detect the largest area as the moving object area, and can set the largest area as a tracking target.

  Furthermore, the moving object detection means can end the moving object detection without setting the moving object area as a detection target when the size of the moving object area in the image is equal to or larger than a predetermined threshold. For example, when the rate exceeds 30%, the detection can be stopped because it is not necessary to track a large amount as a tracking target.

  Further, the moving object detection means can end the moving object detection without setting the moving object region as a detection target when the horizontal or vertical size of the moving object region is equal to or larger than a predetermined threshold value. Even if the occupation ratio is within a predetermined range, when the moving object region covers a wide area, the moving object detection can be stopped as noise.

  Further, the moving object detection means can end the moving object detection without detecting the moving object region when the number of moving object candidate blocks is equal to or greater than a predetermined threshold, and a plurality of moving object candidate blocks are detected. The moving object detection can be stopped because there are a plurality of moving objects or noise.

  The moving object detection method according to the present invention includes a block difference value calculation step of calculating a block difference value obtained for each block composed of a plurality of pixels, the difference between images adjacent on the time axis imaged by the imaging means, And a moving object detection step of detecting a block having a block difference value equal to or greater than a predetermined threshold as a moving object candidate block, and detecting a block connection area formed by adjoining the moving object candidate blocks as a moving object area.

  The robot device according to the present invention is a robot device that autonomously determines and expresses an action based on a difference between an imaging unit that captures an image and an image that is adjacent on the time axis captured by the imaging unit. A moving object detecting means for detecting a block difference value calculating means for calculating a block difference value obtained by calculating a difference between the images for each block composed of a plurality of pixels, and the block difference. A moving object detection unit that detects a block having a value equal to or greater than a predetermined threshold as a moving object candidate block and detects a block connection area formed by adjoining the moving object candidate blocks as a moving object area is provided.

  In the present invention, a robot apparatus that operates autonomously is equipped with a moving object detection device that calculates a difference for each block and determines a moving object, thereby reducing the resolution and increasing the sensitivity to achieve high speed and high accuracy. A moving object can be detected.

  Moreover, it can have a control means to reflect the detection result of the said moving body detection means in the said action, for example, the said control means is made to express the different action according to the magnitude | size of the detected said moving body area | region etc. By doing so, in addition to the action of tracking the moving object based on the size of the moving object, the robot apparatus further develops entertainment properties by expressing an action that combines various actions.

  According to the present invention, when detecting a moving object, a difference value in units of blocks obtained by dividing each image into blocks each having a plurality of pixels as a unit is calculated between two images consecutive on the time axis. In addition, since it is determined whether or not it is a moving object candidate based on the difference value for each block, the resolution is lower than that of determining whether or not it is a moving object in units of pixels. The amount of processing for extracting candidate moving object candidate blocks is small, and as a result, the moving object detection sensitivity can be improved and the moving object detection processing speed can be increased.

  Moreover, according to the robot apparatus equipped with such a moving object detection device, only a moving object for tracking purposes can be detected at high speed and accurately, and entertainment can be further improved.

  Hereinafter, specific embodiments to which the present invention is applied will be described in detail with reference to the drawings. In this embodiment, the present invention is equipped with a moving object detection device that detects a moving object included in an input image, and autonomously expresses an action according to an external environment (external stimulus) or an internal state. Is applied. Here, the robot apparatus will be described first, and then the moving object detection process in the robot apparatus will be described.

(1-1) Configuration of Robot Device FIG. 1 is a perspective view showing a robot device according to the present embodiment. As shown in FIG. 1, the robot apparatus 1 according to the present embodiment is a four-legged legged mobile robot apparatus, and leg units 3A, 3B, 3C, and 3D are connected to the front and rear and left and right of the body unit 2, respectively. In addition, the head unit 4 is connected to the front end of the body unit 2.

  As shown in FIG. 2, the body unit 2 includes a CPU (Central Processing Unit) 10, a DRAM (Dynamic Random Access Memory) 11, a flash ROM (Read Only Memory) 12, a PC (Personal Computer) card interface circuit 13 and A control unit 16 formed by connecting the signal processing circuit 14 to each other via the internal bus 15 and a battery 17 as a power source of the robot apparatus 1 are housed. The body unit 2 also stores an angular velocity sensor 18 and an acceleration sensor 19 for detecting the direction of the robot apparatus 1 and acceleration of movement. Further, a tail portion 5 is formed at a rear end portion located in a direction opposite to the head portion of the body unit 2.

  Further, the head unit 4 captures an external situation and detects a CCD (Charge Coupled Device) camera 20 for detecting ambient brightness and a physical action from the user such as tilting forward and backward. Touch sensor 21, a distance sensor 22 for measuring the distance to an object located in front, a microphone 23 for collecting external sounds, a speaker 24 for outputting various sounds, a head A headlight 25 that can be stored in the unit unit 4 and an LED (Light Emitting Diode) (not shown) corresponding to the “eyes” of the robot apparatus 1 are arranged at predetermined positions. In the robot apparatus 1, in addition to the touch sensor 21, a plurality of touch sensors are arranged at predetermined positions of the body unit 2 and the head unit 4.

Furthermore, the joint portions of the leg units 3A to 3D, the connection portions of the leg units 3A to 3D and the body unit 2, and the connection portions of the head unit 4 and the body unit 2 have a degree of freedom respectively. Actuators 26 _{1 to} 26 _n and potentiometers 27 _{1 to} 27 _n are arranged. For example, the actuators 26 _{1 to} 26 _n have a servo motor as a configuration. The leg units 3 </ b> A to 3 </ b> D are controlled by the drive of the servo motor, and the target posture or operation is changed.

These angular velocity sensor 18, acceleration sensor 19, touch sensor 21, distance sensor 22, microphone 23, speaker 24, various sensors such as potentiometers 27 _{1 to} 27 _n and headlight 25, LEDs and actuators 26 ₁ to 26. _n is connected to the signal processing circuit 14 of the control unit 16 via the corresponding hubs 28 _{1 to} 28 _n , and the CCD camera 20 and the battery 17 are directly connected to the signal processing circuit 14 respectively.

  The signal processing circuit 14 sequentially takes in sensor data, image data, and audio data supplied from each of the above-described sensors, and sequentially stores them at predetermined positions in the DRAM 11 via the internal bus 15. In addition, the signal processing circuit 14 sequentially takes in battery remaining amount data representing the remaining amount of the battery supplied from the battery 17 and stores it in a predetermined position in the DRAM 11.

  The sensor data, image data, audio data, and battery remaining amount data stored in the DRAM 11 in this way are used when the CPU 10 controls the operation of the robot apparatus 1 thereafter.

  In practice, when the power of the robot apparatus 1 is initially turned on, the CPU 10 loads the control program stored in the memory card 29 or the flash ROM 12 loaded in the PC card slot (not shown) of the body unit 2 into the PC card interface circuit 13. Or directly read out and stored in the DRAM 11.

  In addition, the CPU 10 thereafter, based on each sensor data, image data, audio data, and battery remaining amount data sequentially stored in the DRAM 11 by the signal processing circuit 14 as described above, Determine whether there are instructions and actions.

  Here, for example, when executing the face detection task in the present embodiment, face candidates are detected from the input image captured by the CCD, and values such as a distance sensor are used to apply the above four conditions. A non-face candidate is detected, and a non-face candidate excluded from the face candidate is output as a face area.

Further, the CPU 10 determines the action to be continued based on the determination result and the control program stored in the DRAM 11, and drives the necessary actuators 26 _{1 to} 26 _n based on the determination result, thereby causing the head unit 4 to move. Actions such as swinging up and down, left and right, or driving each leg unit 3A to 3D to walk are performed.

  At this time, the CPU 10 generates audio data as necessary, and outputs the audio data to the speaker 24 as an audio signal via the signal processing circuit 14 so that the audio based on the audio signal is output to the outside. Turn on, turn off, or blink the LED. Further, as will be described later, the CPU 10 detects ambient brightness using the CCD camera 20 and turns on the headlight 25 according to the detection result.

  In this way, the robot apparatus 1 can act autonomously according to the situation of itself and surroundings, and instructions and actions from the user.

(1-2) Software Configuration of Control Program Here, the software configuration of the above-described control program in the robot apparatus 1 is as shown in FIG. In FIG. 3, a device driver layer 30 is located in the lowest layer of the control program and is composed of a device driver set 31 composed of a plurality of device drivers. In this case, each device driver is an object that is allowed to directly access hardware used in a normal computer such as the CCD camera 20 (FIG. 2) or a timer, and receives an interrupt from the corresponding hardware. Process.

The robotic server object 32 is located in the lowest layer of the device driver layer 30 and provides an interface for accessing hardware such as the various sensors and actuators 26 _{1 to} 26 _n described above. A virtual robot device 33 made up of software, a power manager 34 made up of software that manages power supply switching, a device driver manager 35 made up of software that manages other various device drivers, and a robot It is composed of a designed robot apparatus 36 that is a software group for managing the mechanism of the apparatus 1.

  The manager object 37 includes an object manager 38 and a service manager 39. The object manager 38 is a software group that manages the activation and termination of each software group included in the robotic server object 32, the middleware layer 40, and the application layer 41. The service manager 39 includes: This is a software group for managing the connection of each object based on the connection information between the objects described in the connection file stored in the memory card 29 (FIG. 2).

  The middleware layer 40 is located in an upper layer of the robotic server object 32, and includes a software group that provides basic functions of the robot apparatus 1 such as image processing and sound processing. In addition, the application layer 41 is located in an upper layer of the middleware layer 40, and determines the behavior of the robot apparatus 1 based on the processing result processed by each software group constituting the middleware layer 40. It is composed of software groups.

  The specific software configurations of the middleware layer 40 and the application layer 41 are shown in FIG.

  As shown in FIG. 4, the middle wear layer 40 is for noise detection, temperature detection, brightness detection, scale recognition, distance detection, posture detection, touch sensor, motion detection and color recognition. Recognition system 60 having signal processing modules 50 to 58 and input semantic converter module 59 for output, output semantic converter module 68, posture management, tracking, motion playback, walking, fall-returning, and lighting And an output system 69 having signal processing modules 61 to 67 for sound reproduction.

  The signal processing modules 50 to 58 of the recognition system 60 correspond to the corresponding data among the sensor data, image data and audio data read from the DRAM 11 (FIG. 2) by the virtual robot device 33 of the robotic server object 32. Are processed based on the data, and the processing result is given to the input semantic converter module 59. Here, for example, the virtual robot apparatus 33 is configured as a part that exchanges or converts signals according to a predetermined communication protocol.

  Based on the processing result given from each of these signal processing modules 50 to 58, the input semantic converter module 59 is “noisy”, “hot”, “bright”, “ball detected”, “falling detected”, Self and surrounding conditions such as “boiled”, “struck”, “I heard Domiso's scale”, “Detected moving object” or “Detected an obstacle”, and commands from the user And the action is recognized, and the recognition result is output to the application layer 41 (FIG. 2).

  As shown in FIG. 5, the application layer 4 l includes five modules: a behavior model library 70, a behavior switching module 71, a learning module 72, an emotion model 73, and an instinct model 74.

In the behavior model library 70, as shown in FIG. 6, “when the remaining battery level is low”, “when falling down”, “when avoiding an obstacle”, “when expressing emotion”, “ Independent behavior models 70 ₁ to 70 _n are provided in correspondence with a plurality of preselected condition items such as “when a ball is detected”.

These behavior models 70 ₁ to 70 _n are respectively used as necessary when a recognition result is given from the input semantic converter module 59 or when a certain time has passed since the last recognition result was given. As will be described later, the following behavior is determined while referring to the corresponding emotion parameter value held in the emotion model 73 and the corresponding desire parameter value held in the instinct model 74, and the decision result is switched to the action. Output to module 71.

In the case of this embodiment, each of the behavior models 70 ₁ to 70 _n is used as a method for determining the next behavior from one node (state) NODE _{0 to} NODE _n as shown in FIG. determined probabilistically based on the set transition probability _P 1 to P _n respectively whether a transition to nODE ₀ ~NODE _n relative to the arc _aRC 1 _{~ARC n} that connects to each node nODE ₀ ~NODE _n An algorithm called a finite probability automaton is used.

Specifically, the behavior model ₇₀ 1 to 70 _n are each respectively made to correspond to the node NODE ₀ ~NODE _n to form a self-behavior model ₇₀ 1 to 70 _n, Fig each of these nodes NODE ₀ ~NODE _n 8 A state transition table 80 as shown in FIG.

In this state transition table 80, input events (recognition results) as transition conditions in the nodes NODE _{0 to} NODE _n are listed in the “input event name” column in priority order, and further conditions regarding the transition conditions are “data name”. ”And“ data range ”columns are described in corresponding rows.

Therefore, in the node NODE ₁₀₀ represented by the state transition table 80 in FIG. 8, when the recognition result “ball detected (BALL)” is given, the “size (SIZE)” of the ball given together with the recognition result is given. ) ”Is in the range of“ 0 to 1000 ”, or when a recognition result“ OBSTACLE ”is given, the“ distance (DISTANCE) to the obstacle given along with the recognition result ” "Is in the range of" 0 to 100 "is a condition for transitioning to another node.

Further, in this node NODE ₁₀₀ , each emotion and each desire parameter held in the emotion model 73 and the instinct model 74 that the behavior models 70 ₁ to 70 _n periodically refer to even when no recognition result is input. Among the values, when the parameter value of “joy”, “surprise”, or “sadness” (SUDNESS) held in the emotion model 73 is in the range of “50 to 100”, It is possible to transition to a node.

In the state transition table 80, node names that can be transitioned from the nodes NODE ₀ to NODE _n are listed in the “transition destination node” row in the “transition probability to other node” column, and “input event name” ”,“ Data value ”, and“ data range ”, the transition probability to each of the other nodes NODE ₀ to NODE _{n that} can transition when all the conditions described in the columns are met is“ transition probability to other nodes ”. The action to be output when transitioning to the nodes NODE _{0 to} NODE _n is described in the “output action” line in the “transition probability to other nodes” column. Yes. The sum of the probabilities of each row in the “transition probability to other node” column is 100 [%].

Therefore, in the node NODE ₁₀₀ represented by the state transition table 80 of FIG. 7, for example, “ball is detected (BALL)” and the “SIZE (size)” of the ball is in the range of “0 to 1000”. When the recognition result is given, it is possible to transition to “node NODE ₁₂₀ (node 120)” with a probability of “30 [%]”, and the action of “ACTION 1” is output at that time.

Each of the behavior models 70 ₁ to 70 _n is configured such that a number of nodes NODE _{0 to} NODE _n described as the state transition table 80 are connected to each other, and a recognition result is given from the input semantic converter module 59. The next action is determined probabilistically using the state transition table of the corresponding nodes NODE ₀ to NODE _n , and the determination result is output to the action switching module 71.

Action switching module 71 shown in FIG. 5, of the action output from each behavior model ₇₀ 1 to 70 _n of the action model library 70, is output from a predetermined higher priority behavior model ₇₀ 1 to 70 _n A command indicating that the action should be executed (hereinafter referred to as an action command) is sent to the output semantic converter module 68 of the middleware layer 40. In this embodiment, higher priority is set for the behavior models 70 ₁ to 70 _n shown on the lower side in FIG.

  Further, the behavior switching module 71 notifies the learning module 72, the emotion model 73, and the instinct model 74 that the behavior is completed based on the behavior completion information given from the output semantic converter module 68 after the behavior is completed.

  On the other hand, the learning module 72 inputs the recognition result of the teaching received from the user, such as “struck” or “boiled”, among the recognition results given from the input semantic converter module 59.

Then, based on the recognition result and the notification from the action switching module 71, the learning module 72 reduces the probability of the action when “struck (struck)” and “struck (praised). ) ”, The corresponding transition probabilities of the corresponding behavior models 70 ₁ to 70 _n in the behavior model library 70 are changed so as to increase the probability of occurrence of the behavior.

  On the other hand, the emotion model 73 is a sum of “joy”, “sadness”, “anger”, “surprise”, “disgust” and “fear”. For six emotions, a parameter representing the strength of the emotion is held for each emotion. Then, the emotion model 73 uses the parameter values of each emotion as specific recognition results such as “struck” and “boiled” given from the input semantic converter module 59, the elapsed time and the behavior switching module 71. It is updated periodically based on notifications from.

Specifically, the emotion model 73 is calculated by a predetermined arithmetic expression based on the recognition result given from the input semantic converter module 59, the behavior of the robot apparatus 1 at that time, the elapsed time since the last update, and the like. its emotional fluctuation amount △ E [t] of the time being, E [t] of the current parameter value of the emotion, the next cycle coefficient representing the sensitivity of the emotion as k _e, by (1) The parameter value E [t + 1] of the emotion in is calculated, and the parameter value of the emotion is updated so as to replace the current parameter value E [t] of the emotion. In addition, the emotion model 73 updates the parameter values of all emotions in the same manner.

  It should be noted that how much each notification result or notification from the output semantic converter module 68 affects the parameter value variation amount ΔE [t] of each emotion is determined in advance. For example, “struck” The recognition result has a great influence on the fluctuation amount ΔE [t] of the emotion parameter of “anger”, and the recognition result of “boiled” has a fluctuation amount ΔE [t] of the parameter value of the emotion of “joy” It has come to have a big influence on.

Here, the notification from the output semantic converter module 68 is so-called action feedback information (behavior completion information), which is information on the appearance result of the action, and the emotion model 73 changes the emotion also by such information. Let This is, for example, that the emotional level of anger is lowered by an action such as “barking”. The notification from the output semantic converter module 68 is also input to the learning module 72 described above, and the learning module 72 changes the corresponding transition probabilities of the behavior models 70 ₁ to 70 _n based on the notification.

  Note that the feedback of the behavior result may be performed by the output of the behavior switching module 71 (the behavior to which the emotion is added).

  On the other hand, the instinct model 74 has four independent needs of “exercise”, “affection”, “appetite” and “curiosity” for each of these needs. It holds a parameter that represents the strength of the desire. The instinct model 74 periodically updates the parameter values of these desires based on the recognition result given from the input semantic converter module 59, the elapsed time, the notification from the behavior switching module 71, and the like.

Specifically, the instinct model 74 uses the predetermined calculation formula for “exercise greed”, “loving lust” and “curiosity” based on the recognition result, elapsed time, notification from the output semantic converter module 68, and the like. Equation (2) is used in a predetermined cycle, where ΔI [k] is the calculated fluctuation amount of the desire at that time, I [k] is the current parameter value of the desire, and coefficient k _i is the sensitivity of the desire. Then, the parameter value I [k + 1] of the desire in the next cycle is calculated, and the parameter value of the desire is updated so that the calculation result is replaced with the current parameter value I [k] of the desire. Further, the instinct model 74 updates the parameter values of each desire except “appetite” in the same manner.

  Note that how much the recognition result and the notification from the output semantic converter module 68 affect the fluctuation amount ΔI [k] of each desire parameter value is determined in advance, for example, from the output semantic converter module 68. This notification has a great influence on the fluctuation amount ΔI [k] of the parameter value of “fatigue”.

In the present embodiment, the parameter values of each emotion and each desire (instinct) are regulated so as to fluctuate in the range from 0 to 100, respectively, and the values of the coefficients k _e and k _i are also set for each emotion. And it is set individually for each desire.

  On the other hand, the output semantics converter module 68 of the middleware layer 40, as shown in FIG. 4, “forward”, “joy”, “ring” given from the behavior switching module 71 of the application layer 41 as described above. "Or" tracking (following the ball) "is given to the corresponding signal processing modules 61-67 of the output system 69.

Then, these signal processing modules 61 to 67, when given an action command, based on the action command, servo command values to be given to the corresponding actuators 26 _{1 to} 26 _n (FIG. 2) and speaker 24 (FIG. 2), sound data output from the sound and / or drive data to be given to the “eye” LED are generated, and these data are used as the virtual robot device 33 and the signal processing circuit 14 (for the robotic server object 32). 2) are sequentially sent to the corresponding actuators 26 _{1 to} 26 _n or the speaker 24 or the LED via the sequential order.

  In this way, the robot apparatus 1 can perform autonomous actions in accordance with its own (internal) and surrounding (external) situations, and instructions and actions from the user, based on the control program. Has been made.

(2) Motion Detection In the robot apparatus 1 described above, the motion detection module 57 in the middle wear layer 40 described above can detect a motion from an image captured by the camera 20 as the imaging means.

  FIG. 9 is a block diagram showing only main parts related to the moving object detection process in the robot apparatus 1 described above. As shown in FIG. 9, input images from the camera 20 are sequentially captured by the signal processing circuit 14 (FIG. 2) and sequentially stored at predetermined positions in the DERAM 11 via the internal bus 15. An image used for moving object detection is a luminance image, and when the image is stored in the DRAM 11, the captured image is converted into a luminance image.

As described above, the robotic server object 32 is located in the lowest layer of the device driver layer 30 and provides an interface for accessing hardware such as various sensors and actuators 26 _{1 to} 26 _n. It has a virtual robot device 33 consisting of software groups.

  The middleware layer 40 is positioned above the robotic server object 32, and is a software group that provides basic functions of the robot apparatus 1 such as image processing such as motion detection and voice processing. It is configured.

  Furthermore, the application layer 41 is located in an upper layer of the middleware layer 40, and determines the behavior of the robot apparatus 1 based on the processing result processed by each software group constituting the middleware layer 40. It is composed of software groups.

  In the moving object detection process, first, the CPU 10 (FIG. 2) sequentially activates the robotic server object 32, the virtual robot apparatus 33, and the image processing objects constituting the middleware layer 40. Then, the luminance image data is fetched from the DRAM 11 by the virtual robot device 33 of the robotic server object 32, and this luminance image data is sent to the motion detection object 57. The motion detection object 57 calculates a block difference value obtained for each block composed of a plurality of pixels for a difference between adjacent images on the time axis, and selects a block whose block difference value is equal to or greater than a predetermined threshold as a moving object candidate And a process of detecting a block connection area formed by adjoining moving object candidate blocks as a moving object area is executed. Then, the detection result is output to the behavior model library 70 of the application layer 41. The behavior model library 70 determines the behavior based on the information that the moving object has been detected, and outputs the determination result to the behavior switching module 71.

  The behavior switching module 71 sends a command indicating that the behavior should be executed (hereinafter referred to as a behavior command) to the output semantic converter module 68 of the middleware layer 40. The output semantic converter module 68 An abstract action command such as “tracking (chase the ball)” is given to the tracking module 62 which is a corresponding signal processing module.

When the action command is given, the tracking module 62 generates a servo command value to be given to the corresponding actuators 26 _{1 to} 26 _n to perform the action based on the action command, and the robot module The server object 32 is sequentially sent to the corresponding actuators 26 _{1 to} 26 _n via the virtual robot device 33 and the signal processing circuit 14 (FIG. 2) in order.

  In this way, the robot apparatus 1 can autonomously track the moving object using the moving object detection result based on the control program, enables accurate moving object detection in real time, and smooth communication with the user. Can be realized.

Next, the moving object detection process in the motion detection module 57 will be described in detail. FIG. 10 is a flowchart showing the moving object detection method. As shown in FIG. 10, first, it is determined whether or not to start the moving object detection process (step S1). As the start determination, for example, it can be determined depending on whether or not 1: during posture transition 2: during walking 3: whether the neck is moving. That is, when the robot apparatus 1 itself moves, the camera 20 also moves. Therefore, in the present embodiment, for example, during posture transition such as when standing up from a lying position, during walking, and It is assumed that the moving object detection process is not performed during the neck movement such as rotating the head unit 4.

  Next, the previous image (previous frame image) input as a luminance image is compared with the current image (current frame image) which is adjacent to the previous image on the time axis and is also a luminance image, and the difference value is obtained. Ask. At this time, both the previous image and the current image are divided into blocks made up of a plurality of pixels, and the sum of squared differences for each block is obtained between the images. If the sum of squared differences is greater than or equal to a certain value, it is determined that the block has movement (step S2). Hereinafter, a block determined to have motion is referred to as a moving object candidate block.

FIG. 11 is a diagram for explaining a method of detecting moving object candidate blocks in step S2. As shown in FIG. 11, the current image C and the previous image P are composed of, for example, 104 pixels in the horizontal direction and 80 pixels in the vertical direction. A luminance difference value between the current image C and the previous image P is calculated in pixel units, and a difference value in block units including a plurality of pixels is obtained. First, each image is divided into i blocks in the horizontal direction and j blocks in the vertical direction, and as shown in FIG. 12, the sum of squares of differences between the current image block BC _ij and the previous image block BP _ij for each block. Is calculated. Then, it is determined whether or not the sum of squared differences for each block is equal to or greater than a predetermined threshold. Here, one block in the current image C and the previous image P is a block BC _ij and a block BP _ij (1 ≦ i ≦ 26, 1 ≦ j ≦ 20) each having 4 pixels in the horizontal direction and 4 pixels in the vertical direction. .

  Here, for example, when a pixel having a difference value equal to or larger than a predetermined threshold is extracted for each pixel, it is necessary to determine only the number of 104 × 80 = 8320 pixels. Thus, when the determination target is shown in FIG. 11, the number is equal to 26 × 20 = 520 blocks. That is, by lowering the resolution in this way, it is possible to eliminate pixel-by-pixel detection that has been detected in the past even though it is noise or the like, and to speed up the processing. In addition, even if the threshold value for each block is lowered to some extent and the detection sensitivity is increased, the detection target is a block unit, so it is possible to detect even a small amount of moving object while efficiently eliminating noise. Can do.

  Note that, here, a block composed of 4 × 4 pixels will be described, but the size of the block is appropriately set according to the environment such as camera performance and ambient brightness. Also, the threshold for determining a moving object candidate block is set as appropriate according to the camera and the environment.

Returning to FIG. 10, the moving object candidate block whose difference square sum is greater than or equal to the threshold among the blocks BC _ij detected in step S2 is labeled (step S3). Labeling can be performed by setting a flag on a block that is a moving object candidate.

  Then, adjacent regions of moving object candidate blocks that are labeled blocks are connected as one region (hereinafter referred to as a block connection region), and it is determined whether or not the size of the block connection region is equal to or larger than a predetermined value ( Step S4).

  In this step S4, it is determined that the block connection area whose size is less than the predetermined value is not an area showing a moving object. That is, when all the sizes of the block connection areas are less than the predetermined value, the moving object detection process is terminated. Here, as the size of the block connection area, the total number of blocks (number of labels) of the moving object candidate blocks included in the area can be used.

  When a plurality of block connection areas having a predetermined size or more are detected in step S4, the largest area among them is selected as a target for detecting a moving object area (step S5). ).

  When labeling is performed in a normal pixel unit, in order to delete fine irregularities and grooves in the labeled area, for example, expansion / contraction processing that connects or deletes, for example, 4 neighborhoods or 8 neighborhoods of the labeled pixels. Is required and the amount of calculation becomes large. On the other hand, in the present embodiment, since labeling is performed in units of blocks, the connectivity of labeled moving object candidate blocks is improved, and the above-described expansion / contraction processing is unnecessary, thereby reducing the time required for labeling and the like. Can do.

  FIG. 13 is a diagram illustrating an example of a labeling result. As shown in FIG. 13, as a result of labeling, block connection areas S1 to S3 obtained by connecting adjacent moving object candidate blocks are detected. For example, the block connection area S1 is 10 blocks, the block connection area S2 is 3 blocks, and the block connection area Is composed of one block, the block connection area Sa is the largest, and is selected as a moving object area as a moving object area (moving object candidate area) in step S5.

  Here, for example, the block connection area Sc is too small to be a moving object and is determined to be noise and is canceled in step S4, and the block connection area Sb is not canceled in step S4. In S5, since it is smaller than the block connection area Sa, for example, it is regarded as another separated moving body and removed from the target.

Returning to FIG. 10, finally, it is determined whether or not the target block connection area is a moving object (step S6). In this determination, for example,
Condition 1: Occupancy ratio of the block connection area to the image is larger than 30% Condition 2: The vertical or horizontal spread of the target is 70% or more of the vertical or horizontal size of the image Condition 3: The number of labels is extremely large 3 If the two conditions are met, the block connection area targeted in step S5 is not detected, and the moving object detection process is terminated. On the other hand, when the block connection area does not satisfy the above conditions, the block connection area is detected as a moving object area.

  Here, in the above condition 1, when the ratio of the block connection area to the image is larger than 30%, for example, it can be assumed that an object moves in front of the camera 20 of the robot apparatus 1, so The object is canceled without being a detection target so that the object is not a tracking target.

  In condition 2, even if the occupation ratio of the block connection area with respect to the image is 30% or less, the horizontal expansion of the block connection area is larger than 70% of the horizontal size of the image. The case where the vertical extent of the region is larger than 70% of the size of the image in the vertical direction and the block connection region is spread in a high region is shown. For example, in the case of the robot device 1 that can be walked as described above, the servo motor or the like vibrates slightly, which is transmitted to the camera 20 or the like, and as a result, the block at the edge portion of the image is detected as a moving object candidate. For example, there are cases where these are connected. Accordingly, such a block connecting area having a large horizontal and vertical spread should be canceled without being detected as a target.

  Furthermore, in the condition 3, for those determined as moving object candidate blocks in step S2, labeling is performed in step S3. If the number of labeled moving object candidate blocks is extremely large, for example, a plurality of moving objects are included. Although the case where it is included in the imaging range is assumed, also in such a case, the robot apparatus 1 determines that it is not appropriate for tracking the block connection area, and cancels it without setting it as a detection target. .

  And the lock adjacent area | region which remained without being canceled is detected as a moving body area | region where a moving body exists. In this way, by labeling and applying to each condition that is not subject to detection, multiple moving objects are separated, noise is removed, and large moving objects that occupy the entire screen or multiple small moving objects are removed from the tracking target. This eliminates erroneous detection due to slight motor vibration.

  The information on the moving object area thus detected is transmitted to the action model library 70 as described above, and the action is determined by the action switching module 71. For example, the moving object area (moving object) is tracked with respect to the tracking module. An action command is output.

  FIG. 14 is a diagram for explaining the detected moving body region. As illustrated in FIG. 14, the image 110 includes block connection regions 111, 112, and 113 that are connected to moving object candidate blocks and are determined to be moving object regions in step S <b> 4. As described above, in these three block adjacent regions 111, 112, and 113, the block connection region 111 that is the largest region is set as a moving object candidate region in step 5, and the moving object is assumed to satisfy the predetermined condition shown in step S6. Is detected.

Here, the tracking module 71 obtains the center of gravity position in the moving body region 111, detects the horizontal position and the vertical position of the center of gravity position, rotates the head unit 4, and the center of gravity position is the captured image of the camera 20. The camera tracking can be started so that it becomes the center of. That is, the tracking module 71 are those in accordance with the gravity center position, it generates a control signal for controlling the actuator 26 _n for rotating the neck or the like, to control the tracking.

  Here, for example, in the robot apparatus 1, when other functions in the middleware layer 40 are not used and only the moving object detection is performed, or when there is a margin in the arithmetic processing, the resource In a state where other motions can be expressed within a range where no conflict occurs, different motions may be expressed depending on the size of the detected moving body region.

  For example, the size of the moving body region and the behavior that appears in response thereto can be set as shown in Table 1 below.

  As shown in Table 1 above, if the occupancy ratio of the block connected region labeled in step S3 and connected in step S4 is less than 6% (level L1), it is suitable as a target in step S4. Canceled as no. Also, if the occupation ratio is greater than 30% (level L6), it is deemed inappropriate as a target and canceled in step S6.

  On the other hand, when the occupation ratio is 6% or more and less than 12% (level L2), the tracking module 62 starts camera tracking of the moving object by the neck. Furthermore, when the moving object region is 12% or more and less than 18% (level L3), in addition to moving object tracking by the neck, for example, the tail 5 shown in FIG. 1 is shaken or the ear is fluttered. The action indicating the indicated gesture is also expressed.

  When the occupation ratio is 18% or more and less than 24% (level L4), in addition to the camera tracking of the moving object by the neck, a sound is output from the speaker 24 or the like as a reaction indicating that the moving object has been detected, The operation of causing the light emitting means such as light to emit light is exhibited.

  Furthermore, when the occupation ratio is 24% or more and less than 30% (level L5), for example, an action such as actually moving in the moving direction of the moving object, approaching the moving object, touching the moving object, or kicking the moving object By expressing, we will try to promote more active interaction. Alternatively, in addition to the camera tracking of the moving body by the neck, the operation expressed by the level L2 and the operation expressed by the level L3 may be performed together, the manner of swinging the tail 5 may be increased, or a plurality of light emitting means may be used. You may make it light a color light sequentially. When moving in the moving direction of the moving object or approaching the moving object, the robot apparatus 1 itself moves, so that it is difficult to detect the moving object. In such a case, the moving object detection may be started again when the robot apparatus 1 stops moving, such as when it approaches the moving object or when it touches the moving object. For example, if the time period from when moving object detection is stopped by an action approaching the moving object until the next time when moving object detection is restarted is short, for example, the moving object is again detected from its own moving speed and the moving speed of the moving object that was detected. The moving object detection may be started by predicting a position where detection is possible.

  In this embodiment, a difference value for each pixel between the current image and the previous image which is the previous frame is obtained, and this is subjected to threshold processing in units of blocks, and a flag is attached to a block which is equal to or greater than a predetermined threshold. And label. Then, among the labeled moving object candidate blocks, the adjacent blocks are connected to each other as a block connection region, and only a predetermined size is a target to be detected as the moving object region. In this way, threshold processing is performed in units of blocks and labeling is performed, so that there are fewer targets for threshold judgment than in the case of performing processing in units of pixels, and expansion / contraction is not required during connection processing, reducing the amount of computation. can do.

  Furthermore, when there are a plurality of target block connection areas, the largest one is selected, and the selected target is smaller than the predetermined size and does not spread over the entire screen, and the moving object determined earlier. Only when the number of candidate blocks is equal to or less than the predetermined number, the target is detected as a moving object region, so that only a suitable moving object as a tracking target can be accurately detected. Furthermore, when threshold processing is performed in units of blocks as described above, even if the sensitivity is increased by lowering the threshold value, there are too many labeled blocks, the target area is too large, or the target area is too large. In such a case, since a process for canceling this is included, even a slight movement can be detected, and only a moving object region suitable for tracking can be detected accurately.

  Furthermore, according to the detection result of the robot apparatus, not only the moving object is tracked by the camera but also different movements are expressed together, so that the moving object moving near the robot apparatus 1 is more natural. And lifelike interactions are possible.

It is a perspective view which shows the external appearance structure of the robot apparatus in embodiment of this invention. It is a block diagram which shows the circuit structure of the robot apparatus. 3 is a block diagram showing a software configuration of the robot apparatus. FIG. It is a block diagram which shows the structure of the middleware layer in the software structure of the robot apparatus. It is a block diagram which shows the structure of the application layer in the software structure of the robot apparatus. It is a block diagram which shows the structure of the action model library of the application layer. It is the figure used in order to demonstrate the finite probability automaton used as the information for the action determination of the robot apparatus. It is a figure which shows the state transition table prepared for each node of a finite probability automaton. It is a block diagram which shows only the principal part in connection with the moving body detection process in the robot apparatus. It is a flowchart which shows the moving body detection method in embodiment of this invention. It is a figure for demonstrating the method to detect a moving body candidate block in the moving body detection process. It is a figure explaining the method of calculating a difference for every block in the moving body detection process. It is a figure which shows an example of the result labeled as a moving body candidate block after dividing into blocks in the moving body detection process. It is a figure for demonstrating the moving body area | region detected in the moving body detection process.

Explanation of symbols

1 robot device, 10 CPU, 11 DRAM, 14 signal processing unit, 20 CCD _camera, 26 1 ~ 26 _n actuators, 30 device driver layer, 32 robotic server object 33 virtual robot apparatus 40 middleware -Layer, 41 Application layer, 57 Motion detection object, 62 Tracking module, 68 Output semantic converter module, 70 Behavior model library, 71 Behavior switching module

Claims (20)

A block difference value calculating means for calculating a block difference value obtained for each block composed of a plurality of pixels, the difference between adjacent images taken on the time axis imaged by the imaging means;
A moving object detecting device comprising: a block having a block difference value equal to or greater than a predetermined threshold value as a moving object candidate block; and a moving object detecting unit that detects a block connection area formed by adjacent moving object candidate blocks as a moving object area . The moving object detection means determines whether or not the number of blocks included in the block connection area is equal to or greater than a predetermined threshold, and sets the block connection area that is equal to or greater than the predetermined threshold as the moving object area. The moving object detection apparatus according to claim 1. The said moving body detection means detects the largest area | region as said moving body area | region, when there are two or more said block connection area | regions. The moving body detection apparatus of Claim 1 characterized by the above-mentioned. 2. The moving object according to claim 1, wherein the moving object detection unit ends the moving object detection without setting the moving object area as a detection target when the size of the moving object area in the image is equal to or larger than a predetermined threshold. Detection device. The moving object detection means ends the moving object detection without setting the moving object region as a detection target when the horizontal or vertical size of the moving object region is equal to or larger than a predetermined threshold value. Moving object detection device. The moving object detection device according to claim 1, wherein the moving object detection unit ends moving object detection without setting the moving object region as a detection target when the number of moving object candidate blocks is equal to or greater than a predetermined threshold. A block difference value calculating step for calculating a block difference value obtained for each block composed of a plurality of pixels, the difference between adjacent images taken on the time axis imaged by the imaging means;
A moving object detection method comprising: detecting a block having a block difference value equal to or greater than a predetermined threshold as a moving object candidate block, and detecting a block connection area formed by adjoining the moving object candidate blocks as a moving object area. . In the moving object detection step, it is determined whether or not the number of blocks included in the block connection area is equal to or greater than a predetermined threshold, and the block connection area that is equal to or greater than the predetermined threshold is defined as the moving object area. The moving object detection method according to claim 7. 8. The moving object detection method according to claim 7, wherein in the moving object detection step, when there are a plurality of the block connection areas, the largest area is detected as the moving object area. 8. The moving object according to claim 7, wherein in the moving object detection step, when the size of the moving object region in the image is equal to or greater than a predetermined threshold, the moving object region is not detected and the moving object detection is terminated. Detection method. 8. The moving object detection step ends the moving object detection without setting the moving object region as a detection target when the horizontal or vertical size of the moving object region is equal to or greater than a predetermined threshold value. Motion detection method. 8. The moving object detection method according to claim 7, wherein, in the moving object detection step, when the number of moving object candidate blocks is equal to or greater than a predetermined threshold, the moving object detection is terminated without setting the moving object region as a detection target. In a robotic device that autonomously determines and expresses actions,
An imaging means for capturing an image;
Moving object detection means for detecting a moving object based on a difference between adjacent images taken on the time axis imaged by the imaging means;
The moving object detecting means includes a block difference value calculating means for calculating a block difference value obtained by calculating a difference between the images for each block including a plurality of pixels, and a block having the block difference value equal to or larger than a predetermined threshold. A robot apparatus comprising: a moving object detection unit configured to detect a block connection area formed by adjoining moving object candidate blocks as a moving object area. The moving object detection means determines whether or not the number of blocks included in the block connection area is equal to or greater than a predetermined threshold, and sets the block connection area that is equal to or greater than the predetermined threshold as the moving object area. The robot apparatus according to claim 13. The robot apparatus according to claim 13, wherein the moving object detecting unit detects the largest area as the moving object area when there are a plurality of the block connection areas. The robot according to claim 13, wherein the moving object detection unit ends the moving object detection without setting the moving object area as a detection target when the size of the moving object area in the image is equal to or larger than a predetermined threshold. apparatus. The moving object detection means ends the moving object detection without setting the moving object region as a detection target when the horizontal or vertical size of the moving object region is equal to or greater than a predetermined threshold value. Robotic device. The robot apparatus according to claim 13, wherein the moving object detecting unit ends moving object detection without setting the moving object region as a detection target when the number of moving object candidate blocks is equal to or greater than a predetermined threshold. The robot apparatus according to claim 13, further comprising a control unit that reflects a detection result of the moving object detection unit in the action. The robot apparatus according to claim 19, wherein the control unit develops different behaviors according to the detected size of the moving body region.

Images