EP1584175A2

EP1584175A2 - Device for remote video surveillance by portable telephone, operating method, use and remote surveillance network

Info

Publication number: EP1584175A2
Application number: EP04700470A
Authority: EP
Inventors: Patrick Pirim
Original assignee: Holding BEV SA
Current assignee: Holding BEV SA
Priority date: 2003-01-08
Filing date: 2004-01-07
Publication date: 2005-10-12
Also published as: FR2849738B1; WO2004066590A2; FR2849738A1; WO2004066590A3; WO2004066590A8; JP2006516077A

Abstract

The invention concerns a device for remote video surveillance by portable telephone (1, 110), the telephone comprising a transmitter (2) and a receiver (4) under the control of a control member (8) for connection to a telephone network (120), the telephone being capable of receiving and transmitting audio signals (BFm) and video signals (Vc) respectively from a microphone (3) producing the audio signals (BFm) and a camera (6) producing the video signals (Vc) of video images (Ve) and of receiving and restoring audio signals (BFe) and video signals (Ve) respectively to a sound reproduction means (5) and a video reproduction screen (7), the telephone including a user input interface (9) and a self-contained power supply source (10). The invention is characterized in that the telephone further comprises a module (20) for analyzing and diagnosing at least video images, an input of said module (20) being connected in output of the camera (6) and receiving the video signals (Vc), the module being connected at least in output to the control member (8) for sending at least one diagnosis data (D) to said control member (8) based on the analysis and the diagnosis of the video signal by a computing module (23, 20') programmed (25) with an applicative programme of said module, the diagnosis data (D) being instructions acting on the control member. The invention also concerns a remote surveillance operating method and application.

Description

Video remote monitoring device by mobile phone, operating method, application and remote monitoring network

The present invention relates to a video remote monitoring device by mobile phone as well as an operating method, applications as well as a public remote monitoring network. It allows remote monitoring with autonomous decision making based on a video environment captured by the mobile phone. It can be implemented both in professional fields, for example security of premises, road and in particular climatic or traffic conditions (vehicle speed, crossing of traffic lights), as in private fields, for example surveillance of a infants, monitoring of a swimming pool or others with a private individual.

With the evolution of technologies, video surveillance is entering more and more areas. The multiplicity of devices for each application leads to an increasingly sophisticated network mechanism. A video surveillance device comprises, a camera, a means of analysis and diagnosis of video images depending on the application, and a means of communicating the alarms and programming the device.

At the same time, the telephone is becoming more and more perfected, after having durably fulfilled its primary function of a remote sound link, it has become mobile and today integrates visual communication by shooting by camera and viewing on a flat screen.

Telephone operators are constantly looking for new uses for the mobile phone and it becomes a video game companion that from time to time an SMS message disrupts and sound communication now comes to the fore.

Thus in application EP0 884 905 concerning a method of producing an image to to transmit by a terminal and terminal, it has been proposed to use a telephone with camera comprising a shape recognition means making it possible to locate the face of the user for transmission. In application EP1 170 953 concerning a mobile telephone, remote monitoring system, portable information terminal and method of use, remote monitoring has been proposed by transmission of images as a function of an authorization signal. transmission preventing access of the image to third parties. These methods and devices are however dependent on the user for their operation and the transmission of images.

The present invention aims to create a video surveillance device from a telephone having means of visual communication by integrating an image analysis module having its own diagnostic and dialogue processing with the input of the module, the reading of the video signal from the camera and, at the output, a link on the telephone control unit to enable activation / deactivation and autonomous operation of said telephone.

Furthermore, methods and devices are known for locating objects in an image. In some of them it is proposed the statistical analysis of the points or pixels of a digital video signal coming from an observation system for the realization of efficient devices capable of operating in real time.

More recently, it has been proposed to make these devices by the association of information processing units of the same kind each addressing a particular parameter extracted from the video signal to analyze it. This is the case of the patent applications of Mr. Patrick Pirim, inventor: FR96 / 09420 (identification of a moving area), FR96 / 09420 and FR98 / 00378 (vigilance monitoring of a driver), FR98 / 16679 and WO99 / 00425 (compression system), WO98 / 05383 (image processing), FR00 / 02355 (automatic perception), FR01 / 02539 filed on February 23, 2001 (location of an object), FR01 / 02530 (adaptation), FR01 / 1 0750 (processing system), FR02 / 10067 (visual perception), FR02 / 1 0066 (analysis), FR02 / 10065 (functional unit), FR02 / 1 0064 (dynamic recruitment) as well as their extensions, in particular WO98 / 05002, WO98 / 05383, WO99 / 36894, WO99 / 36893, WO00 / 1 161 0, WO00 / 1 1 609, WO00 / 1 3420, WO01 / 63557 or the article "The mechanism of vision integrates on a chip" in Electronics, June 2000 N ° 104 in which we propose the implementation of blocks or units or modules, these terms being here equivalent, of calculation and processing of histogram constituting real space-time electronic neurons, called STN, each analyzing a parameter, said parameter being processed by a function (f ₀ g) to individually produce an output value. These output values, all together, form a retro-annotation available on a bus for use in blocks during analysis. At the same time, each of these histogram calculation and processing units constitutes and updates an analysis output register providing statistical information on the corresponding parameter. The choice of the parameter analyzed by each calculation block and histogram processing, the content of the analysis output register as well as the function (f ₀ g) that it fulfills, are determined by software executed in a programming interface. application API (Application Program Interface).

It also explains that each STN unit is likely to have automatic classification, anticipation and learning functions.

In a histogram calculation and processing unit as described in WO01 / 63557, for a given parameter, the maximum RMAX of from the histogram calculated and stored in a memory is determined. the histogram, the position of said maximum POSRMAX, a number of NBPTS points in the histogram. Classification limits are also determined which make it possible to delimit an area of interest for the parameter and it has been proposed to take as a criterion for determining the limits, a ratio of the maximum of the histogram, for example RMAX / 2, and d '' get the limits by scanning the memory data from the origin in search of the zone limits corresponding to the criterion. An application of the STN blocks is more particularly detailed in application FR-01/02539 where it has been proposed to hierarchically decompose the object to be identified according to its properties which allows, for example, to first determine the general outline of a moving object with respect to a relatively stable background, then search within this outline for characteristic elements by their tint, their color, their relative position ... Such an analysis allows the rapid development of applications multiples involving the location of an object. These applications can be developed, either from an earlier formalization that has identified the significant characteristics of the object, or, thanks to a learning function by examining a scene in which the object in question is present. , the device allowing itself to extract parameters characteristic of the object. In the following, the term element or object will be used without distinction for what is to be identified in a scene of the environment.

The invention proposes to incorporate into a portable telephone having sensors of an environment, in particular of an environment of sounds and images, a module allowing an analysis of the signals coming from the sensors and carrying out a diagnosis in order to get a decision. This decision-making may be an absence of action, in particular in the event of a negative diagnosis - or one or more actions in the event of a positive diagnosis (for example recognition and exceeding of parameters threshold of an element of a scene). Analysis consists in extracting and measuring parameters of significant elements of the environment. For example, if the analysis must relate to a face: recognition of the face within the scene then in the face recognition of the eyes, of the mouth then, possibly, inside these elements of the face, determine significant sub-elements: eyelids, pupil or lips ... The diagnosis consists in determining or detecting the existence of particular elements of a scene and / or the exceeding of thresholds of parameters linked to an element of the scene. In the context of the invention, the term scene covers everything captured by a sensor, for a camera it is a series of images, for a microphone these are sounds, for a temperature sensor a temperature measurement ... and the corresponding element can for example be a moving object / human in the images, sounds of a particular frequency range and / or duration and / or intensity of sounds, temperature as such . The invention therefore relates to a video remote monitoring device by mobile telephone, the telephone comprising a transmitter and a receiver under the control of a control device for connection to a telephone network, the telephone comprising means for picking up BFm sound signals and video signals Vc by means of a microphone delivering the sound signals BFm and a camera delivering the video signals Vc of video images and, on the other hand, receiving and reproducing sound signals BFe and video signals Ve at destination a sound reproduction means and a video reproduction screen respectively, the telephone comprising a user input interface and an autonomous power source.

According to the invention, the telephone also comprises a module for analyzing and diagnosing at least video images provided with a memory for storing at least one remote monitoring application program, an input of said module being connected to the output of the camera and receiving the video signals Vc, the module being connected by at least one output to the control unit for sending at least one diagnostic data item D to said control member as a function of the analysis and diagnosis of the video signal by a calculation means programmed by an application program of said module, the diagnostic data D being instructions acting on the control member. In the context of the invention, the term video for the analysis module concerns both still or moving images as well as sounds and more generally concerns what is generally called multimedia.

In various embodiments of the invention, the following means, taken individually or in combination, are used:

- the camera is a camcorder,

- the camera is of the CCD (charge transfer) or CMOS type, - the program memory of the analysis and diagnostic module is preloaded by the application program (the program is originally available)

the application program is downloaded into the program memory of the analysis and diagnostic module, the application program comprises at least one telephone number on the network,

the application program comprises at least one computer address for calling on the network,

- the analysis and diagnostic module also receives the audio signals BFm as input and the module produces diagnostic data D also as a function of the analysis and diagnosis of the audio signals,

- the instructions acting on the control body are one or more actions chosen without limitation from the following list: organ wake-up on phone standby, phone organ standby, calling a number on the network, transmission of recorded sounds, transmission of live sounds, transmission of a short message (in particular SMS), transmission of a live video sequence, transmission of a recorded video sequence, transmission of a video analysis, transmission of a video diagnosis, multimedia compatible data transmission for the INTERNET network, dialogue between the telephone and the number called, in the case of using a positioning device, control of said device by positioning instructions from the network, end of call,

the analysis and diagnosis module includes a reprogrammable memory for the program of the calculation means,

the analysis and diagnostic module also receives digital data D from the control unit and the program of the calculation means comes from the network and is transferred to the reprogrammable memory in the form of digital data D,

the analysis and diagnostic module also receives digital data D from the control body, the telephone further comprising a smart card reader readable by the control body and the program of the calculation means comes from the smart card and is transferred to the reprogrammable memory in the form of digital data D,

the analysis and diagnostic module comprises a module for acquiring and storing video images and a sound detection means for interfacing with the video and sound signals, a means of calculation of the microprocessor type or digital signal processor, memory for the application program and a dialog unit for interfacing with the control unit,

the analysis and diagnosis module comprises at least one spatial and temporal processing unit and a set of histogram calculation units whose hardware and functional configuration is dependent on the application program,

- the analysis and diagnostic module is an IP block that can be integrated (materially and / or functionally) into an electronic circuit,

- the electronic circuit is an electronic component,

the electronic component is an integrated circuit comprising one / more telephone functions (component with one / more usual functions in telephony: for example transmission, reception, signal processing, signaling, etc.) of the telephone,

the telephone is placed on a positioning device making it possible at least to orient said telephone in space according to positioning instructions (OM) originating from the telephone, said positioning instructions being produced by the analysis and diagnostic module,

the positioning device comprises a first support in which the telephone is placed, the first support itself being disposed in a second support, the first and the second support being articulated with one another by at least one motor controlled by a control circuit receiving positioning instructions (OM),

the second support comprises two motors, the two rotors of which are arranged along two substantially perpendicular axes, the control circuit and a charger intended for recharging the autonomous power source of the telephone, the positioning device of said telephone also makes it possible to move said telephone in space according to positioning instructions (OM),

- in addition, positioning instructions come from the network,

- the positioning instructions (OM) are transmitted to the peripheral positioning device via a telephone connector comprising a serial data transmission link, - the telephone comprises one or more of the following sensors, the output of which is connected to the analysis and diagnostic module, said module analyzing and diagnosing the measurements of said sensors:

- angular position sensor, - acceleration sensor,

- speed sensor,

- temperature sensor,

- light sensor,

- barometric pressure sensor, (possible use in intrusion alarm: the opening of a door causing a pressure wave in a room),

- biometric sensor (fingerprint, DNA ...),

- the telephone sensors are connected to the telephone via a telephone connector comprising a serial data transmission link.

The invention therefore also relates to a positioning device compatible with the telephone of the invention and allowing at least to orient said telephone in space according to positioning instructions (OM) originating from the telephone, said positioning instructions being produced by the analysis and diagnostic module.

The invention also relates to an operating method which results from the implementation of the device as described above according to all of its possibilities and which is a method of operating a video telesurveillance device by mobile telephone, the telephone comprising a transmitter and a receiver under the control of a control device for connection to a telephone network, the telephone being able, on the one hand , receive and transmit BFm sound signals and Vc video signals respectively from a microphone producing BFm sound signals and a camera producing Vc video signals from video images and, on the other hand, receiving and reproducing signals BFe sound signals and Ve video signals intended respectively for a sound reproduction means and a video reproduction screen, the telephone comprising a user input interface and an autonomous power source, in which images from the camera are transmitted to a receiving device connected to the network and capable of viewing the video images, in which a device according to any one is implemented previous characteristics isolated or in combination and in which the telephone further comprises a module for analyzing and diagnosing at least video images, provided with a memory for storing a remote monitoring application program, an input of said module being connected to the output of the camera and receiving the video signals Vc, the module being connected by at least one output to the control member for sending at least one diagnostic datum D to said control member as a function of a decision resulting from the analysis and diagnosis of the video signal by a calculation means programmed by an application program of said module, the diagnostic data D being instructions acting on the control member and in that the connection to the receiving device results from the decision produced by the analysis and diagnostic module.

In variants of the method, the receiver is a portable or wired telephone and / or the receiver is a computer means capable of viewing compatible data. multimedia for network I NTERNET, a server connected to the network and arranged between the telephone and the receiver ensuring the interface between the telephone's data and the INTERNET network. The invention also relates to a particular application of the preceding operating method in which an infant is monitored remotely, the application program allowing at least the analysis and diagnosis of infant movements and the connection to the receiver when movements are diagnosed.

The invention finally relates to a public remote monitoring network which is constituted by a network of portable telephones comprising a central transmission, by this network, of remote monitoring application programs for video remote monitoring device by telephone, programs capable of being downloaded by them. this.

The mobile phone of the invention therefore includes autonomous means of decision making based on the analysis and diagnosis of an environmental context (images, sounds or other) captured by the telephone's sensors and can therefore on its own initiative decide to alert compatible equipment (conventional or specialized telephone, for example the equipment used by firefighters, the police and central surveillance or IT) remotely: decision-making is automatic. In addition, the analysis and the diagnosis being dependent on an application program, the analysis and diagnostic parameters can be adapted to the envisaged application and, preferably, by reprogrammable means and in particular by transmission over the network. by telephone from the program to the telephone of the invention or by inserting into the telephone a smart card containing the program which is then transferred to the analysis and diagnostic module of the telephone. By this decision upstream, at the source of the environment, the downstream user is relieved of monitoring tasks and, moreover, the network is freed, the connection and transmission being made only according to the result of 'a diagnosis.

The use of this process trivializes the video surveillance market, and the notion of associated services becomes predominant. In addition to the fact that the mobile phone only uses the network for the transmission of digital data, diagnostic data (alarm and type of alarm for example) and / or video data depending on the analysis and diagnosis of a video environment, typically application programs preloaded or downloaded, from the telegram transmission center, which reduces the influence of telemonitoring on the network from the point of view of bandwidth occupation, fine analysis of the image by an elaborate visual perception can also make it possible to achieve compression of the video information to be transmitted. Finally, it should be noted that, within the framework of the invention, the mobile telephone can as well be intended for a public network as for a private network, for example within a company or a fixed or mobile building. Furthermore, it is preferable to consider the implementation of a portable telephone because it, due to its development, is now relatively low cost and its modification to adapt an analysis and diagnostic module n 'will result in a moderate additional cost. However, the invention is more generally applicable to means of radio communication of sounds, video (images) and network data specifically developed for the occasion.

The present invention will now be exemplified in relation to: FIG. 1 which represents a video portable telephone of the state of the art, Figure 2 which represents a mobile telephone according to the invention with an analysis and diagnostic module, Figure 3 which represents an analysis and diagnosis module produced with conventional calculation means, Figure 4 which represents a module of analysis and diagnosis carried out with means of calculation with type of spatio-temporal neurons, Figure 5 which represents an example of application of the telephone of the invention, Figure 6 which represents the schematic material structure of a second example of application of the telephone of the invention, and Figure 7 which represents a telephone in this second example of application of the invention. FIG. 1 represents a simplified block diagram of a portable telephone 1 integrating visual communication, corresponding to the prior art. This telephone comprises a basic core with a transmitter 2 and a receiver 4 under the control of a control member 8. In addition, in order to allow interaction with the user, control interface means 9 at input to type of keyboard and / or voice recognition are connected by an electronic bus Te to the control member 8. On the basic core are connected at the input, a microphone 3 sending a low frequency speech signal BFm and a video signal Vc coming from a camera 6 on phone 1. On the basic core are also connected at output, headphones 5 or loudspeaker for sound diffusion of a BFe signal coming from the receiver 4, as well as a video screen 7 which can reproduce both received video data Ve, stored in the phone or captured by the phone's camera 6. In addition to the video reproduction role of the video screen 7, this, as an output control interface means, can also allow interaction with the user by displaying menus or the like. It is also possible in some versions that the sound reproduction means, in particular headphones 5, form an output control interface means by broadcasting instructions or sound menus. The telephone 1 being portable, an autonomous power source is incorporated in said telephone. The functional representation of the mobile telephone in FIG. 1 being very schematized for reasons of simplification, we have not shown in detail all the organs of the telephone such as for example the hardware security means (smart card) and software (programs encryption / decryption) or network and communication management (network search, network connection programs ...).

Figure 2 shows how the invention applies in an extremely simple way to a phone of the prior art. In fact, the invention consists in the main lines of incorporating into a portable telephone of the type which has been described in relation to FIG. 1, a video analysis and diagnostic module 20. This module receives as input at least the signal Vc video from camera 6 of cell phone 1. In an advanced version, the module 20 can also receive the speech signal BFm coming from the microphone 3. At output the module 20 is connected by a digital link D to the control member 8 in order to transfer diagnostic data to it. It will be seen later that the module 20 can also take over certain functions of the control unit and, for example, the compression of data, in particular video. It can be noted that the link D is indicated bidirectional and it will be seen that the module 20 can also receive digital data coming from the control unit and, for example programming data, in particular the application program, for the analysis. and the diagnosis, notably from the network and received by the mobile telephone 1. Indeed, the invention can be implemented at within a network structure in which a supervisor / controller operator provides means of communication (including gateways between computer and telephone networks) and management (in particular transmission of application programs allowing video / audio analysis and diagnosis in the telephone module, the module programs being downloaded) between users of the telemonitoring device.

In other modes of implementation, the analysis and diagnosis module is an additional entity protected in particular by copyright ("copyright") and / or by semiconductor topography and / or and / or patent law and / or other sui generis rights, which is qualified as an IP block (or virtual component which represents knowledge related to the design of functions that can be integrated into integrated circuits), and which is integrated within a component with conventional telephone functions: the entity is added in a component or circuit conventionally used in telephony (functionally by adding a program and / or materially by adding a functional hardware block of logic and / or mixed doors by configuration or reconfiguration programmable logic gate structure or combination of program and hardware). IP blocks are functional entities now widely used for the production of integrated circuits and they correspond to hardware objects (door structures for example) or software objects (instructions) or a combination of both. IP blocks allow the creation of function libraries from which integrated circuit designers draw for the creation of specific circuits. To use the previous example, an IP block corresponding to the analysis and diagnostic module is added to a set of IP blocks intended for telephony to produce an integrated circuit intended for producing a telephone according to the invention. Preferably, the analysis and diagnostic module is a single integrated circuit, possibly associated with external memory, which is arranged in a telephone with conventional video means. The addition of such an integrated circuit can be carried out in a very simple manner with a minimum of modification of the existing circuits of the telephone. In particular, the connection of the module to the control member can be carried out by connection to a communication bus of the control member which is generally implemented within mobile telephones for the control of peripheral organs (transmitter, receiver , keyboard, screen ...).

FIG. 3 represents the organization of the image analysis and sound detection module 20 according to a first embodiment using computer calculation means, conventional computer, in particular digital signal processor (DSP) or microprocessor. The video signal Vc from the camera 6 enters the video image acquisition and storage subset 21. The electrical sound signal BFm from the microphone 3 enters the sound detection sub-assembly 22. A computer 23 executes the application program (s) 25 which makes it possible to analyze and perform a diagnosis on the image and the sound for a predefined time period, or even time slot. This (or these) application program (s) may (may) already be stored in the phone and / or preloaded (s) and / or downloaded from the network. Depending on the diagnosis resulting from the analysis, and in particular in the event of non-compliance with diagnostic criteria, a digital signal D compatible with the control member 8 is generated by a dialogue unit 24. This digital signal D, which corresponds to actions in particular on the network connection means of the telephone, can be the transmission to a predefined number on the network of an appropriate message (SMS) with display of the scene recorded and / or in direct. The means used in this embodiment of Figure 3 are particularly greedy in electrical energy and preferably one uses the means of analysis and diagnosis in relation to Figure 4 with space-time neurons (STN) which are particularly adapted to these functions and particularly flexible to use (reorganization and reuse of neurons according to the depth of analysis and / or application), and this for a relatively reduced energy cost. Said means can also be used for video compression purposes as we have seen in the presentation of the state of the art in the first part of the description.

FIG. 4 represents a preferred embodiment with space-time neurons (STN) comprising, by way of example, a set of sixteen polyvalent histogram calculation units or modules (or space-time neurons, these terms being equivalent) in the form a generic visual perception processor (GVPP) 20 'which forms the analysis and diagnostic module with space-time neurons. This processor further comprises a sound detection means 22 acting directly on an electronic processor 23 itself in relation to an interface or dialogue unit 24 by a bidirectional bus D to and from the control member 8. These calculation units of histogram constitute a matrix, they are connected to a bus 56 on which the parameters DMVT (direction of movement), VMVT (speed of movement), L (light), T (hue), S (saturation), pO, p1 , p2, ..., p15 are available (pO, p1, p2, ..., p15 are slopes of reference axes produced by unit 54). Bus 57 carries the retro-annotation information. Such a processor is described in particular in the following publications FR-2.61 1 .063 and WO-98/05002.

This perception processor 20 'implements a spatial and temporal processing unit 53 which, receiving a video signal, produces a number of parameters for each pixel. These are for example speed (VMVT), direction (DMVT), a time constant and a validation binary parameter in addition to a delayed video signal and the various frame synchronization signals , line and pixel. In such a device, we have already underlined the interest of constituting histograms of these parameters allowing the constitution, manipulation and exploitation of statistical information. The purpose of this type of image processing device is to output a signal which carries for each pixel significant information of the result of the application of recognition or selection criteria according to classification. These criteria are predefined or developed by the image processing methods and devices themselves. In this same document WO-98/05002, a set of STN calculation modules making it possible to calculate histograms is described in relation to the spatial and temporal processing unit 53. Each STN module 51 _a oo •• - 51 _a33 receives data by a data bus 56 and is interconnected to the others by a retro-annotation bus 57 conveying results after classification. API software (application programming interface) makes it possible to configure the modules and the operating parameters of the perception processor with one another via a hardware and functional configuration bus 55. This API software, as well as the application program 25, is managed by the electronic processor 23 which is in relation to all the STN modules by the bus 55 which also transmits in return the results of the calculations of the STN modules. The entire GVPP processor is therefore controlled by the electronic processor or control unit 23, of the microcontroller type, which determines, as a function of an application program 25, those of the parameters DMVT, VMVT, L, T, S, pO, p 1, p2 p 15 which are processed at a given time by a STN unit or a group of dedicated multipurpose histogram units and as a function of initialization (INIT), write (WRITE) and end (END) signals produced by a sequencer 52. Each histogram calculation unit is connected to a data bus 56 which supplies the various parameters to be processed, and to a retro-annotation bus 57 which supplies the classification signal of each STN neuron and the learning function signals to the various other STN neurons. Each histogram calculation unit includes a memory, a classifier and a retro-annotation unit.

Each unit 1 a is capable of having automatic classification functions of anticipation and learning.

The generic visual perception processor (GVPP) 520 thus formed can be integrated on a single semiconductor substrate. The number of versatile histogram calculation units depends on the application and the technologies for manufacturing semiconductor components available.

We have therefore seen that it was possible to implement the invention either by essentially using software means operating in generic computing circuits of the central unit or digital signal processor (DSP) type such as that used in relation to the Figure. 3, either by essentially using hardware means with dedicated circuit (s) in wired logic or in adaptable / programmable logic such as the GVPP circuit presented in relation to FIG. 4 with its STN modules. It is understood that such a distinction of means is not absolute since software means need hardware means to operate and that the adaptation / programming of dedicated circuits can call on software. Thus, even in the case of the dedicated circuit of FIG. 4, an electronic processor 23 has been implemented which makes it possible to supervise the structural and functional operation of said dedicated circuit. In addition, these techniques can be combined, part of the analysis processing can use a dedicated circuit and another to a generic central unit. The means implemented by the invention are therefore both hardware and software, the latter term being able to be taken in the sense of program but also of instructions for structuring of material means as in the case of instructions for configuring circuits. logic with programmable operational structure (networks of programmable logic gates, in particular VHDL - high density logic gate circuits - within ASIC).

In these various possible embodiments, the means used allow continuous sequencing

(real time) images of a scene from the sensor

(camera, CCD, CMOS) with sub-sequencing of each of the images received, this temporally and spatially. Each pixel (point of the image) is known by its value P as a function of the luminance and / or chrominance and its instantaneous position (i, j, t) where i and j define the spatial position within the image matrix and t sequence or time. The consecutive P values are processed in the time domain (in a time processing unit in the case of a dedicated GVPP circuit) for extraction of parameters related to each of the pixels, said parameters being chosen from the luminance, the hue, the saturation, direction of movement, speed of movement, edge orientation, curvature of edges, etc. and the respective positions i, j of the sequence t are processed spatially (in a spatial processing unit in the case of a dedicated GVPP circuit) in order to determine spatial orientations. The information thus obtained will be analyzed statistically and hierarchically in order to perceive the elements composing the images and to arrive at a diagnosis. Thus, in the case of a dedicated GVPP circuit, a dynamic recruitment process for STN modules will be implemented over the sequences to allow active perception of the elements image, recruitment is done according to objectives and in particular a priori knowledge (a face) or not (generic perceptual analysis) of the type of element that can be found in the image. From the perception, a diagnostic process is implemented in order to take action decisions, in particular on the control organ, such as, for example, the activation of the telephone and the sending of a short message. (SMS) by phone when an element perceived and diagnosed as a face appears in the image. FIG. 5 represents an example of a video surveillance application of an infant 103 who is sleeping with parents who want to be warned at a distance from his alarm clock. Any movement 1 01 of the baby and / or noise picked up by the camera and / or the microphone of the telephone 1 1 0 are analyzed and a diagnosis is carried out using the application program, by the mobile telephone 1 1 0 of the invention which has been posed so that it perceives the scene 1 02 by its camera and its microphone, and transmits 121 an alarm and the scene by means of reception 122 and 123 of the telephone network 120 to a second mobile telephone 130 of the parents and corresponding to the call number requested by the diagnostic part of the application.

The second telephone 1 30 which can be a conventional portable telephone, makes it possible to view the scene by a screen 131 and to listen to the sounds by a sound reproduction means 132. The second telephone 130 can be a fixed telephone with or without video means and, in the latter case, only the baby's alarm and sound will be received. The second telephone 130 can be replaced by computer means of the microcomputer type with wired or even radio link or, even a portable telephone-computer, and in particular by means of the INTERNET computer network which allows the transmission of multimedia data including images. and sounds. In the latter case, an INTERNET server will interface within the network 120 between the telephone of the invention 110 and the computer means replacing the conventional portable telephone 130.

More generally, the telephone of the invention can also be applied to the surveillance of an apartment or a building, of a vehicle for example in the event of fire, theft or other, of a swimming pool within the framework of prevention of drowning thanks to the detection of sounds (falling into water, lapping) and images: moving mass or not on the surface or depth of the liquid.

The telephone of the invention can also find an application in teleconferencing and, for example, being able to automatically center on an interlocutor whose lips move, the GVPP analyzing the faces and parts of the face (eyes, mouth ...) and the diagnosis being programmed to detect the movement of the face part corresponding to the lips according to an anthropomorphic model.

By changing the application program, it is possible to modify the video surveillance function, for example in a vehicle, recognize the owner or the persons authorized to drive said vehicle, the alarm is transmitted to a recipient which can be a security organization in case of non-recognition and it is then recorded and instructions are given accordingly to the vehicle. Automobile traffic monitoring applications on road or motorways are also possible thanks to the means of determining the speed of the perception elements that the GVPP allows: fluidity measurement and automatic alarm or detection of authorized speeding and transmission of an alarm means of repressing road violence.

Depending on the application, the alarm receiving station (1 30) may for example also be an improved telephone of the firefighter or police or central station type. monitoring. Thus, the receiving device can be a specific device.

Other applications are possible as will now be seen in relation to FIG. 6 where the telephone of the invention makes it possible to control, as a function of the video and / or audio analysis and diagnosis, a peripheral positioning device in the space of said phone. The positioning of the telephone can relate to the orientation of the shooting (the telephone being able to turn on itself to orient the camera in a given direction), the telephone then remaining at a given location. The positioning may also relate, in combination or not with the orientation, to the choice of location in the space of the telephone, the latter being able to move in said space. These possibilities depend on the positioning device used, from a simple rotary support that can be controlled along one or two axes, a mobile device moving on a surface (motorized trolley controlled), even in a three-dimensional space (airship, plane or submarine ordered in a reduced model). It is understood that the notion of positioning is relative and that if it is in general the telephone of the invention which is positioned in a fixed space (is oriented or moves in space to obtain a direction and / or origin of different point of view), we consider the case where space moves and we try to keep a direction and / or an origin from a constant point of view. The latter case can, for example, correspond to an automated lookout with a video telephone placed on the front of a ship that can be rolled or pitched and intended to prevent a collision by video obstacle detection: the module analysis and diagnostic is then configured to detect bodies floating forward of the ship towards the horizon.

Adapting the telephone with video means of the invention to such an application is relatively simple because in most cases, the mobile phone has a connector for recharging its power supply and for exchanging data (generally by serial link) between the control unit and external peripherals. FIG. 6 is thus represented, under the general reference 1, a telephone such as that of FIG. 2 with a connector 1 01 for connection for recharging the power supply 1 0 by the line V and for data exchange communication by the OM link shown here unidirectional from the control unit to the outside but which more generally allows bidirectional data exchanges.

The telephone 1 is positioned in a stable manner in a first support 100. This first support 100 is itself disposed in a second support 106 in an angularly orientable manner by controlled positioning means, the second support being assumed to be placed on a fixed location of space in this version. The telephone 1 is therefore fixed relative to the first support 1 00 and the first support 1 00 can be oriented angularly relative to the second support 1 06. Consequently, the telephone and therefore the image shooting can be oriented angularly in space. The support system presented thus makes it possible to maintain, orient according to positioning instructions and, incidentally, power (recharge) the telephone 1.

The controlled positioning means comprise a motor control circuit 103 acting on two motors 104 and 105 by motor control signals CM. One of the motors referenced 1 05 is fixed immobilized by its stator on the second support 106 and its rotor drives the stator of the other motor referenced 104 whose rotor is fixed to the first support 1 00. The line V and the connection OM of the connector 1 01 reach the second support 1 06 through the first support 1 00. The second support 106 includes a charger 102 (to recharge the telephone 1) receiving an external supply, in particular the sector of the public electrical distribution network, and which also allows the supply of the controlled positioning means (control circuit 1 03 and motors 104 and 1 05). The positioning instructions from the control unit 8 of the telephone 1 and acting on the control circuit 103 are conveyed by the link OM. The initial source of the positioning instructions is preferably the analysis and diagnostic module which uses the link D towards the control member (for example followed by a person or a face). It is however possible to transmit by telephone, from the telephone network, such positioning instructions which are then received by the receiver 4 of the telephone 1 and sent by the link OM via the control member 8. In the latter case, the orientation can for example be obtained from another telephone by acting on its keys or, in the case of the use of computer means, in particular INTERNET, action on one or more display cursors (mouse) of a computer terminal on the telephone network with computer gateway.

The line V and the link OM are preferably embodied, in particular by a wired link, or even by or associating one or more rotary contacts (the first and the second support being movable between them). In a more advanced version, the V line and the OM link are dematerialized, for example an electromagnetic induction means allowing recharging of the power supply of the telephone and the OM link being effected by radio waves (for example BLUETOOTH®) , the second support then further comprising compatible radio communication means connected to the motor control circuit 1 03.

Preferably, the rotors of the motors are offset by 90 ° in order to allow efficient scanning of the space of shooting. Preferably, a maximum angular displacement, that is to say 360 °, will be ensured for the plane of the space where the elements of scene images are most likely to be found. For example for a telephone placed on a table in a room, it is preferable to be able to scan for shooting horizontally throughout the room (360 °) where one is more likely to detect a person than towards the ceiling. On the other hand, due to mechanical constraints, in particular the presence of the supports, there are areas where shooting is not possible.

It is understood that the possibilities of angular scanning of the space from the point of view are adapted according to the needs and the cost of implementation. Thus, if two motors have been represented in FIG. 6, which allows a volume scanning of the space, in a simplified version, only one motor can be implemented, preferably, allowing scanning in a plane where the elements of the images are most likely to be found.

The motors are preferably stepper motors. An indexing means (origin position marker) can be implemented either directly on the rotors, or, in a more complex manner, by image analysis. The indexing means makes it possible to know a reference position from which the displacement can be determined (number of steps for stepping motors, which corresponds to a computable angular displacement). The indexing means, in a variant, can return a signal to the telephone 1 by the bidirectional OM link. The positioning interface may also include other sensors such as, for example, angular, speed and / or acceleration, battery level or power supply sensors, the information of which is sent back to the telephone. .

Figure 7 shows the arrangement of the telephone 1 of the invention in the first support 100, itself even in the second support 106. In the rest position, the telephone 1 is substantially vertical and the camera's shooting means has a substantially horizontal shooting axis. Thanks to the motor 105, the first support 100 can rotate relative to the second support 106 allowing the shooting axis to scan the space in a substantially horizontal plane. As for the motor 104, it makes it possible to tilt the first support 1 00 with respect to the second support 106, that is to say to cause the telephone 1 to leave its substantially vertical position for positions approaching the 'horizontal, the shooting axis then moving from bottom to top and vice versa.

As indicated for Figures 6 and 7, it is assumed that the second support is arranged at a fixed location in space. However, in other variants of the invention, the telephone 1 can be made mobile in space by using support (s) which have means of moving in said space. In such cases, it is understood that it is preferable that the external power supply is not permanently connected, unless there is a reduced displacement radius (the length of the external power supply wire) and that a supply means autonomous, in particular with type of rechargeable battery, is implemented in the mobile support of the telephone. In such a version, the simplest with rolling means on the ground, the device in FIGS. 7 and 8 further comprises, on the underside of the second support, rolling and directional means controlled by means of the link OM. In another variant, the telephone 1 can be arranged substantially vertically on a support carriage comprising drive and directional wheels (an additional means of controlled tilting of the telephone which may possibly be implemented). It is understood that the displacement means controlled on the ground (see another reference) are adapted to the envisaged uses, as simple rolling means (wheels, caterpillar) or articulated legs, suction cups ... Likewise if one wishes to move in a three-dimensional space, the means of movement will be adapted to said space. Conventional robotic management tools can be usefully implemented in such applications.

As previously indicated and thanks to the possibility of hardware and functional configuration of the STN modules of the GVPP according to an application program via the API, it is also possible to use the perception processor in as a video image compression system as described in FR98 / 16679 and WO99 / 00425 in order to improve the quality and the fluidity of the images transmitted by the telephone 1, 1 10, a greater quantity of compressed information can be transmitted on the transmission channel. Thus, after diagnosis leading to a positive decision leading to a connection to the network, the perception processor can be reconfigured to operate in video compression. It is well understood that in such a case, the reception means 130 and / or the network server in the case where I NTERNET is used for connection with the remote device 1 30, compatible decoding means will be placed there. In this case, a connection between two telephones of the invention is particularly advantageous since they both comprise means with a perception processor compatible and configurable for compression / decompression.

It is well understood that the invention can be declined in various ways and in particular with regard to the diagnoses as a function of the analyzes which can produce actions of various types: simple alarms, transmission of computer messages, transmission of video data and / or sound, or even multimedia to various equipment of the telephone or even computer type, via a radio-telephone network to which the cell phone of the invention. The applications of the invention are also multiple, both for private or family surveillance with a conventional video portable telephone comprising the analysis and diagnostic module as for professional applications and in this case, the telephone of the invention can be a specific achievement, in particular a militarized mobile phone or in the form of an industrial module. These applications can in particular result from the inventions using space-time neurons and described in the requests whose inventor is Mr. PI RIM and which are in particular listed in the first part of the description.

Claims

1. Video remote monitoring device by mobile telephone (1, 1 1 0), the telephone comprising a transmitter (2) and a receiver (4) under the control of a control device (8) for connection to a telephone network (120) , the telephone comprising means for picking up BFm sound signals and Vc video signals by means of a microphone (3) debiting the BFm sound signals and a camera (6) debiting the video signals Vc of video images and, on the other hand, receiving and reproducing sound signals BFe and video signals Ve intended respectively for a sound reproduction means (5) and a video reproduction screen (7), the telephone comprising a user input interface (9) and an autonomous power source (1 0), characterized in that the telephone further comprises a module (20) for analyzing and diagnosing at least video images provided with a memory for storage at least one remote monitoring application program, u ne input of said module (20) being connected to the output of the camera (6) and receiving the video signals Vc, the module being connected by at least one output to the control member (8) for sending at least one diagnostic data D of said control device (8) as a function of the analysis and diagnosis of the video signal by a calculation means (23, 20 ′) programmed (25) by an application program of said module, the diagnostic data D being instructions acting on the control organ.

2. Device according to claim 1, characterized in that the analysis and diagnostic module additionally receives the BFm sound signals as an input and that the module produces diagnostic data D also as a function of the analysis and diagnosis of audio signals.

3. Device according to claim 1 or 2, characterized in that the instructions acting on the control member are one or more actions chosen, but not limited to, from the following list: waking up organs put into phone standby, putting organs into standby from the phone, calling a number on the network, transmitting recorded sounds, transmitting sounds in direct, transmission of a short message, transmission of a live video sequence, transmission of a recorded video sequence, transmission of a video analysis, transmission of a video diagnosis, multimedia compatible data transmission for network

INTERNET, dialogue between the telephone and the number called, in the case of using a positioning device, control of said device by positioning instructions from the network, end of call.

4. Device according to claim 1, 2 or 3, characterized in that the analysis and diagnosis module includes a reprogrammable memory for the program

(25) of the calculation means (23, 20 ').

5. Device according to claim 4, characterized in that the analysis and diagnostic module also receives digital data D from the control unit and in that the program of the calculation means comes from the network and is transferred to the reprogrammable memory in the form of digital data D.

6. Device according to claim 4 or 5, characterized in that the analysis and diagnostic module also receives digital data D from the control unit, the telephone further comprising a smart card reader readable by the control unit and in that the program of the calculation means comes from the smart card and is transferred in reprogrammable memory as digital data D.

7. Device according to any one of the preceding claims, characterized in that the analysis and diagnosis module comprises a video image acquisition and storage module (21) and a sound detection means (22) for interfaces with video and sound signals, a calculation means (23) of the microprocessor or digital signal processor type, the memory for the application program (25) and a dialogue unit (24) for interfacing with the control (8).

8. Device according to any one of claims 1 to 6, characterized in that the analysis and diagnosis module comprises at least one spatial and temporal processing unit (53) and a set of histogram calculation units (51 _a oo - •• 51 _a 3 ₃ ) whose hardware and functional configuration is dependent on the application program.

9. Device according to any one of the preceding claims, characterized in that the analysis and diagnostic module (20) is an IP block which can be integrated into an electronic circuit.

10. Device according to any one of the preceding claims, characterized in that the telephone (1, 1 10) is arranged on a positioning device (100, 106) allowing at least to orient said telephone in space according to positioning instructions (OM) from the telephone, said positioning instructions being produced by the analysis and diagnostic module (20).

1 1. Device according to claim 10, characterized in that the positioning device comprises a first support (100) in which the telephone is arranged, the first support (100) being itself disposed in a second support (106), the first and the second support being hinged together by at least one motor (104, 105) controlled by a control circuit (103) receiving the positioning instructions (OM).

12. Device according to claim 1 1, characterized in that the second support comprises two motors (104, 105) whose two rotors are arranged along two axes substantially perpendicular to each other, the control circuit (1 03) and a charger intended for recharging the autonomous power source (10) of the telephone.

13. Device according to claim 10, 1 1 or 12, characterized in that the positioning device of said phone further allows to move in space said phone according to positioning instructions (OM).

14. Device according to any one of claims 10 to 132, characterized in that, in addition, positioning instructions come from the network.

15. Device according to any one of claims 10 to 14, characterized in that the positioning instructions (OM) are transmitted to the peripheral positioning device (100, 106) via a telephone connector comprising a link serial data transmission.

16. Method for operating a video remote monitoring device by mobile telephone (1, 1 10), the telephone comprising a transmitter (2) and a receiver (4) under the control of a control device (8) for connection to a telephone network (120), the telephone being able, on the one hand, to receive and transmit BFm sound signals and Vc video signals originating respectively from a microphone (3) producing the BFm sound signals and from a camera ( 6) producing the video signals Vc of video images and, on the other hand, receiving and reproducing sound signals BFe and video signals Ve intended for respectively a sound reproduction means (5) and a video reproduction screen (7), the telephone comprising a user input interface (9) and a stand-alone power source (10), in which images from the camera (6) are transmitted to a receiving device (1 30) connected to the network (120) and capable of viewing the video images, characterized in that a device is implemented according to any one of the previous claims and wherein the telephone further comprises a module (20) for analyzing and diagnosing at least video images provided with a memory for storing at least one remote monitoring application program, an input of said module (20) being connected to the output of the camera (6) and receiving the video signals Vc, the module being connected by at least one output to the control member (8) for sending at least one diagnostic data item D said control unit (8) as a function of a decision resulting from the analysis and diagnosis of the video signal by a calculation means (23, 20 ′) programmed (25) by an application program of said module, the diagnostic data D being instructions acting on the organ control and that the connection to the receiving device results from the decision produced by the analysis and diagnosis module.

17. Method according to claim 16, characterized in that the receiver (130) is a portable or wired telephone.

18. The method of claim 16, characterized in that the receiver is a computer means capable of viewing multimedia compatible data for the INTERNET network, a server connected to the network (120) and disposed between the telephone (1, 1 10) and the receiver (130) providing the interface between the telephone data (1, 1 10) and the INTERNET network.

19. Application of the operating method of any one of claims 16, 17 or 18 in which an infant is monitored remotely, the application program allowing at least the analysis and diagnosis of infant movements and the connection to the receiver when movements are diagnosed.

20. Public remote monitoring network, characterized in that it is constituted by a network of portable telephones comprising a central transmission, by this network, of remote monitoring application programs for video remote monitoring device by telephone according to any one of claims 1 to 15, programs that can be downloaded by them.