WO2017116492A1 - Method for integrating parallel streams of related sensor data generating trial responses without prior knowledge of data meaning or the environment being sensed - Google Patents

Abstract

A method for integrating parallel streams of sensed data across time and between streams without prior knowledge of the environment being sensed or the information in the data streams. This temporal and spatial integration creates abstract data planes that are compared to abstract data planes stored in memory and full or partial matches generate output to the environment. If the output achieves a goal within the environment or meets an internal reference, the data planes associated with that output are reinforced. The buildup of successful output data planes are temporally and spatially integrated to form retrievable abstract output data planes so that small input changes can generate complex output.

Description

Non-PROVISIONAL PATENT APPLICATION TITLE OF INVENTION

Method for Integrating Parallel Streams of Related Sensor Data and Generating Trial Responses without Prior Knowledge of Data Meaning or the Environment being sensed CROSS-REFERENCE TO RELATED APPLICATIONS

Not Applicable

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

Not Applicable

REFERENCE TO SEQUENCE LISTINGS, A TABLE, OR A COMPUTER

PROGRAM LISTING COMPACT DISC APPENDIX

Not Applicable

BACKGROUND OF THE INVENTION Field of the Invention: [0001] The present invention relates generally to methods, processes and structures for processing data in which all output is learned from and validated by an environment. Multiple sensor elements provide steams of encoded data relevant to a specific environment providing the sensed data. Data stream values are scaled to a system wide common range, spatially and temporally integrated and stored as data planes. Data streams from other sensor elements relevant to the same environment are also scaled; and spatially and temporally integrated. System output, associated with each data plane, is initially random until iterative feedback from the environment reinforces the output action. Internal and external inferencing both with approximate match reinforcement also provide output to the environment. This method is unique in that a priori knowledge of the environment being sensed is not required.

Description of the Related Art: [0002] Current Mathematical Science relies on symbolic, heuristic or algorithmic approaches to produce information that affects an environment in a desired manner. These methods, including learning systems, require a priori knowledge of the environment and results to be produced. The creators of systems that integrate sensor data use symbolic, heuristic and algorithmic methods for that integration and for computing values for output. To be effective, such systems require a priori knowledge of the subject matter; and context and value ranges of the sensed data. SUMMARY [0003] In one aspect of the method presented, very large data streams can be spatially integrated through context specific mappings conditioned by the sensed data, and temporally integrated to produce data planes that are meaningful representations of environmental artifacts. Such data planes can be compared to stored data planes and meaningful full and partial matches can be made. This method does not require prior information about the data value context or meaning. [0004] In another aspect, common environmental artifacts in input data can be isolated based on characteristics of that input such as frequency of occurrence or similarity to occurrences of other environmental artifact features. This isolation provides efficient recognition of environmental artifact features through developed mappings that eliminate or filter surrounding common input data and provide spatial integration of input data. [0005] In another aspect, regions of extracted features in input data streams, as defined by the developed mappings can be stored sequentially in a memory. The method can then produce data streams that are the differences between the current input stream and stored data from earlier input. The difference stream becomes a temporally integrated data stream that can be input to other sections where additional spatial and temporal integration can also take place. Temporal integration results in data abstractions that can be used for further operations or additional abstractions. [0006] In another aspect, input data planes, integrated or otherwise, are associated with memory sections that contain output data through data streaming between memory sections or through synchronize memory scanning. Output data is initially random but is conditioned by the environment so that only useful output data is retained for any set of inputs. 76

77 [0007] In another aspect, the input data is scaled, linearly or non-linearly, to a range

78 common to the entire system. Scaling adjustment data becomes part of the input

79 stream.

80

81 [0008] Further areas of applicability of the invention will become apparent from the

82 detailed description provided herein. It should be understood that the detail

83 descriptions and specific examples, while indicating at least one exemplary

84 embodiment of the invention, are intended for purposed of illustration only and are

85 not intended to limit the scope of the invention.

86

87

88 BRIEF DESCRIPTION OF THE DRAWINGS

89

90 [0009] Fig. 1 illustrates sensor elements within a sensor set, a data path for each

91 element, a data plane path, exemplary analog data, exemplary digital data represented

92 as being stored in a sequential memory and a data plane.

93

94 [0010] Fig. 2 illustrates spatial integration of individual data paths. The initial

95 undifferentiated integration is the top view. Data path deletions and extensions

96 created by uncommon sensed data is the bottom view.

97

98 [001 1] Fig. 3 illustrates data elements in memory and the starting locations for

99 searching and storing.

100

101 [0012] Fig. 4 illustrates temporal integration of data elements conveyed in data paths

102 including comparison to memory values and summation of differences.

103

104 [0013] Fig. 5 illustrates typical data flow structures and pathways for sensor data,

105 integration of sensor and other data, output data, and inferencing.

106

107 [0014] Fig. 6 is an abstract representation of the components of temporal integration

108 and is the Illustration of the Patent.

109 110 DETAILED DESCRIPTION

111

112 [0015] The invention is a method that when implemented, provides continuous and

113 parallel data flow from sensors through various integrations and then output. In the

114 environment which is being sensed, feedback from effects caused by the output

115 reinforces that output for the same or similar input. Integrations are created based on

116 data values within the parallel data streams even though the information conveyed by

117 the data is not accessible to the system. The data path structure, the integration

118 structure, and the output data path structure are developed from sensed data and

119 environmental feedback without prior knowledge of the environment nor external

120 goal setting. There are other distinguishing characteristic of the method. The term

121 "epoch" means a short and variable interval of time in this application. There are

122 three section types in this description: Sensor, integration, and output. Each section

123 may have one or more "specialized areas."

124

125 [0016] Fig. lis a block diagram illustrating a sensor, sensor components, data paths

126 and data. Box 100 represents a sensor system made up of individual sensor elements

127 102. The data from the individual elements continuously flow through the data paths

128 104 as illustrated by the horizontal arrows. Ellipses 106 indicates that any sensor

129 may have any number of sensor elements. Sensor data flowing in the data path can

130 be analog as represented by the exemplary waveforms 108. In order to be stored in a

131 memory, the analog data flow will be "chopped" into discrete values as illustrated by

132 the series of vertical lines associated with each exemplary waveform 110. Each set

133 of "chopped" values from the same epoch or interval of time is a "data plane" 112.

134 Data read from memory enters output data paths as analog data or a series of digital

135 values. As a convention in this application, data plane flow is represented by an

136 arrow with three perpendicular marks near the tail of the arrow 1 14.

137

138 [0017] The data value range, both analog and "chopped," is constant throughout the

139 system. Initial scaling occurs in the Sensor section. Rescaling occurs in the

140 integration sections to maintain the data within the common range and insure

141 integration with other data streams is possible. Scaled sensor element values can be

142 linear between the highest and lowest values defining the range; they can also be the

143 integral of the shape of the analog waveform within the epoch or any other 144 distribution between the limits of the scaling range. At any moment, up to 20% of the

145 sensor element values for the same output of the sensor may equal the maximum scale

146 range value.

147

148 [0018] The duration of the waveform being "chopped" is determines by the sensors'

149 recovery time and is specific to the sensor section. Different sensor may have

150 different epochs for "chopping."

151

152 [0019] Fig. 2 View 1 illustrates the data paths that result in spatial integration of the

153 individual data elements. Initially the input data paths from the sensor elements or

154 from another section or specialized area 200 are interconnected in a manner that

155 evenly distributes each data elements value 202 to nearby downstream data paths 204.

156 The number of connections between a single input path and nearby output paths is

157 determined by integrating or distributing instantaneous analog signals across time and

158 distributing fractional amounts of that value to nearby output paths. The effectiveness

159 of spatial integration is contingent upon the correct ratio of number of input paths

160 mapped to output path. The optimal ratio is specific to the implementation. This

161 distribution map generation can also be done with sequential digital data as the input

162 and the distribution of data values done with common distribution methods.

163

164 [0020] Fig. 2 View 2 illustrates a method of pathway definition enhancement that will

165 improve the accuracy of spatial integration. The sensed environment will present

166 common background data and meaningful uncommon foreground data simultaneously

167 to the sensor. Uncommon foreground data has average data values that are higher

168 than typical background data values. The integration of any input analog signal 200

169 containing uncommon foreground values will require more pathways to the output

170 data paths to meet the scaling requirement in the output pathways. Input pathways

171 that contain less uncommon foreground data will lose connected pathways as output

172 paths gain the additional connections 206. This will result in the specialization of a

173 group of output paths based on sensed data 208. Uncommon foreground data will

174 create multiple specialized data paths that, with the associated memory, are called

175 specialized areas. The creation of specialized areas will only occur if the environment

176 being sensed includes uncommon foreground data that are distinctly different.

177 178 [0021] Spatial integration relates data in parallel data streams to each other to

179 establish data context.

180

181 [0022] Fig. 3 illustrates a memory structure. 302 is a set of "chopped" values in

182 memory in the sequence in which they were sensed with the most recent entries at the

183 bottom of the list. Input for memory searching and memory storing is an exemplary

184 data element path 300 but all data paths in a specific section will operate the same and

185 scanning will be synchronized with all other memory store and recall operations in

186 that section. The Recall view shows that memory scanning begins with the least

187 recent data in memory. When data is stored, it is stored at the most recent end of the

188 memory sequence 304.

189

190 [0023] Input matches to memory maybe a single value per data stream or a long

191 sequence of values. Input data streams are continuously compared to memory and re-

192 stored if a match or near match is found.

193

194 [0024] To insure the input streams aligns with the start of the same or similar streams

195 in memory, each input stream value will be compared to the entire memory for that

196 stream until a match is found for most of the data streams as determined by a control

197 path between memory comparators. See Fig. 4. If subsequent input values do not

198 match memory, the control system will start a new memory scan with the unmatched

199 input values for all data streams.

200

201 [0025] In some circumstance within the environment, input streams may not contain

202 uncommon foreground data in which case, memory recall scanning is "free scanning." 203

204 [0026] The system runs continuously except that periodic abeyance must occur during

205 which least used and duplicate memory entries are deleted

206

207 [0027] Fig. 4 illustrates temporal integration that relates data across a short period of

208 time, the "temporal epoch". Input data streams 400 are compared to memory 404 at a

209 comparator 402. The comparator subtracts the two values and if a close match is

210 made (i.e. the difference is zero or near zero), the data value is reentered into memory

211 404, and differences data streams are summed in an accumulator stage 406. The 212 control paths 408 determines if the matching is occurring across most of the data

213 streams and, if so, will allow the accumulation to continue until the upper scaling

214 limit is reached by some of the data streams. When that condition is met, data is

215 streamed to the next section through the individual data paths 410. That data plane

216 contains integrated data that is unique and relevant to the environment or it is the

217 summation of data values that are not consistent matches across the specific memory

218 section. In both cases the data plane will be stored in the next section. Valid data

219 planes will be reinforced as the environment presents similar data. Invalid planes will

220 not be reinforce, and since they are not unique to the environment, will not be recalled

221 in the future. They will accumulate at the distal end of memory and be removed

222 during memory cleanup.

223

224 [0028] Some of parts of the output data planes 410 could contain valid data while

225 other parts would contain pseudo random data and are called associative data planes.

226 The associative data planes will serve as trial data planes for inferencing in the

227 integration section and could produce valid or unique outputs that could be reinforced

228 by the environment. The content of the data planes are a non-symbolic language

229 specific to a data plane path. They are not understandable or decodable elsewhere. 230

231 [0029] The control mechanism, shown only as a double headed arrow 408 and which

232 is a separate set of spatial and temporal integrators within that section, for determining

233 memory matches or partial matches senses the difference values for a sample of

234 comparators in that section. There are three probable conditions: a reasonable match,

235 a partial match, and a regional match. For a reasonable match, all control data values

236 will be close to the same value and the control systems will send a match signal to all

237 comparators in the section when for example only 10% of the comparator values

238 reach the scaling limit. The signal at the comparator will result in the data plane at the

239 comparators being stored in memory and conveyed to the next section. If the data

240 values sensed by the control system increase slowly, the control system will provide

241 the match signal to the comparators when the end of the temporal epoch is reached.

242 The control signal will again cause the temporal integrator to store and convey that

243 resulting data plane which is less likely to be valid. The regional matching occurs

244 when some inputs data streams match better and faster than others. For example,

245 feedback from a temporal integrator from a different section may provide excellent 246 matches while other input to the section provides poor matches. The control

247 systems' own spatial integrator will treat this case as uncommon foreground data and

248 overtime remap its spatial integration giving great distribution to this source of data.

249 This results in the control system giving preference to matches in a particular region

250 of the input sources.

251

252 [0030] Temporal integration relates the recent end of a data stream to earlier data in

253 that stream. The data structure and scaling requirement insures that information is not

254 lost during the integration steps. Temporal integration in a sensor section has a

255 sensor specific temporal epoch that insures that data is not lost due to sensor system

256 limitations. Temporal integration in non-sensor and non-output sections have

257 temporal epochs that are common to insure integration of different sensor sections are

258 consistent in time. Temporal integration in output sections are limited to temporal

259 epochs defined by speeds at which the output mechanisms can accept and act on data.

260 Fig. 6 is an abstract representation of this temporal integration method. Memory 602

261 is compared to an incoming data stream 604 with comparator output presented at the

262 base 600.

263

264 [0031] Temporal integration abstracts data so that many small variations in

265 uncommon foreground data can be represented in single data plane. Further temporal

266 integration provides further abstraction.

267

268 [0032] Fig. 5 illustrates exemplary flow of data planes between sensor sections,

269 inferencing sections and the output sections. In Fig. 5, the lines with arrowheads and

270 three vertical perpendicular marks at the tail represent streams of data planes.

271

272 [0033] Generally, sensor sections provide data to the inferencing sections and the

273 output sections. The inferencing section integrates data from various specialized

274 areas from both sensor sections and other inferencing sections. The output sections

275 receive data from the sensor sections and the inferencing sections and produces

276 outputs that have some effect on the environment. These data paths are specific to

277 the implementation and are shown here as general examples. All inputs to a section

278 504 are passed to each specialized area 502 in that section.

279 280 [0034] The number of spatial integration pathways in any section are specific to the

281 environment and when temporal integration 500 is included, produces specialized

282 areas 502 within that respective section. Data streams from the spatial integrators are

283 temporally integrated 500 and resulting values for that specialized area move as data

284 planes to a next section. Specialized areas may be hierarchical and may change as the

285 sensed environment changes or as system accuracy improves. Minimally a sensor

286 section and an output section are required. The number of sections of any kind are

287 specific to the environment.

288

289 [0035] Exemplary input and outputs in the sensor sections are:

290 [0036] Positive or negative feedback from the environment that is reinforcement of

291 the validity of the current outputs 506, sensor data 508, data related to adjustments

292 required during scaling 510, positional data relative to the sensor and environment

293 512, the measured effort required to achieve that position or overall output effort (if

294 appropriate) 514, and output from one or more specialized areas in that section 516.

295 This last item allows consolidation or abstraction of sensor data planes and improves

296 search results.

297

298 [0037] The sensor section temporal integrators may send data directly to the output

299 section 522 or via a pathways to the integration sections 520 and 524. Such branching

300 only requires that the data values appear on all pathways. Data flow is not switched

301 between pathways.

302

303 [0038] Exemplary inputs and outputs in the inferencing sections are:

304 [0039] Input from other integration sections 530, input from sensor sections as

305 described above, and inputs from specialized areas within the output section 532.

306 Outputs from the specialized areas convey data to other integration sections 526 and

307 output sections 528. This mapping insures there are data path loops within and

308 between sections that allow for inferencing between inputs, integrations and output. 309

310 [0040] Exemplary inputs and outputs in the output section are:

311 [0041] Inputs as described above, data about output actions taken within the

312 environment 540 and the effort required to produce those actions 542. Outputs from 313 the output sections include feedback to integrations sections 532, output instructions

314 534 and 538, and output to other output sections 536.

315

316 System Operations

317

318 [0042] System memory contains random data prior to initial operations. Spatial

319 integration initial mapping is done with random data. The operations of the system is

320 as follows:

321

322 [0043] Sensor data is streamed to the spatial integrators and then the temporal

323 integrators. Overtime the spatial integrator remaps data streams into specialized

324 areas based on sensed data. Data from temporal integrators are conveyed to other

325 sections and specialized areas. An output, initially random, is generated for data

326 planes conveyed to the output sections. The initial memory matches as determined

327 by the controllers is poor and the output is random data values. The environment

328 validates these outputs by direct positive or negative feedback, or as effort required to

329 achieve a result. The latter is either computational effort or output effort. The system

330 can be conditioned through positive or negative feedback to minimize the effort

331 required to achieve the results. As the system gains valid data, desired trial results in

332 the environment other than minimizing effort can be established as goals.

333

334 [0044] The system operates in two modes: iterative recall and output in which the data

335 has already been validate by the environment and built up in memory; and associative

336 recall in which trial outputs are generated that could lead to build up in memory if

337 those outputs are validated by the environment. The trial outputs, generated by the

338 temporal integrators are pseudo random and insures that the system does not become

339 stable to the extent that it cannot generate new outputs.

340

341 [0045] If a data plane is matched or partially matched for a given input, it is copied

342 into memory again, resulting in a buildup of that data plane. In the associate mode:

343 invalid data may buildup in the memory resulting in the system divergence and

344 invalid output trials. During periods of abeyance, system operation is suspended

345 while duplicate and rarely used data planes, those at the distal end of memory, are 346 eliminated. This memory cleanup prevents the buildup of invalid associative data

347 planes and resulting system divergence.

348

349 [0046] With either iterative or associative data planes, the system can infer results

350 through the feedback paths within and between sections without generating an output.

351 This internal inferencing can converge on reinforced memory entries and their

352 associated outputs or can generate new outputs that may or may not be validated by

353 the environment. In all cases, data within the data planes are specific to the sections

354 and details about the environment are not retrievable except though the output

355 sections.

356

357 [0047] The preferred embodiment the system described herein is a contemporary

358 electronic data processors containing a central processing unit, memory, storage, and

359 a communication method for communication with other data processors. In this

360 embodiment, all data, once converted from analog sensor signals to digital values,

361 will be processed as digital data. In this embodiment, data does not flow in a data

362 stream but is passed between memory elements by a computer program which also

363 insures all data values in a data plane are processed as part of that plane.

364

365 [0048] In this preferred embodiment, the spatial integration connections from input to

366 output are established by distributing each input data steam value proportionately to

367 nearby output data paths. Output paths whose average data values over time have

368 exceeded the scaling range for the current mapping do not receive any further

369 connections. Output paths whose average data values do not meet the scaling range

370 requirement overtime are dropped from use. The connections are a mapping from

371 input data streams to output data streams. Initial random data will establish the initial

372 mapping and input from the environment will establish the enhanced mapping. In

373 this embodiment, the mapping can be a distribution table in memory that contains

374 output memory locations and distribution fractions from each input memory location. 375

376 [0049] In this preferred embodiment, the number of input sensors, input elements and

377 resolution, the number of specialize area and their connections to other specialized

378 areas; the connections between sections are specific to the environment being sensed

379 and can only be specified experimentally. Over specifying connectivity will not 380 degrade the effectiveness of the system and redundant or unused paths can easily be

381 identified and deleted.

382

383 [0050] In this preferred embodiment, memory recall is accomplished by comparing

384 the current input data plane to the oldest stored memory data plane, continuing

385 towards the newest memory entry until a match is found and then continuing with the

386 next input data plane. For every match as determined by the controller logic, a new

387 entry at the most recent end of memory is made. During this process, the differences

388 between the memory value and the input data plane values are accumulated and stored

389 when the controller determines that specific thresholds are met. The new stored data

390 plane values are convey to the next sections. Output can be numerical data, control

391 data for mechanical system, sound or any other medium that can interact with the

392 environment. Validation of output actions can be done initially with direct positive or

393 negative feedback. Output effort and computational effort can be estimated

394 algorithmically for feedback to the system unless output effectors that provide such

395 data are used.

396

397 [0051] An alternate embodiment for sensing, coding, storing, recalling, comparing

398 and conveying data values are electronic devices, digital or analog, designed

399 specifically for this method. Generally, the active element might include input data

400 value memory, memory, a comparator with controller logic connections and active

401 electronic switching to establish all internal connections. Adding active elements

402 would improve system performance until all environmental artifacts are sensed and all

403 possible inference pathways are accommodated.

404

405

406

Claims

1. An embodiment of this method can encode sensed information, recall from memory, make partial matches in memory and generate output that can be validated by the environment being sensed without prior knowledge of the information provided by the environment and without access to that information during execution of the method.

1.1 Within the conditions of claim 1 , a set of data values from individual sensor element of a sensor or recalled from individual memory elements of a memory system at the same instant of time can be spatially integrated into a data plane.

1.2 Within the conditions of claim 1, a series of data planes can be temporally integrated so that early data values and later data values are related in a single data plane.

1.3 Within the conditions of claims 1, spatially and temporally integrated data planes are abstract data planes that can be further abstracted resulting in data planes that contain data across longer periods of time and broader data sources.

1.4 Within the conditions of claim 1, spatial and temporal data integration allows lower level valid output data planes to combine with other lower level valid output data planes that then become retrievable abstract valid output data planes.

1.5 Within the conditions of claim 1, matching spatially and temporally integrated data planes from input sources to spatially and temporally integrated data planes in a memory can provide exact or partial matches between those data planes that can generate an output that is meaningful in the environment be sensed.

1.6 Within the conditions of claim 1, partial matches can introduce randomness in the data planes that may improve or degrade the validity of the output.

1.7 Within the conditions of claim 1, feedback from the environment being sensed or internal references reinforce the abstract data planes that generate output that achieves goals in the environment.

1.8 Within the conditions of claim 1, non-reinforced data planes are purged from memory to improve recall of reinforced data planes.

1.9 Within the conditions of claim 1, spatially and temporarily integrated data are only meaningful in the system in which they were integrated.

2. Computations using this method can be done with purpose built analog computers, simulation programs running on digital computers, or any other instantiation that can add, subtract, store and convey values in multiple data structures.