A new approach to filtering and adaptive control: stability results

摘要

A new approach to adaptive control is proposed. The principal distinguishing feature is the direct estimation and updating of the criterion function by means of a filtering operation on a vector of transitional pseudo-return functions. The data storage and computational problems often associated with explicit probability distribution updating via Bayes' rule are thus avoided. Convergence properties are established for a simple linear criterion function filter designed for a class of adaptive control problems typified by a well-known two-armed bandit problem. Optimality properties are established for the filter in a companion paper.