This paper proposes density estimation as a feasible approach to the wide class of learning problems where traditional function approximation methods fail. These problems generally involve learning the inverse of causal systems, specifically when the inverse is a non-convex mapping. We demonstrate the approach through three case studies: the inverse kinematics of a three-joint planar arm, the acoustics of a four-tube articulatory model, and the localization of multiple objects from sensor data.
The learning algorithm presented differs from regression-based algorithms in that no distinction is made between input and output variables; the joint density is estimated via the EM algorithm and can be used to represent any input/output map by forming the conditional density of the output given the input.
In M.C. Mozer, P. Smolensky, D.S. Touretzky, J.L. Elman, & A.S. Weigend (eds.), Proceedings of the 1993 Connectionist Models Summer School. pp. 316--323. Hillsdale, NJ: Erlbaum Associates, 1994. gzipped postscript.