Deterministic EM algorithms with penalties

Recently, O'Sullivan introduced roughness penalties for use in stochastic problems where the use of Markov random fields may not arise naturally. In this study, these penalties are used for the deterministic problem. θ ε R^p is considered to be the vector of parameters to be estimated. The available data are y_m = Σ_n=1^N H_mn X_n, 1 ≤_m ≤ M where y ε R₊^M, H_mn ≥ 0 and x ε R₊^N depends on θ. The manner in which x depends on θ yields slightly different algorithms. The matrix H is assumed to have at least one positive entry in each column. It is shown that x may be considered as the complete data for θ. The incomplete data I-divergence is shown to equal an averaged complete data I-divergence plus an additional term. The deterministic FM algorithm then consists of minimizing the averaged complete data I-divergence. Lastly, a maximum entropy penalty and a roughness penalty are incorporated into the problem.