Nonnegative Matrix Factorization with Toeplitz Penalty

Matthew Corsetti, Ernest Fokoué

Abstract


Nonnegative Matrix Factorization (NMF) is an unsupervised learning algorithm that produces a linear, parts-based approximation of a data matrix. NMF constructs a nonnegative low rank basis matrix and a nonnegative low rank matrix of weights which, when multiplied together, approximate the data matrix of interest using some cost function. The NMF algorithm can be modified to include auxiliary constraints which impose task-specific penalties or restrictions on the cost function of the matrix factorization. In this paper we propose a new NMF algorithm that makes use of non-datadependent auxiliary constraints which incorporate a Toeplitz matrix into the multiplicative updating of the basis and weight matrices. We compare the facial recognition performance of our new Toeplitz Nonnegative Matrix Factorization (TNMF) algorithm to the performance of the Zellner Nonnegative Matrix Factorization (ZNMF) algorithm which makes use of data-dependent auxiliary constraints. We also compare the facial recognition performance of the two aforementioned algorithms with the performance of several preexisting constrained NMF algorithms that have non-data-dependent penalties. The facial recognition performances are evaluated using the Cambridge ORL Database of Faces and the Yale Database of Faces.

Keywords


Nonnegative matrix factorization; Auxiliary constraints; Toeplitz matrix; Zellner g-Prior; Image processing; Facial recognition; Subspace methods

Full Text:

PDF

References


C. Boutsidis and E. Gallopoulos, Svd based initialization: A head start for nonnegative matrix factorization, Pattern Recognition 41(4) (2008), 1350 – 1362.

Z. Chen and A. Cichocki, Nonnegative matrix factorization with temporal smoothness and/or spatial decorrelation constraints, in Laboratory for Advanced Brain Signal Processing, RIKEN, Tech. Rep (2005).

A. Cichocki, R. Zdunek and S.-I. Amari, Csiszár’s divergences for non-negative matrix factorization: Family of new algorithms, in Proceedings of the 6th International Conference on Independent Component Analysis and Blind Signal Separation, ICA’06, pp. 32–39 , Berlin, Heidelberg. Springer-Verlag (2006).

M.A. Corsetti and E.P. Fokoué, Nonnegative matrix factorization with zellner penalty, Open Journal of Statistics 5(7) (2015), 777 – 786.

E.P. Fokoué, D. Sun and P. Goel, Fully bayesian analysis of the relevance vector machine with consistency inducing priors, Technical report, Rochester Institute of Technology, 1 Lomb Memorial Dr, Rochester, NY 14623 (2009).

E. Gonzalez and Y. Zhang, Accelerating the Lee-Seung algorithm for non-negative matrix factorization, in Tech. Rep. TR-05-02, Dept. Comput. & Appl. Math., Rice Univ., Houston, TX, 1 – 13 (2005).

A.B. Hamza and D.J. Brady, Reconstruction of reflectance spectra using robust nonnegative matrix factorization, IEEE Transactions on Signal Processing 54(9) (2006), 3637 – 3642.

P.O. Hoyer, Non-negative sparse coding, in Neural Networks for Signal Processing XII (Proc. IEEE Workshop on Neural Networks for Signal Processing, pp. 557–565 (2002).

P.O. Hoyer, Non-negative matrix factorization with sparseness constraints, Journal of Machine Learning Research 5 (2004), 1457 – 1469.

D.D. Lee and H.S. Seung, Algorithms for non-negative matrix factorization, in Proceedings of the 13th International Conference on Neural Information Processing Systems, NIPS’00, pp. 535–541, Cambridge, MA, USA. MIT Press (2000).

C.-J. Lin, Projected gradient methods for nonnegative matrix factorization, Neural Comput. 19(10) (2007), 2756 – 2779.

P. Paatero and u. Tapper, Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values, Environmetrics 5(2) (1994), 111 – 126.

V. Pauca, J. Piper and R. Plemmons, Nonnegative matrix factorization for spectral data analysis, in Linear Algebra and its Applications, Vol. 416 (2005), 29 – 47.

J. Piper, V.P. Pauca, R.J. Plemmons and M. Giffin, Object characterization from spectral data using nonnegative factorization and information theory, in Proceedings of the 2004 AMOS Technical Conference (2004).

R. Sandler and M. Lindenbaum, Nonnegative matrix factorization with earth mover’s distance metric for image analysis, IEEE Transactions on Pattern Analysis and Machine Intelligence 33(8) (2011), 1590 – 1602.

H.S. Seung and D.D. Lee, Learning the parts of objects by non-negative matrix factorization, Nature 401(6755) (1999), 788 – 791.

S. Sra and I.S. Dhillon, Generalized nonnegative matrix approximations with Bregman divergences, in Y. Weiss, B. Schölkopf and J.C. Platt (editors), Advances in Neural Information Processing Systems 18, pp. 283 – 290, MIT Press (2006).

M.E. Tipping, Sparse bayesian learning and the relevance vector machine, Journal of Machine Learning Research 1 (2001), 211 – 244.

Y. Wang, Y. Jia, C. Hu and M. Turk, Fisher non-negative matrix factorization for learning local features, in Proc. Asian Conf. on Comp. Vision, pp. 27 – 30 (2004).

S. Wild, W.S. Wild, J. Curry, A. Dougherty and M. Betterton, Seeding nonnegative matrix factorization with the spherical k-means clustering, Technical Report, University of Colorado (2002).

S.M. Wild, J.H. Curry and A. Dougherty, Motivating non-negative matrix factorizations, in Proceedings of the Eighth SIAM Conference on Applied Linear Algebra, SIAM (2003).

Y. Xue, C.S. Tong, W.-S. Chen andW. Zhang, A modified non-negative matrix factorization algorithm for face recognition, in 18th International Conference on Pattern Recognition (ICPR’06), Vol. 3, pp. 495 – 498 (2006).

R. Zdunek and A. Cichocki, Non-negative matrix factorization with quasi-Newton optimization, in Eighth International Conference on Artificial Intelligence and Soft Computing (ICAISC), pp. 870 – 879, Springer (2006).

A. Zellner, On assessing prior distributions and Bayesian regression analysis with g-prior distributions, in Bayesian Inference and Decision Techniques: Essays in Honor of Bruno de Finetti, pp. 233–243, Elsevier Science Publishers (1986).




DOI: http://dx.doi.org/10.26713%2Fjims.v10i1-2.851

eISSN 0975-5748; pISSN 0974-875X