NCSI: Scaling Rank-Revealing Randomized Singular Value Decomposition Algorithms Using Spark

Status	Completed
Mentor Name	Hao Ji
Mentor's XSEDE Affiliation	Education Allocation
Mentor Has Been in XSEDE Community	4-5 years
Project Title	Scaling Rank-Revealing Randomized Singular Value Decomposition Algorithms Using Spark
Summary	Constructing a low-rank matrix approximation with a suitable rank is critical to many data analytic applications, but for big data, its scalable distributed implementations have not been investigated much. In this project, we plan to design, analyze, and implement scalable rank-revealing randomized singular value decomposition algorithms by bringing together recent advances in randomized algorithms and Spark's big data processing. We will benchmark their performance on a variety of matrix data from the SuiteSparse Matrix Collection as well as the datasets for large-scale recommender systems.
Job Description	The student will study rank-revealing randomized algorithms for singular value decomposition, develop and optimize their implementations in Spark, and benchmark them over several real-life matrix data.
Computational Resources	XSEDE Bridges' Hadoop and Spark resources
Contribution to Community
Position Type	Intern
Training Plan	I will have weekly meetings with the student to discuss ideas for carrying out the proposed work. Please note that the student listed is already familiar with GraphX and Spark systems.
Student Prerequisites/Conditions/Qualifications	Must be a student at California State Polytechnic University, Pomona Must have a good understanding of linear algebra and good programming skills in Scala.
Duration	Semester
Start Date	01/19/2019
End Date	05/10/2019