Parallizing the SVD Computation for Latent Semantic Analysis

1
Parallizing the SVD Computation for Latent
Semantic Analysis

• Jorge Civera Saiz
• Borislav Stoyanov

2
Introduction Information Retrieval Methods

• Term-matching retrieval techniques.
• Based on sintaxis.
• Typical in search engines.
• Latent Semantic Analysis.
• Based on semantic.
• Allows document- document, term-document and
  term-term comparison. Examples of queries
• Important terms in a document.
• Related documents to one given.
• Related terms to one given.

3
Latent Semantic Analysis

• Idea Model the relationship between terms and
  documents using a frecuency term-by-document
  matrix.
• Element aij is how many times word i occurs in
  document j.
• This matrix represents the underlying semantic
  structure along these documents.
• We apply SVD decomposition to this matrix to
  remove noisy information and reveal semantic
  structure.

4
SVD decomposition

• Matrix A T x S x D
• Matrix T are the eigenvectors of matrix A x At
• Matrix D are the eigenvectors of matrix At x A
• Matrix S are the non-negative square roots of the
  eigenvalues of At x A or A x At
• We use SVDPACKC library to carry out SVD
  decomposition of sparse matrices.

5
Previous work

• Parallelization of the SVD Computation Lanczos
  Algorithm. University of Berkeley.(Ongoing)
• Parallel SVD (since 1993, ongoing). University of
  Wuppertal.
• A fine-grained parallelization of the
  block-Jacobi SVD algorithm.(Gabriel Oska)

6
SVDPACKC library comparison

• Several algorithms to resolve SVD decomposition

Sparse Matrix 374 x 82 (4 dense)
7
Analysis of LAS2 code
of total time spent in that function
Sparse matrix size (3 dense)
8
Performance Study (a priori)

• Speed-up Theoretic Limit ( Amdahls Law )
• S 1 / ( (1 - F) F/N )
• In our last case

If N inf S 1 / (1 - 0.894) 9.43
9
Conclusion

• Parallelize matrix and vector operations, since
  they are 90 of the time.
• Matrix x Matrix x Vector
• Matrix x Vector
• Constant x Vector Vector
• Vector x Vector
• Questions ??