University of Illinois Urbana-Champaign

World embedding models evaluation and word inference accelerator

Liu, Zizhen

Content Files
ECE499-Fa2018-liu.pdf

Permalink

https://hdl.handle.net/2142/105408

Description

Title
World embedding models evaluation and word inference accelerator
Author(s)
Liu, Zizhen
Contributor(s)
Hwu, Wen-Mei
Issue Date
2018-12
Keyword(s)
  • NLP
  • GPU
  • Word Embedding
Date of Ingest
2019-09-04T17:26:27Z
Abstract
Many word2vec algorithms have been developed to represent vocabularies with more accurate number vectors. The most popular ones to date are word2vec, FastText, and GloVe. Despite their popularity, there has been a lack of published work that helps new users to understand how models trained by different algorithms are performing on words from different categories. Additionally, despite the great work being done in implementing word2vec training algorithms on GPUs, there has been a lack of published work in implementing word2vec inferencing algorithm on GPUs. Therefore, this thesis will first present our research on inference accuracy of models trained by word2vec, FastText and GloVe across multiple categories of datasets and explain why we see such differences in prediction accuracy. Secondly, the thesis will present an algorithm for performing batched word analogy queries running on GPUs, that can adapt to different models trained by any algorithm and thus can significantly reduce the efforts required in comparing models trained by different word vector generation algorithms.
Type of Resource
text
Genre of Resource
Other
Language
en
Permalink
http://hdl.handle.net/2142/105408

Owning Collections

Senior Theses - Electrical and Computer Engineering PRIMARY
The best of ECE undergraduate research