Word2Vec / Skip-gram
Models
CBoW: Context words → predict CENTER word
Skip-gram: Center word → predict CONTEXT words
Parameters
Embedding Matrix = V × E Context Matrix = E × V Total params = 2 × V × E V = vocabulary size, E = embedding size
CBoW: Context words → predict CENTER word
Skip-gram: Center word → predict CONTEXT words
Embedding Matrix = V × E Context Matrix = E × V Total params = 2 × V × E V = vocabulary size, E = embedding size