Learning Word Embedding

Lilian Weng · 8 years ago

Word embedding transforms text words into numeric vectors of lower dimensions than one-hot encoding, enabling machine learning models to process natural language efficiently. Skip-gram and CBOW are context-based approaches that learn embeddings by predicting words from context, with techniques like hierarchical softmax reducing computational complexity from O(V) to O(log V) during training. This foundational method enables models to capture semantic relationships between words and has become standard for natural language processing tasks.