Geographical Evaluation of Word Embeddings
Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers) (2017)
Abstract
Word embeddings are commonly compared either with human-annotated word similarities or through improvements in natural language processing tasks. We propose a novel principle which compares the information from word embeddings with reality. We implement this principle by comparing the information in the word embeddings with geographical positions of cities. Our evaluation linearly transforms the semantic space to optimally fit the real positions of cities and measures the deviation between the position given by word embeddings and the real position. A set of well-known word embeddings with state-of-the-art results were evaluated. We also introduce a visualization that helps with error analysis.
Authors
BibTex
@InProceedings{konkol-EtAl:2017:I17-1, author = {Konkol, Michal and Brychc\'{i}n, Tom\'{a}\v{s} and Nykl, Michal and Hercig, Tom\'{a}\v{s}}, title = {Geographical Evaluation of Word Embeddings}, booktitle = {Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers)}, month = {November}, year = {2017}, address = {Taipei, Taiwan}, publisher = {Asian Federation of Natural Language Processing}, pages = {224--232}, abstract = {Word embeddings are commonly compared either with human-annotated word similarities or through improvements in natural language processing tasks. We propose a novel principle which compares the information from word embeddings with reality. We implement this principle by comparing the information in the word embeddings with geographical positions of cities. Our evaluation linearly transforms the semantic space to optimally fit the real positions of cities and measures the deviation between the position given by word embeddings and the real position. A set of well-known word embeddings with state-of-the-art results were evaluated. We also introduce a visualization that helps with error analysis.}, url = {http://www.aclweb.org/anthology/I17-1023} }
