R session abort (SIGABRT) when training word2vec — out-of-bounds std::vector access

[ssppkenny]

Bug Description

Training a word2vec model causes the R session to abort on Arch Linux and Fedora with a std::vector assertion failure:

/usr/include/c++/16/bits/stl_vector.h:1253: Assertion __n < this->size() failed.

Steps to Reproduce

library(word2vec)
word2vec(x = c("i saw the queen yesterday", "the queen saw a queen"), type = "cbow", dim = 15, iter = 20)

This crashes the R session immediately on any distribution that has _GLIBCXX_ASSERTIONS enabled in libstdc++ (Arch Linux, Fedora, and others with debug-friendly C++ standard library builds).

Root Cause

In src/word2vec/lib/word2vec.cpp:72-73, the code uses &_trainMatrix[index] to compute pointers for std::copy:

std::copy(&_trainMatrix[wordIndex * m_vectorSize],
          &_trainMatrix[(wordIndex + 1) * m_vectorSize],
          &v[0]);

When iterating over the last word in the vocabulary, (wordIndex + 1) * m_vectorSize equals _trainMatrix.size(). Calling operator[] with an index equal to size() is out-of-bounds — std::vector::operator[] requires n < size().

Using &_trainMatrix[size()] to get a past-the-end pointer is undefined behavior. The correct approach is _trainMatrix.data() + size(), which is explicitly valid per the C++ standard.

Affected Versions

All versions of the word2vec R package on Linux distributions with _GLIBCXX_ASSERTIONS (not reproducible on CRAN because they build without these assertions).

Fix

A fix has been contributed in PR #27:

• Replace &_trainMatrix[i] → _trainMatrix.data() + i
• Replace &v[0] → v.data()

Workaround

Rebuild the package from source after applying the patch, or set CXXFLAGS="-U_GLIBCXX_ASSERTIONS" before installation to disable the assertion (not recommended — masks the bug).

[jwijffels]

Isn't this just because the default of word2vec is min_count = 5L, meaning to include only terminology which occurs at least 5 times. In your example the terminology contains only the space separator.

[ssppkenny]

I have the same problem, when I use the example from the documentation
library(udpipe)

Take data and standardise it a bit

data(brussels_reviews, package = "udpipe")
x <- subset(brussels_reviews, language == "nl")
x <- tolower(x$feedback)

Build the model get word embeddings and nearest neighbours

model <- word2vec(x = x, dim = 15, iter = 20)

[jwijffels]

Can you prove that your suggested changes give the same embeddings on e.g. debian

[ssppkenny]

Unfortunatelly no, for the moment I have Arch Linux installed, but can test it later.