Research questionHow can speech BCIs be compared fairly across different vocabularies and recording conditions?Reported accuracy and word error rate can overstate communication ability when they are calculated only over a system’s supported words. Comparisons also depend on which words a user is expected to communicate.