Splits the speakers into k folds, fits a random forest from ranger on
all folds but one, and predicts the tokens of the speakers left out, so
that every token is predicted by a forest that never saw its speaker.
This estimates the accuracy for new speakers, which the out-of-bag
accuracy overestimates when tokens of the same speaker are alike.
Arguments
- formula
Formula of the forest, such as
l1 ~ vot + vowel_ratio + rate.- data
A data frame with one row per token.
- speaker
Name of the column with the speakers, as a string.
- k
Number of folds.
- seed
Seed for assigning speakers to folds and for breaking ties between the votes of the trees. The forest of fold
iuses seedi. Your own random seed is left as it was.- num_trees
Number of trees in each forest.
Value
A tibble with one row per token and the columns speaker,
fold, observed and predicted. The accuracy for new speakers is
mean(cv$predicted == cv$observed).
Details
When the response is a factor with one value per speaker, such as the L1, each fold gets the same number of speakers from each group. With the default seed, the folds and forests are the ones in the slides of Advanced Statistics, session 6.
Examples
if (requireNamespace("ranger", quietly = TRUE)) {
library(simling)
cv <- cv_speakers(l1 ~ vot + vowel_ratio + rate, accent, num_trees = 100)
mean(cv$predicted == cv$observed)
}
#> [1] 0.875
