NCI National Computing Infrastructure also notes in a Google Doc
Eventually, do_all_model_recovery.html
not_multilevel_model_recovery . Works for estimating a few factors but blows up to almost 5 hrs when estimating all factors, and at least with limited runs (700 iterations, 2 chains), lab factor blew up. But reference group might be correct.
If I don’t add variance term for participants’ eta, and probably too for lapse and sigma, my estimates will probably be wrong because of all the dispersion.
Not modeling a factor, including participant differences, results in a shallower slope.
Discrepancy of 0.37 between my logSigma, -1.61 and what brms estimated, -1.24 Works in this seemingly super-simple case: logSigma_wrong
Also works when simultaneously estimating alpha logSigma_plus_lapse_try
I’m thinking that creating unmodeled dispersion by adding back in the target penalty will make sigma be estimated too shallow. Yes, that was it! logSigma_plus_lapse_unmodeled_dispersion actually i had eta fixed rather than modeled so eta was underestimated now having it be estimated , but I added estimation of eta and result the same.
Hopefully modeling effect of target will restore correct sigma estimation. It does! logSigma_plus_model_dispersing_factor
CIs of estimates stop short at zero
Check out visualize_and_select_priors.html
A remaining question is why lb and ub are not accepted for prior that applies only to eta_Intercept. If one sets lb and ub, one gets this error:
` Error : Prior argument ‘coef’ may not be specified when using boundaries. `
So I dropped lb and ub.
That yields a warning because the uniform prior I set does have bounds, and brms doesn’t like that discrepancy.
` Warning :It appears as if you have specified a lower bounded prior on a parameter that has no natural lower bound. If this is really what you want, please specify argument ‘lb’ of ‘set_prior’ appropriately. Warning occurred for prior b_eta_Intercept ~ uniform(0, 2.5) `
with “Error : Prior argument ‘coef’ may not be specified when using boundaries.” make it difficult to have a uniform prior, should I go for a non-uniform prior even for effects of groups that I’m interested in estimating (as opposed to the Intercept)
Age_group estimate went crazy in not_multilevel_model_recovery.qmd, maybe by confining the prior to the intercept I no longer have a prior on the coefficient.
not_multilevel_model_recovery - why did this stop working for estimating effects of age and gender? Try make_stancode()
why are gender and ageGroup estimates zero
also looking at what is happening during a chain:
, goes straight to zero almost. And posterior density is ridiculously peaked at zero.
More minimal yet still
Save warm-ups and then do diagnostics too.
fit$fit@sim$samples[[1]] includes stepsize and attr(,”sampler_params”)$divergent
Although it’s often off by maybe 0.16
I’m trying to get back to eta being so wrong when both lapse and logSigma estimated.
At first I thought lapse biased high, but maybe that byproduct of high eta?
I upped the trials and even with lapse estimation on it seems ok. With eta =1.1, 1.6 and 1.7 as well it does fine!
eta_lapse_estimation_no_apparent_bias
Can’t figure out why it always just splits the difference and gives zero for effect of age.
Splitting the difference and assigning close to zero effect of age
Trying to work back down from not_multilevel_model_recovery but it also now isn’t working for gender and age? It really seems like sometimes brms won’t recover the effect of these factors.