Defining semi-informative priors

[BrunoEtter]

First of all, thank you very much for this great package! It is really nice to have an easy to use R package to generate designs for choice experiments.

I would like to use the CEA procedure to generate a bayesian d-efficient design. I have enough knowledge to make assumptions about the directions of the effects of the attributes, but not about the exact strength. If I read your documentations correctly i can only define a point estimate for the priors. But in the article of Traets et al. (2020) describing the idefix package, they suggest to describe semi-informative priors with a truncated normal distribution. If i understand the bayesian approach correctly the idea is in general to define a distribution of likely parameters for the prior. How can I approach this in the cbcTools package? Instinctively I defined a very small effect with the right direction at the moment (see code below). Is there another way to define a prior distribution for an attribute where only the direction of the effect is known?

Many thanks in advance!

Code:

priors1 = list(
ProteinSource =c(-0.1),
NutritionalClaim = c(0.1),
QualityClaim = c(0.1, 0.1, 0.1),
Price = -0.1)

bayesian_design1 <- cbc_design(
profiles = profiles1,
n_resp = 500, # Number of respondents
n_alts = 2, # Number of alternatives per question
n_q = 12, # Number of questions per respondent
n_start = 5,
no_choice = FALSE,
priors = priors1,
method = "CEA",
parallel = FALSE
)

[jhelvy]

Thanks! So right now I'm just using a diagonal matrix for Sigma, you can see that here. I don't know why I didn't make this an optional argument before, but that's all I'd need to do to allow users to specify semi-informative priors. I'll take a look into adding that. In the mean time, you can directly use the idefix package. I found the package to be rather clunky and not super easy to use, which is one reason I made cbcTools (it's mostly a wrapper package that does some helpful things for you).

[BrunoEtter]

Thank you very much for this extremely fast response!
Because the cbcTools package is much more user friendly is exactly why i planned to use it instead of idefix.

If i understand correctly, does that mean that the prior distributions have a variance of 1 around the defined estimates? And do you think defining small effect sizes for the priors could be an approach to take information about the direction of the effects into account?

Thanks again for your answer and the package!

[jhelvy]

Yes that's correct. In my experience though, I've found this quite difficult to do. Because the sizes reflect relative differences in value, and I have struggled to find values that are reasonable without more prior information. From what I've read on the matter, the ability to inject priors into making the design is helpful so long as your priors are close, but if they're off, they can result in a worse design. So most of the time I just use a randomized design these days and aim for a larger sample size.

[BrunoEtter]

Thank you for this further elaboration.
Because D-efficient designs seem to be recommended the most in my field, i would like to stick with that. I therefore plan to do a pilot study to get more precise priors for the design. Just two more questions in that regard.

Would you recommend a design for the pilot study using the CEA algorithm with zero priors or an orthogonal main effect design?

I was playing a bit with different priors, just to understand how the D-error is affected by the priors. For some reason the D-error of the design never seems to go below 1. I do not have that many attributes and levels (2 x 2 x 4 x 3) and chose a fair amount of choice sets (total of 24 choice sets, distributed in two blocks), so i do not understand why the D-error remains that high. But i also struggle to find any recommendations about how small an acceptable D-error should be. Do you have any advise for interpreting the D-error?

[jhelvy]

This is a constant frustration for me. There doesn't seem to be a lot of good guidance out there on this. My understanding is that D error is really only helpful in a relative sense. That is, you compute the D error of two different designs, and then you can compare them to see which design is more D efficient. That's how you end up with a 0 to 1 score for D efficiency. Like, relative to design 1, design 2, is XX% more or less efficient. I could be wrong though. At the end of the day, I'm still not even convinced that things like D efficiency are all that important compared to other design considerations. I think this was a legacy from the field when you could only generate a single design to give to all respondents. That's not the case these days. All surveys are online and you can send each respondent a unique design, so a simple randomized design gives you a lot of variation still. Also, things like balance and overlap I think are probably more important from a design perspective.

[jhelvy]

I have now implemented the major overhaul. The latest version is v0.6.0, which you can install from GitHub with:

# install.packages("pak")
pak::pak("jhelvy/cbcTools")

I'll be on CRAN soon as well. In the meantime, please take a look at the revised documentation page.

For defining priors, I have a whole new approach now using cbc_priors(), which you can read about here.