Well, as vasopressin stimulates CRH/ACTH release, that would actually cause a
decrease in the amount of "work" the adrenals need to do.
And yup. It takes a lot of caffeine to even really cause many of these changes to a noteworthy degree
@OP/topic, peep this paper:
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3046013/the introduction says: "Adrenal insufficiency is a relatively rare disease. Primary adrenal insufficiency has a prevalence of 90–140 per million, and the estimated prevalence of secondary and tertiary (central) adrenal insufficiency is 150–280 per million."
Given the ubiquity of caffeine/coffee, I'd say the chance for them to cause this adrenal problem is virtually 0%.