That's correct.
The accessory charm is only used for healers / mages who use Enhanced Effect (EE) on charms. Stats will not work in Accessory charm.
Keep in mind, Enhanced Effect in Accessory slot have a 25% reduced effect, and 25% reduced mana use.
Quote
The Accessory Charm is an additional Equipment slot reserved for a second charm. The Accessory slot however does not reflect stats that are on that charm, with exception to Enhanced Effect. Efficiency is reduced by 25% of the total displayed. This charm is used just like the main charm slot and is not reflected by your total inventory space restrictions like the rest of your equipment. When used in Catacombs, the mana usage is also reduced by 25%.
In the inventory, it is abbreviated as Acc. Charm