how to avoid this? I tried turning off dmg numbers, ball lightning enchantment, and a few passives related to crackling energy, none helped.

Remove dots (poison vampiric and firebolt), and remove monster pulling if you use it.

Smart blizzard devs created so much useless calculations that modern pcs can't even handle it.