You're right, unique quality is ckecked first. But chance of getting unique items remains low, even at 850%.
Hence, set items would take the lead, and most of them would be downgraded to double-durability magic items.

For example, Baal loot at 850%mf (hell, no quest drop) :
unique item : 19 to 24% (*)
set item : 43 to 46% (*)
rare item : 33 to 35% (*)
magic item : almost 0%

(*) for non class-specific items, percentages vary according to QLvl of dropped item.

not only does having 800 mf means your char can't possibly be efficient, it also reduces the chances of getting what you're looking for
drop to ~200mf