2 Reasons you may not see a quest giver in your garrison.
1. You abandoned your dungeon group early (Usually after receiving your item which is shady/mean)
2. Your quest giver is in your outhouse taking a stinker. (You can turn the quest in while he's in there)
Hotfixed because people wait 30mins for queue, then they watch numerous people leave/join the group. So an average dungeon that shouldn't take long to complete starts to feel like a LFR with 20 wipes.
The downfall is that sometimes groups will fall apart @ the last boss. Resulting in you getting locked from that dungeon for the day... Which forces you to run randoms over and over until you get it. ORRR wait until the next day to queue again.
ORRR find friends/local people to help you complete your dungeon.. Which sounds like an amazing punishment to those that leave early.
So now people have to play legit. Oh darn.