Hm, I guess what it could be is that he has two quest objectives and he swaps to another one that does not have a quest marker.
This is the only way I have found to remove the quest marker by tracking a quest that does not have a quest marker. There are some, especially if it is a mission that...