this tournament is "supposed" to be the best of the best, not the average.
It's just "the best of the best" from every confederation, so instead of likely better qualified referees who officiate f.e. Premier League & campions league games regularly, there are some African referees (amongst others) with little to no experience at that level of play. Not only is there a difference attendance-wise between a world cup game & f.e. a game in the African qualifiers, but more importantly the pace of play is way quicker and therefore harder to officiate.
But as long as the FIFA insists on the same selection principle like they have for the teams (divided in confederations), they pretty much accept/ask for bad officiating. There would still be lots of questionable calls, but probably less.
I believe the ref that was sent packing in a recent tournament was from South America
Imo you refer to Coulibaly, the ref that disallowed the goal in the 2010 world cup, that would have been the game-winner for the US team. Guy was from Mali and horrible, didn't get another game afterwards (only acted as the 4th official) and was send packing after group stage.