Keep in mind that there is a difference between "dedicated space" and "working clearance." You are talking about the former item. The space directly above and below the panel (i.e., the footprint) is dedicated to the installation of conduits. That is why they don't want "foreign" systems in that area. In this context, "foreign" means "not electrical, and not related to the electrical installation."
The 6 feet magic number begins at the top of the panel. You want to be able to add conduits later, so the first six feet above the panel can't have anything "foreign" in the way. Higher than that, they figure you will have bent the conduits off in some other direction, so it's OK to put pipes and ducts up that high. Just put a drip shield below the pipes or ducts.