It isn't that the load is more important, it is that the load is more likely to exist. If you only have 3000VA of dwelling unit lighting, it is likely that the entire 3000VA could be on at the same time. The larger the lighting load, the more likely that it will not be all on simultaneously. That diversity is what the demand factors are trying to capture.
I would imagine that the numbers themselves are tied to some real-world study or condition, but are also somewhat arbitrary. But, you have to draw a line somewhere.