I've done this before and never had an issue as long as the conductors protected by the 100 amp breaker are rated for 100 amps. What size are the conductors feeding the sub panel? They have to be rated for 100 amps. They can't be #4 copper or #2 aluminum like they could be the service entrance of a dwelling, although some inspectors will let you do that too.