Hearing this drives me nuts.  I agree that there aren't enough smart people working in INFOSEC, but the hiring managers write the position requirements so onerously that it's incredibly hard for smart people to get those jobs.  And if we're going to discuss government billets, then you have to deal with the government hiring process, which is a complete turn-off for a lot of people (smart ones included).
Source: 
Infosecurity - Black Hat 2012: There