If you can get in on the fast stream, perhaps do this first (I don't think you can go back in as an experienced hire onto fast track?). You can always come in as experienced hire into consultancy, and graduate training in consultancies is mostly on-the-job with the odd week long course thrown in (where you tend to simulate a project anyway)
Secondments happen both ways, so have a look at Whitehall in Government (google it) to get an idea of the opportunities here. You can second in to consultancy first to try it out, and then make a permanent move if you like.
All the best consultants I know have first hand experience of working in Whitehall - either through secondment or direct.
Also, relationships are critical here - so don't burn bridges at either end....