Making a few assumptions from your thread:
1. there is a family you need to support.
2. you have extensive experience in an area and you managed to secure a good package this sector during an economic downturn.
3. You rightly point out, roles in VC are hard to come by especially if they're good ones.
4. The VC package is less but enough money to live well on.
5. The VC job may be technically too challenging
I know the feeling, that you're taking a massive plunge but in all fairness it is not as massive as it feels. You're not launching a start-up from scratch which might go entirely pear-shaped and lose all your savings.
Secondly, you managed to secure a good consulting package when times are hard, if the VC role doesn't work out you should be able to secure an event better package in say 1 or 2 years assuming the economy improved.
I think you should also weigh into your consideration your time commitments and skill set. I gather that the VC job is technically challenging and you might have to facture in eve more hours than the ones you'd expect. How does your partner feel about this?
To put it bluntly, you're already demotivated even before starting the new consulting job, imagine how bad you'd feel if the hours got out of hand or the team didn't turn out to be that great. Your productivity would drop and you may end up worse off than when you started.
Take the VC role, do what you wanted to do since you used to work for a few back in the day, if it doesn't work out you can come back to your old specialism with some new experiences and a renewed confidence in your old job and your abilities.
Hope that helps!