It depends on what your agreement says. The sad thing is that if you take her to court it may be too late for you to take him. Check with your lawyer. If it states that you  can have him for vacation then you should be able to take him. It all depends on the wording.
If nothing else and you really want to take him, maybe you should consider saying, ok , you can go. This is when we are going. You are responsible for your own everything .(hotel etc) . This is what it cost me..make your arrangements.
Chances are that she might not have the money, nor be able to get off work. If you act like it doesn't matter then she might think it doesn't.
In the mean time, check out your agreement and see your lawyer. If you end up not being able to take him this year, lesson learned for you. Fix it so that it can't happen again.
Good Luck. Let me know how it works out as I will soon be in a similar situation.