Contempt charges

Hello, I'm new to the group and thought it would be nice to talk to people that have delt with difficult Exs before. Here's my situation. I was granted sole custody of our three children and my ex-husband has supervised visitation. The court ordered that the supervisors must conact me at least 24 hrs in advance with the visitis beign from 12 - 6 pm every other Saturday/Sunday. For the past year I have been trying to be felixable with them. They have "forgotten" to call or called 12 hrs before the visit. My ex-husband has also shown up with two supervisors that were not yet approved by the court. He has filed a motion for contemp against me for denying him a visitation on a non scheduled day. Trying to accomidate him I told him a that I was going out of town on the day he wanted and would need a supervisor to call me ASAP so that I could make arrangments with the children. The call came in 2 1/2 hrs past the court's 24 hr deadline and the supervisor who called was only going to be supervising the visit in the afternoon and wasn't sure who was supervising in the morning. I already made arrangements for the children since I had to be out of town. My attorney said they don't have a case since I didn't do anything wrong other than not "obeying" the demands of my ex-husband. I know the court put the visitation guidelines in place for a reason, but at what point does the court get tired of dealing with people like me ex?