TRO and suspend visitation??????? I called the cops...baby is sick...STBX wont cooperate with Dr Orders

MY STBX and I do NOT have a parenting agreement in place, nor do we have anything in writing. I have been giving him visits on the honor system and have been flexible with him. But, Im done. This is the last straw. We have attorneys but I have been trying to keep the matter out of court while they discuss settlement which isnt working so far. He sees the baby (22 months old) no less than 5 days per week for 3 hours each which is what he wants. He loves his freedom and does not want the responsibility of caring for a child. He just wants to play with him and send him home.  This week and last weekend, the baby has been sick. He has had high fever (over 103), cough, runny nose, and a cold. Coxsackie virus and hand foot mouth disease has been going around. He started to feel better and then got sick again after my STBX took him to the zoo last saturday, and had him out in the heat for several hours. The baby spiked a fever of 104 that night which I treated with TYlenol suppository and Motrin - piggy back - per Dr orders over the phone. I was very upset - but I cannot talk to my STBX - he takes everything as I am trying to be controlling and he does what he wants - his plans, not what is best for the baby. Anyway,  He spiked a fever of 104 and has been sick with fever that comes and goes ever since. I have had him home with me but take him to my STBX's parents house for visits and to my parents for visits for a change of pace - he is kept cool and indoors and away from the public. I took him to the pediatrician Tuesday and found out he has a raging ear infection and his ear drum blew as a result of vomiting hard. He is on painkillers and Antibiotics. Here's the issue: my STBX refuses to keep the baby indoors and out of the public while sick. He keeps taking him out to the mall, store, running errands, out in the heat. I asked the Dr.  The pediatrician said that he should remain indoors in the cool and stay relatively isolated from the public until he has been fever free for 48 hours to give his body a chance to recuperate. This is exactly what I explained to my STBX. I for the weekend during visitation per Drs orders. My STBX refused and told me to "not tell him what to do with his child" and that he will " take the baby where he wants according to his 4th of july plans".  AGain, there is no parenting schedule. I was going to let him take the baby 9-12 today, 4-7 Saturday, 9-12 Sunday which is what he asked for. This was not agreed. I said we'll see. With he baby sick and his refusal to cooperate, Im not ok with him taking the baby. The baby and I have been up for several nights battling the illness and have had little respite from STBX. Last night, the baby was up for 3 hours crying due to pain in the ear and relative malaise. When the meds, kicked in - he felt relief. Today, I asked him again to follow the Drs orders to keep baby quiet, cool and out of the public. He started belittling me and the  it escalated into a fight with him calling me names and my telling him to calm down. But he refused. His parents got involved bc I suggested that I drop off the baby to his parents house for visitation since he was being difficult - he refused to keep the baby indoors and in the cool and was making it a power struggle when its not about us. Anyway, he demanded that the baby be ready at 10 am he said he was coming to get him whether I liked it or not. He refused to talk to me. I called him left a msg and I said no. I am not letting the baby go with him unless he agreed to keep the baby indoors, cool and not take him out and all about town with him bc he is sick. I called his mom and dad and asked them to talk to him. They tried, but he ended up yelling at his mom and making her cry calling her names too. His dad, didnt want to get involved and didnt want to deal with him. He is uncontrollable when he is angry. I told him not to come to my house at 10 am bc I was afraid that the matter would escalate and it did. I refused to open the door. He called me names etc through the door. My 16 year old and the baby were in the house, but I asked my teenager to take the baby to another room so that they were out of ear shot. I called the cops while he was there. He threatened me to give the baby to him or he will rip him from my arms. I refused to open the door. The cops came.  But, he left before they arrived. The cop took my statement which is essentially above. The police suggested I file a report with Family court and seek a TRO. I called my lawyer, but due to the holiday, she is away. I am worried. I do NOT want to allow my STBX to have the baby this weekend bc the baby is sick and STBX is off the handle. He is very combative and argumentative and takes it out on me and the baby. Not physically, but verbally and emotionally. I am afraid to leave the baby with him this weekend bc he doesnt seem to realize that the baby needs to be cared for - he's not ok. I have offered to let the baby go to STBX parents house for visits and the STBX can see him there since they will keep him in he cool and take care of his fever and do what is best for the baby whereas I cannot trust that my STBX will bc he has proven that he just does what he wants, not what the baby needs (ie: taking him to the zoo last weekend with a fever of 103 when it was 77 outside. The baby came home so sick). Come monday I will have to call my lawyer. I am scared about the "punishment" that the STBX will deliver for calling the police.  Should I file for a TRO???? WHat to do now????? How should I proceed???