I'm speechless. We have common ground here, not identical, but close enough that your words hit my heart. You are not unreasonable. Sometimes, one needs to be strong enough to make the hard decisions.
First, please remember that this is the year before middle school. Middle school is not here, and no amount of worrying yourself sick now will decrease the amount you worry yourself sick when you actually are there. There's no preemptive angst. I was miserable with worry when Boy started middle school last year. I had a full homeschool plan in place because I knew how fragile he is (physically and emotionally). Guess what? I didn't need it. Middle school was awesome. He's actually popular. Yes, he's in a big "nerd school" with no athletics, cheerleaders, etc., but even there, he stands out like a sore thumb, so I was scared. I'm not trying to be Pollyanna here, but you can only fight the monster in front of you. Keep your powder dry for next year. For now, put your feet up and soak in the last of his little boy years.