You're being way too hard on yourself.  Really.  It sounds to me like you've conceded in a thousand ways to make church a positive experience.  He gets to wear what he wants (except pjs) and hangs with other kids.  Jeez, you're not even making him sit through the service yet!

DD is only 3, so I can't speak to the 6yo part.  But I think you should give yourself a break.  It seems to me to be his issue.  Maybe the fact that DH isn't going is making a bigger impression than you think.  Since your DD enjoys it, leave DS home with Dad and go, and enjoy it.  There's no reason to punish yourself and DD.  Either he'll starting feeling left out, or he'll hang with his dad.  Que sera sera.