I'm wonder what his age is. Is it possible for some reason he is having sexual problems, not really problems but more along the lines of "not aroused" issues. He may need a medication. If he is, he could be with drawing because it is emotionally depressing for him. He doesn't want to admit it. He may also just be getting older and is depressed about that. People who are experiencing depression, often withdrawl.
Have you asked him if there is more you can do to reach out to him? You said he finds things to keep busy, I suggest you find things to offer to do with him. Sometimes, just being together and the time not be about one or the other getting attention is all you need to feel closer.
Don't give in to self pity, I agree you need to address these issues and find a plan to move from one place to another as a couple. But, don't speak such negativity to yourself on the way. Never use the word "worthless" when speaking about yourself. If you or anyone was worthless, Jesus would not have had to die. He proved you are not worthless on Calvary, embrace that.
Please read Isiah 54 and see yourself as the woman God is speaking to. Embrace His healing word as your own. Believe what He says is true.
Also, I suggest to you, that you lay hands on your husband at night while he sleeps and pray for him. Whisper or pray silently, and pray for God to help him, to guide him, to protect him and heal him. I've done this since I got married 26 years ago. And once a few years ago, my husband said he had woke up many times when I prayed and it gave him peace to know I loved him and prayed.
I prayed for everything from his job to our marriage. I know God sees the man who wife is praying for him. HTH.