Is the breaking of the bread in Acts 20 considered the Lord's Supper because it was the first day of the week? Would they not have eaten a regular meal, also referred to as breaking bread, on Sunday?
Sometimes I don't know if this is a justification for Sunday worship because there was a communion, or it was a communion because it happened on Sunday.
Act 2:46 Day after day they met together in the temple. They broke bread together in different homes and shared their food happily and freely.
Also, it probably bears noting that the Lord's supper back then was more than the requisite 1.5 mm triangular corner of a matzo cracker and a precise 1/4 oz plastic cup of grape juice. In Corinth later, Paul had to admonish the brethren for gorging themselves and getting drunk during communion. How'd they get drunk on grape juice?? 
I support you here in that as I understand it "break bread" is not an exclusive term meaning to have sacrament, it really means to share a meal. If you look up the term in a entomological dictionary it will bear this out. Bread was a staple and was often not cut up but portions were broken off for each person.
If the entire Sunday worship is hung on this one passage you have to ask yourself. If God really intended for his followers to give up the Sabbath that he spent thousands of years establishing would he have only left us this one tenuous passage to tell us that? Why would He not just come right out and say it? God is very clear with his expectations with us in every other way. All of a sudden he starts to beat around the bush and give us vague passages that are suppose to indicate that his sanctified holy day has changed, without him actually saying it.
God wrote the ten commandments
with his own hand on
stone tablets . Deu 4:13 5:22 9:10 Ex 24:12 31:18 34:1 It's mentioned quite a few time that it's written with Gods own hands on stone tablets, is that significant? It should be an indicator that they were important and not meant to be changed or altered in any way. Even today when we say something is "written in stone" it means permanent and unchanging. The laws that were transitory were all written on parchment by Moses. Why the distinction? Why would not God just have Moses write out all the old covenant laws and the ten commandments and have them all on paper. It would be a whole lot easier.
And then Jesus could have just came right out and told us, '
oh, by the way, you don't have to keep the sabbath after I'm gone.' Jesus Shouldn't have needed to spend so much time telling people how to correctly observe the sabbath when it was to be abolished? Why did God start the 4th commandment with the word "remember", like He knew it would be forgotten?