Infant Baptism and Infant Communion?

This is an honest question: Why is it that people who are paedo-baptists (infant baptisers) don't invite non-believers to share in the Lord's Supper?

As a Baptist, I don't baptise infants simply because infants do not have a clear profession of faith--they are not believers. And I'm as certain as can be that non-believers should not be baptised.

Paedo-baptists have their own theological explanation as to why infants should be baptised, but at this point in my theological acuity, I still can't quite grasp their argument.

Being that as it may, I now wonder why paedo-baptists don't invite non-believers to share in communion. Or, to ask it another way, Why don't paedo-baptists put a speck of bread and a drop of juice into an infants mouth, much in the same way as they baptise the infant. Just as the infant might share in the death, burial and resurrection as portrayed in baptism, the infant might also share in the body and blood of Christ through communion.

Although the covenantal argument for infant baptism is too complex for me to really grasp, I would think that it could also be applied to infant communion. So, with this in mind, why don't paedo-baptists believe in infant communion.

My argument against infant communion is the same as my argument against infant baptism: non-believers shouldn't take communion or be baptised.