Is it okay to poke the baby?

I have just discovered that I can feel where my baby is and if I poke or push down he responds by kicking or moving. My sister who has been pregnant 4 times says she did this with all of her kids and it helps stimulate them, but my husband thinks I'm torturing the baby and gets upset whenever I push on my belly. I am pretty confidant that doing this does not harm my baby in anyway but I would like to know other people's opinions about it and what I can tell my husband to ure him that I am not hurting our baby.