He probably just needs some time to get settled. Where was he before he was with you? He may have just had a bad time before you got him. Imagine the stress of having to adjust to yet another environment. Perhaps having a little child around right away, before you two have bonded, is a bit too soon.
I adopted a mature foster hound last year and she had some little growlies and grumbles the first couple of days but nothings serious. I belive she was just finding her way. I spend serveral days with her, bonding and petting her and giving her lovies. They need to feel loved and secure.
Things will work out fine. Give him a calm and non-threatening environment, lots of affection and attention. Treats

and a little "space" to call his own. Maybe put his dog bed next to yours and reach down and pet him. Let him know he has a forever home now. He'll be fine.
