Witholding for married filing jointly with one child?

This year I filed taxes jointly with my new husband. We had a child last year, and both have filed Single 0 up until this year - now we have to change our withholdings at work. The worksheets we have work out to wanting us to claim much more than I think we should claim (its a W4 and an NC4 North Carolina state). My husband thinks we can claim Him Married 1 and me Married 0 to cover the child, is this right ? Neither of us are Independent contractors if that matters. Help I know nothing about withholdings, I have only ever filed Single 0! Thanks so much!