Do you think I would scare him away?

go to where you think he works and tell him you guys should go get a coffee,or for a walk.The "buddy" comment nay have been a kill;but you never know until you ask,or something.Take care,dear.