Can we be more than friends or not?

I have one guy friend with whom I tend to flirt sometime and he will do the same. Like he will call me hun or babe. i know its casual way of talking among frds but my other best frd thinks that he likes me and would like to get into my pants. Though I don't believe them coz we both knw that I like him as frd only and don't want to have anything more. These days he lives in other town so we hv been emailing each other. Sometime he doesn't respond back for a week and I get mad so he will try to explain me why he replies late. Also he invited me to go for lunch when comes back, but i am doubtful coz he likes to flirt a lot. So I don't want that to happen with me. But at the same time I don't want to ruin my frdship by not going with him for lunch. So I don't knw whats going on in his mind. I hope its more like frds level affection and not more.