A few years ago I read an article that mountain biking is no worse than hiking at eroding trails. A Google search turned up
this article. Judge yourself.
I've never had a problem with hikers while mountain biking. On a narrow trail I will normally slow down to a walking pace, show that I'm willing to yield the trail (they usually yield for me), and then thank them with a smile. I usually get a pleasant response.