It shouldn't be a big deal. It does tend to be more of an issue for some people in BJJ than in other arts because of the contact involved - we're rolling around on the ground tangled up together, laying on top of each other, holding each other between our legs, and so on. Some of those positions can be reminiscent of other activities besides fighting.
For some people there is also the competitive factor. We spend more time sparring than in most arts. Some people seem to have a problem with cross-gender competition, for a variety of reasons.
Interestingly enough, I see a lot more women training judo than BJJ, even though the two arts are closely related. I kind of wish that the BJJ community would step up and learn from the judo community whatever we need to attract and retain more female practitioners.