Depends.
If you are doing forms, starting point is up, by your ear. If you are in a sparring situation, starting position should be guard position, fists in front of your face.
If in doubt, always ask your instructor. In most cases he will gladly explain the technique further to you. Because there is actually a system to that madness: You practice a big circular motion across your body to block a straight line attack, like a punch or a front or side kick. It is important IMHO to get the motoric down because if you don't swipe, but stab at the attack you wrench your wrist. And yes, painful experience!