Oh, no, you should just remember certain relationships around the head and you won't need reference anymore. For example, to find the upper connection of the ear to the head you can extend a horizontal line from the corner of the eye. Of course, you have to imagine it in perspective, like you would draw a band around the head. The lower part of the ear may then roughly be on the height of just under the nose, I imagine... hmmm... I should try and sketch this, but just google "facial proportions" and you should find all you need to learn.
Also, consider orientation/rotation of the ear and placement on the radial axis... ears are kind of behind the middle of the head toward the back of it a little bit. But you can style that to your liking, of course.
Above all else, examine yourself, look at yourself in the mirror and just bring things into relationships with each other. Where is your ear compared to the height of your eye. Where is it compared to the end of your jaw and so on... is it tilted backward a bit... just study yourself!
Make yourself that inner diagram of relationships and consider perspective. You will master it quickly, I am sure!