I hear this alot, and I wonder: is that because I frequent developer forums, where the users are mostly male? I'm a guy, but have really tiny hands (smaller than my wife's, and most every girl I ever dated) I have an S3 (and S5 on order), and have had most iPhone models. The larger phones really are uncomfortable for me, but I suppose the same is true if you reverse it (smaller phones are uncomfortable for larger hands ... I heard the reason why Shaq sucked at the free throw line was because to him, shooting a basketball was like a normal person trying to shoot an orange)
You're not alone, I have noticed as soon as you get out of most tech circles and look at mostly women only things, that is nurses/etc... they tend (tend is key here not always) to prefer iphones.
Funny part is they seem to hate the 5/5s cause its too tall and still prefer the 4s. I know a girl that downgraded back to her 4s because the extra height on the 5 was too much.
With my girl hands I pretty much agree though, I do miss the smaller height of the 4s over my 5. I think when people talk about what phones they prefer they should also bust out their glove size. Mine is a 7, and any phone > 4"ish starts to really frustrate me when I try to text one handed.
While we're generalizing I seem to know many women who will then outfit their Iphone 4(s)/5(s) with big bulky Otterboxes or other huge cases that negate the natural size advantage of the iPhone.