Karera/彼ら/かれら usually IS gender specific, although it's rarely used. Normally you'd just say 'Person X-san'+tachi (where Person X is the person you know) to indicate a group of people. Karera can sound a bit distant and rude, I believe, as can a lot of pronouns ('kimi', 'anata', et al) I'm not sure there is a specifically female equivalent, so karera is just used. I suppose it's somewhat like the way 'man' is used in English; 'one small step for man' doesn't literally mean a single man, nor does it mean only men are affected.
Kanojo/彼女/かのじょ literally means 'she' or 'her'. It tends to be used as 'girlfriend', though. Again, pronouns like this can sound distant or rude, especially if you know the person's name. Onna no hito/女の人/おんなのひと could be directly translated as 'person of female [gender]'. To my ears, kanojo is more like 'girl' and has a childish ring to it. Onna no hito is more like 'woman' and is more formal.
Same deal, with the 'boy' and 'man' connotations, respectively. The word for 'boyfriend in Japanese is kareshi/彼氏/かれし, though, and there's not that element of confusion there as there is with kanojo.
This post was edited by Razzattack on Dec 9 2011 12:18pm