Hex Hall by Rachel Hawkins.
Sophie Mercer, age 16, is a witch. She’s lived all her life with her normal mother and never met her father, the warlock who passed on his powers. Like all Prodigium, Sophie’s had her powers since about age 12, but only now has she flaunted them in front of humans enough to be banished to Hecate Hall. Hecate Hall, popularly known as Hex Hall, is a boarding school/juvie center for magical teens off the coast of Georgia. The term Prodigium encompasses witches, were, shapeshifters, fairies and vampires, and Hex Hall has them all. But just because they’re all magical doesn’t mean they all get along, as Sophie finds when she’s assigned to room with Jenna, the school’s lone student vampire (though Lord Byron is one, and teaches English here.) Right away, Sophie finds herself in the middle of high school social drama. She’s saved from an out-of-control werewolf by the cutest boy in school, Archer. Archer’s dating the superficial but beautiful and popular Elodie, who wants Sophie, whose powers are strong if undeveloped, to join her coven, which needs four members. The coven’s previous fourth member, Holly, died last year under highly suspicious circumstances – drained of blood, with two small holes in her neck. It’s widely suspected that Jenna, who was Holly’s roommate, is responsible, but Jenna is scared and depressed and seems to be sincere when she claims that Holly was her best friend. Meanwhile, Sophie is finding that she knows nothing about the magical world that all the other kids have been raised in. She didn’t even know that her father was head of the Council, and as the Council is responsible for banishing teens and some adults to Hex Hall, that makes her pretty unpopular. Also, though she’s told that she has potential for great power, she can’t seem to control it. The plot thickens as more witches on campus start turning up dead or nearly dead. But if Jenna is innocent, who is really responsible? Were Elodie and her coven successful in their attempt at raising a demon last year, or has L’Occio del Dio, one of the ancient groups founded to wipe out all Prodigium, found a way to penetrate Hecate Hall’s formidable magical defenses? If Sophie doesn’t both learn how to control her powers and find out who is behind the killings soon, it will be too late: the killer is targeting the school’s most powerful witches, and she’s next up on the list. This reminded of Buffy in the very best way, with the felicitous combination of magic, almost-typical high school experiences and snappy dialogue, even if Sophie is on the other side of things. As an example, at one point Sophie is trying and failing to charm herself a ball gown, and describes one failed attempt as looking like “the really slutty bride of Cookie Monster.” Though it has spycraft instead of magic, the Gallagher Girl books have a similar combination of boarding school setting, danger and fun dialogue.
The second book in the series, Demonglass, intensifies everything, as Sophie travels to London with her father and has even less idea who might be trustworthy. But if Sophie might be somewhere on someone’s hit list in Hex Hall, in Demonglass, she’s clearly at the very top of at least one and possibly more, with, you know, the fate of all Prodigium instead of just the Hex Hall students at stake.