I too have often been asked and asked myself "How do you apply a GPO to a certain security group?".  Although not completely obvious, the answer is rather simple.

The LazyAdmin has provided a great little How-To complete with screen shots.