"Punishing criminals comes first"