Maintaining population diversity by maintaining family structures