Prove that if is a finite group in which every proper subgroup is nilpotent, then is solvable.
Let denote the class of all finite groups such that every proper subgroup of is nilpotent and is not solvable. Choose of minimal order.
Suppose is not simple; say is nontrivial and normal. Note that every proper subgroup of and of is nilpotent, and that and have cardinality strictly less than that of . Since is minimal in , and are solvable, hence is solvable, a contradiction.
Thus we may assume that is simple; that is, that has no normal subgroups. Suppose 1 is maximal in ; then has no proper subgroups and we have for some prime . But then is abelian, hence solvable, a contradiction. Thus no maximal subgroup of is trivial.
Let and be distinct maximal subgroups and suppose ; then . Note that and are each nilpotent, so that and are strict. Thus . Recall that if , then for all subsets , so that we have , hence , and thus . Since is nilpotent, by Theorem 3, is not proper and in fact , a contradiction. Thus for all distinct maximal subgroups and of , .
Let be maximal. Now is not normal in , so that by §4.3 #23, the number of elements contained in conjugates of is at most . Some element of is not contained in a conjugate of ; since is finite, every element is contained in some maximal subgroup. Thus has at least two nonconjugate maximal subgroups. Call these and .
Note that and ; moreover, no two maximal subgroups intersect nontrivially. Thus, via the proof of this previous exercise, counting the number of nonidentity elements in conjugates of and , respectively, we see that contains at least elements. Since and are nontrivial, , and has too many elements.
Thus no such group exists.