The short answer is that if 1 were prime, then numbers wouldn't have unique prime factorizations (Fundamental theorem of arithmetic). We'd have to state the theorem as "unique, up to factors of 1." Which is silly. So we don't do that.
Here's another example of this phenomenon, which goes under the general name too simple to be simple. A Graph is, loosely, a bunch of dots connected by a bunch of lines. A connected graph is, loosely, a graph where you can get from any dot to any other dot by traveling along lines. Now, I claim that every graph has a "unique prime factorization" into connected graphs; these graphs are called its connected components. For example, the following graph has 3 connected components:
In order for the "connected factorization" of a graph to actually be unique, though, the empty graph (no dots, no lines) must not count as being connected. The empty graph is like 1 here because, in the same way that when you multiply a number by 1 it doesn't change, when you put the empty graph next to a graph it doesn't change. Formally, they are both Identity elements for certain operations (namely multiplication and the disjoint union of graphs, respectively).