The stock exhaust system uses small-diameter tubing and a fairly restrictive muffler in the effort to combat noise. We hard-core performance buffs actually like that trademark Subaru boxer motor exhaust thump, so it's an added bonus when you can bolt on ten or more horsepower and get a nicer sound at the same time.
The exhaust system is built in multiple sections. The smallest section you can replace extends from about the rear axle to the outlet. (TurboXS makes systems like this, called an "axle-back exhaust" for the 2002-07 cars.) You can also get "cat-back" systems, which add a rear-mid pipe to extend all the way to the rear catalytic converter -- the whole thing is about half the length of the car. An Invidia cat-back exhaust system is pictured here.
Cat-back exhaust systems are nice because they generally free up a good amount of horsepower, and since they are aft of the last catalytic converter, they are emissions legal. (That's not necessarily true in the Great State of California, where even cat-back exhaust systems have to have a state CARB certification.)
You can expect modest power gains from installing a cat-back exhaust -- more on older cars where the stock exhaust system was more restrictive. On the more recent Subarus, the stock exhaust system flows pretty well, so this may not produce substantial power gains until you do other modifications to the car, but as you continue to add modifications, the stock system would become more and more of a restriction.