When we earn money from our investments, Uncle Sam wants his cut, too—just like with the rest of our income.

But since investment gains—meaning what we make when we sell an investment for more than we paid for it—aren’t always taxed the same way as our paychecks, it’s not quite as straightforward.

1. Timing matters.

Investment earnings are grouped as short-term or long-term.

A short-term investment is one you owned for less than a year. When you sell it, the earnings are subject to short-term capital gains tax, which is the equivalent of ordinary income tax.

Long-term capital gains—on investments you owned for more than a year—are “given a bit of special treatment,” says Barbara Taibi, partner at accounting firm EisnerAmper in Iselin, N.J.

Translation: The highest rate is 20 percent—and that only applies to taxpayers whose taxable income is well into the $400,000s.

The other rates are 0—yep, zero, if you’re in the 10 or 15 percent income tax bracket—and 15 percent for the rest of us, till we start raking in $400,000 a year.

2. It’s not just gains we’re taxed on.

You also pay taxes on dividends, those periodic payouts of earnings that some companies offer shareholders. But not all dividends are created—or taxed—equally.

“Qualified dividends” are paid out mostly by U.S. corporations, plus some foreign companies. But in order to be considered “qualified,” you must have held the shares in the company that issues the dividend for a minimum amount of time during what’s called the “holding period.”

Essentially, if you buy a stock after the dividend has been announced, you’ll likely miss out on having “qualified” dividends. But once you’ve held onto it for a few months, the remaining dividends should be qualified. The good news is you’re only taxed at the long-term capital gains rates for those.

Dividends that don’t meet those requirements are considered “nonqualified dividends,” and taxed as ordinary income.

One more: If you invest in mutual funds, you could have to pay taxes—even if you haven’t sold your holdings, Taibi says. “It can be a surprise for some investors to realize that they may owe tax if their fund manager sold holdings at a gain, even if the investor has only entered into the fund at mid-year or near the end of the year.”

3. Not all accounts are taxed equally, either.

Taxable accounts include traditional investment (or brokerage) accounts, bank accounts and money market mutual funds. These are all subject to capital gains taxes.

Meanwhile, a “tax-qualified” account—like a 401(k), IRA and even a 529 college savings account—does not require capital gains taxes to be paid, says Katharine Perry, associate financial consultant at Fort Pitt Capital Group. So you won’t be taxed on investments you sell (as long as you don’t withdraw that money).

You will pay ordinary income tax on the money you take out of a 401(k) or traditional IRA account in retirement, though, unless you’re using a Roth IRA. With a Roth, you contribute money that’s already been taxed—unlike a 401(k)—but you won’t pay taxes when you take the money out of the account in retirement. Just keep in mind, there are income limits to contribute to those.

This article was updated in January 2018.