November 2018 update: competitive banks are paying 2.25+% interest on money-market type savings accounts, FDIC insured, compared to about 0.20% for average banks.
Why earn only 0.2% when you can earn 10 times as much at a competitive bank? In these money-market type accounts, you can deposit or withdraw your cash at any time, in any amount, without penalty.
To find out the best competitive interest rates, go to bankrate.com and click Savings/Checking, then click Savings/Money Market, then choose Sort By: APY (Annual Percentage Yield). You get a list of banks, highest interest rates first.
The ones paying the highest interest are probably not ones you have heard of. They tend to be smaller banks, maybe not present in your state. But that should not keep you from using them. They are all insured by the U.S. government (FDIC, Federal Deposit Insurance Corporation), so your money is safe. You can keep your existing bank account and transfer your savings there online or by mailing a check, and then transfer back to your checking account online, or for some accounts, write a check when you need the money.
To make sure that the bank is insured by FDIC, go the FDIC website, fdic.gov, click Bank Find, and enter the name of the bank. If your bank fails, the FDIC will merge your account into a new bank or pay you back. You won't lose anything in the bank failure.
November 2018 Update: I found that money market fund interest rates have gone up quite a lot in the past several months. They are now competitive with the best banks. Money market funds pool their investors' money and purchase interest-bearing instruments. These funds are not government-insured, but some invest in extremely safe US Treasury obligations. There is no safer investment.
For example, the Vanguard Treasury Money Market Fund (VUSXX) is earning 2.10%. Because it is a US Treasury fund, earnings are exempt from state and local income tax, which is important for a high-tax state like California or New York. The similar Vanguard Government Money Market Fund (VMFXX) earns nearly the same rate and has a lower minimum deposit ($3K vs $50K). These funds are good alternatives to banks.
For bank deposits, if you are so fortunate to have savings that exceed the FDIC insurance limit of $250,000 per bank and account registration, you can divide up your cash between different banks, or register multiple accounts at the same bank with different ownership combinations (for example, with different co-owners or beneficiaries). Keep no more than $250,000 in each account for maximum coverage. For details, see the FDIC website.
Similarly, SIPC (Securities Investor Protection Corporation) protects your stock brokerage assets up to $500,000 per account registration (but only up to $250,000 of assets in cash). Therefore, you should keep no more that $500,000 in assets in each brokerage account registration. (This protection applies to failure of the brokerage, not losses due to your stocks going down in value.)