Agreed!
However, you can use Ctrl-Tab to switch to a different panel. It works differently, but to some extend it provides the same functionality.
When you are in the panel switch window, you have to keep Ctrl pressed down while you navigate to the name of the panel of your choice. You can use Tab to move to the next panel in the list, Shift-Tab to move to the previous, or use the arrow keys. When you let go of Ctrl, you're taken to the selected panel.
