I posted a blog here about how to modify OpenOffice.org Calc to make the Delete key just delete, and another key like the Backspace key bring up that little delete window.
Here's another little modification. A lot of folks who use Excel are used to pressing Enter or Return to go down one row and all the way to the beginning of the row. If you're on E4 and press Enter, then you are suddenly at F1.
When you press Enter in Calc, however, you just go down one, like from E4 to F4. You can go under Tools > Options > Calc > General and change what the Enter key does, but you can't change it there to doing it the Excel Way.
However, to go to the first cell of the next row almost as quickly in Calc as in Excel, you can press Enter, then Home. On my keyboard they're pretty close together. Enter takes you down one cell, and Home takes you all the way to the left of that row.