The simplest and most direct way to permanently delete files or folders is by using the Shift + Delete keyboard shortcut.

Just select the files and folders that you want to delete and hit these keys. A dialog box will pop open asking you to ...