The simplest and most direct way to permanently delete files or folders is by using the Shift + Delete keyboard shortcut.
You're probably aware that most files have an extension such as TXT or EXE after the file name, even though these are not always displayed in Windows. File extensions are how Windows knows what to do ...