To get to the App Store on your Apple TV, look in the top menu bar on your Apple TV 4K or Apple TV HD.
Sideload apps: Follow developer instructions to add to your phone. Not common. Risky: Jailbreak an iOS device and load apps to it. This article explains how to get apps that are not in the App Store ...
Your iPhone apps should update automatically, but if they aren't, you can check for updates manually through the App Store.