You want to push both. The project folder contains all the files that make up your project; the .xcodeproj file (which is actually a package, i.e., a folder that macOS treats as a file, which is why it shows up as a folder in github) tells Xcode how to put them all together.
If you're shipping an app for iOS, macOS, watchOS, or tvOS, you should upload your xcodeproj file and the project directory. If you're building a simple executable using Swift Package Manager, you can skip the xcodeproj and users can make that themselves if they need it. The vast majority of folks are in the first group!