I personally don't think any of these are the hardest. In my opinion it should be golf and/or rugby. The games themselves are quite easy, simple to play, but INCREDIBLY hard to master, as very few people in the world have played it.
Playing soccer at a decent level, wasn't super difficult, was fun, harder as you get higher up(as a goalie), basketball and baseball/hockey are all incredibly hard to play/master. Sports are easy and simple to play with friends, but playing them all at a high competitive level is much different.
I choose rugby because you have to be all around athletic, and strong and be in great shape, compared to other sports which you have to be in shape, but the position you play isn't as difficult. An example could be kicker/punter in football, simple as hel to play, but a lot of fun -- if football is the hardest sport then you have an amazing athlete on your hands, right, right?? ^^
The question itself is very specific and hard to answer since all sports have different positions and you have different aspects of them that make them so unique and ridiculously hard to play
