It depends on what you enjoy doing. I love Mechanical and I was never in Eng Phys so I cannot say if it is harder or not. Eng Phys and Mechanical study different things.
In Mechanical we take a fair share of design courses along with a lot of fluids, thermo-fluid courses. You can view a description of the courses you are required to take in each program from the
course calendar
I was enrolled in international studies (which is an extension of the society program). It was alright however I was not learning much out of it therefore I dropped it and decided to complete a minor instead. The nice thing is that most of the courses I took in society counted as electives and therefore I completed all the electives I needed to take in a straight B. Eng degree. Therefore to answer your question, yes you can drop management or society without any issue.