Because your goal is to REMOVE the content, you have to remove the PHP code with its HTML counterpart. In fact, it's irrelevant whether you remove the CSS or not. You might use that styling for something else.
Anders' solution will work well if your goal is to HIDE the content instead of removing it and as he said, you can later on put it back if you like.
If you are absolutely sure that you won't need to use that ever again (heck, you can just copy it back from the original theme) and that you also won't need to use the styling, then go ahead and remove both the CSS and the PHP/HTML to keep everything clean. I would personally do that since I like my code to be looking good :)