A good friend in graduate school, who was much better schooled in the classics than I, once remarked that "everyone is either a Platonist or an Aristotelian." Plato, as I understand it, sought ideal forms underlying the imperfect reality that we see, but was prone to imposing philosophical prejudices on uncooperative facts. Aristotle was happy to catalog the diversity of Nature, with less need for organizing principles, but could be accused of "stamp collecting." Even after a couple of millennia, differences as to what constitutes an interesting or beautiful science continue cause mutual incomprehension between scientists from different specialties.
Some of the differences are inherent in the types of experiments available. For example, "historical" sciences like astronomy and archaeology are pretty much limited to learning what happened in the past. They can invent better ways to learn about it, and they can devise new explanations, but they can't change it. In contrast, physics and chemistry--and to a growing extent biology-- pose hypothetical questions, and test their understanding by predicting the responses to new, previously untested situations. Experiments make it harder for researchers to fool themselves by making up plausible, but untestable, "Just So Stories."
Other differences reflect the nature of the subject matter. Physicists are fond of invoking "Occam's razor," which states that among possible explanations, the simplest is probably the best. This works pretty well in physics, but many physicists seem to regard Occam's razor as a law of nature itself, rather than a heuristic or guide for further tests.
In biology the situation is quite different. Frequently, if more than one mechanism might explain a biological phenomenon, they're all true, with maybe a couple of others that no one thought of yet! Perhaps the mechanisms act in different species or in different situations, or they may act cooperatively or competitively in a single organism, as part of a complex web of interactions. It's still important to avoid unnecessary complications in the explanations, but Occam's razor is a lot less useful for biology, and interpreting the experiments, and devising new ones that really test assumptions, can be a lot harder.
Quantitative mathematical experiments are increasingly important in biology, and genetic and other tools let researchers test the response to ever more perturbations of biology, rather than merely observing what evolution has produced. But the starting point is still what evolution has produced: complex interacting systems with many components playing multiple roles. For these systems, trying to cast everything in the simplest possible terms doesn't always work.