It was important for both practical, strategic reasons, as well as symbolic reasons.
From a strategic point of view, Germany was located in central Europe and was the key to Western Europe. The USSR wanted to turn it into a communist centre in Europe, from where revolutionary activities could expand to Western Europe. This was tied to the USSR's aims throughout at least the first part of the Cold War - turning as many states communist, so that communist economy could actually work and that as many resources as possible were under communist use. The US was concerned to keep Germany capitalist, so as to prevent this from happening. This would allow them to develop Western Europe into a capitalist haven, where US products could find a ready market. That is why, ultimately, the two powers were happy to leave Germany a divided state - it was better than being united but in the wrong camp. Uniting it was a risk that it could turn to one camp or the other.
Symbolic reasons were important too because the Cold War was a game of appearances - each power wanted to look more mighty than the other so that all other states would gather under its protection, and the superpowers would have access to those states' markets and resources. The need to appear strong meant attracting many of the most important states in the world on their side. Germany was such a state. That is why the two powers were so adamant about the post-war arrangements. Germany was seen as a precious ally.