The idea that embassy grounds are sovereign territory was put forth by
Hugo Grotius, who in the early 1600s wrote essays that became the basis for international law. Under the concept of sovereign territory, actions taken against an ambassador or other diplomat or actions taken against the embassy itself either directly or indirectly, can be viewed as actions taken against the country that the ambassador, diplomat or embassy represent.
That military bases are sovereign territory derives from the treaties and contracts that allowed the military base to be set up outside of territory controlled by the contry in question. Guantanamo Bay, for example, was created in 1903 after the Spanish-American War. It is governed by agreements made with the American controlled Cuban government after the US overthrew the Spanish governor; those agreements give the United States full sovereign rights to the patch of land upon which the military base stands.
With regards to your question, the courts have generally made a distinction between normal law and military law. Many of the rights guaranteed by the Constitution are fully protected only under normal law; some are curtailed or even absent under military law. This is not a new distinction, but one that goes back to the very beginning of the country. Changes in this distinction are left to Congress and the courts have been very reluctant to interfere. On military bases, military and not normal law applies. There are areas where even military law places limits, which is why some people are shipped off to be rendered (like fat in a fire) completely outside of US control.