First, the Baroque Catholic Church, designed by József Jung, was built between 1774 and 1777 an addition of which was the one-storey Baroque mansion built under the personal control of Antal Brunszvik Jr. and his wife Baroness Anna Seeberg. Certainly, the farmstead and other buildings of the manor were built together and reconstructed between 1784 and 1785 according to the plans of József Tallherr court architect. The wing of the Baroque building closer to the church was the ‘female section’ and the farther one was used by the men. For example, the old building part housing the Beethoven Memorial Museum corresponds to tripartite porched dwelling house. When Beethoven was visiting Martonvásár, he saw the castle in that state.

Around 1875, the son of Ferenc, Géza was the next one to rebuild the manor house again, redesigning it into English neo-Gothic style.
In 1893, the Martonvásár estate became the property of Archduke Joseph, who renovated it in 1896 and then sold it to the Dreher family the following year. During the First World War a military hospital was set up in the building. In 1923-1927 Jenő Dreher had the manor reconstructed and the gallery demolished.