In his second marriage, on 16 April 1743, he married Maria Elisabeth von Lobkowitz (1726-1786), daughter of Prince Philipp von Lobkowitz (died 1737).
The couple had
1. a son, Johann Baptist, who died early, and two daughters:
2. Maria Elisabeth Johanna Michaela Januaria (19 September 1747 – 27 January 1791), married in 1765 Count Georg Christian von Waldstein-Wartenberg (1743–1791). Ancestors of several European monarchs through Princess Maria Antonia Koháry.
3. Maria Wilhelmine Anna Josepha (13 May 1744 – 8 May 1800), married Count Josef von Thun und Hohenstein (1734-1800), known for her patronage of Mozart and Beethoven.