Whitlam was prime minister from 1972-75 and Fraser from 1975-83. Both men experienced a number of spectacular resignations and sackings from their ministries.
They have called for a comprehensive review of ministerial accountability, arguing that "this principle is the bedrock of responsible government".
This is the text of the letter to the Herald-Sun from Gough Whitlam and Malcolm Fraser.
In the last two decades the constitutional principle that ministers should be held accountable for the failings of their policies or administration has been seriously undermined.
No matter how grave their failings may be, ministers no longer resign.
This principle is the bedrock of responsible government. In its absence, the capacity of the parliament and the people to hold a government to account for its actions is substantially weakened.
It is 31 years since the last official inquiry regarding the principles of ministerial accountability at a federal level. That inquiry framed the doctrine for simpler times. It could not anticipate the major changes in governance that have occurred since then.
These include an enormous growth in the powers of the executive, the now pivotal role of ministerial advisers, the outsourcing of many crucial governmental functions and the expanding influence of the lobbying industry.
The Freedom of Information Act, an important safeguard introduced in 1982, has also been undermined significantly by the practices of recent governments and restrictive interpretation by the courts.
The Canadian and British governments (of different political persuasions) have recently taken steps to strengthen ministerial accountability. They have recognised its fundamental importance and the need to re-evaluate and fortify it so that the representative democracy may function as it should.
We believe it is critical that this issue is addressed in the forthcoming national election and then acted upon by whichever party forms the new government.
We take this opportunity to urge all political parties to commit to the establishment of an independent and comprehensive review of the operation of ministerial accountability so as to modernise and strengthen it.
This is a matter that transcends party politics. It goes to the very heart of the way we are governed.
Malcolm Fraser and Gough Whitlam, former prime ministers.