Administrative Review Council

1                 Introduction

1.1           The Administrative Review Council

The Administrative Review Council was established under the Administrative Appeals Tribunal Act 1975 as an integral part of the Commonwealth system of administrative law. The Council advises the Attorney-General on a broad range of matters related to Commonwealth administration.

The inquiry into the use of automated assistance in administrative decision making is relevant to two of the Council’s functions pursuant to s. 51 of the Administrative Appeals Tribunal Act:

(aa) to keep the Commonwealth administrative law system under review, monitor developments in administrative law and recommend to the Minister improvements that might be made to the system; and

(ab) to inquire into the adequacy of the procedures used by authorities of the Commonwealth and other persons who exercise administrative discretions or make administrative decisions, and consult with and advise them about those procedures, for the purpose of ensuring that the discretions are exercised, or the decisions are made, in a just and equitable manner.

Section 51 of the Act is reproduced in Appendix A.

1.2           The project

Primary decision making is now ‘big business’ for many government agencies. For example, in 2002–03 Centrelink conducted 4 402 527 reviews of eligibility and entitlement.[1]

Expert systems are increasingly being used by government to make, or help make, administrative decisions. Advances in technology and the increase in the quantity and complexity of administrative decisions have led some government agencies to see expert systems as a means of facilitating and possibly improving decision-making processes.

Expert systems have the potential to make decision making more accurate and consistent. They also have the potential to be more cost effective, particularly for agencies that must make many decisions. Additionally, the technology can be used in the development of new service delivery options.

Despite the potential benefits offered by expert systems, however, care must be taken to ensure that their use supports administrative law values and that administrative review mechanisms keep pace with technological change.

In view of the growing use of expert systems, the Council resolved to inquire into their use in administrative decision making. It wanted to look at the following:

·         current use of expert systems by the Commonwealth

·         how and by whom the systems are designed and used

·         how the systems operate and how they are tested to ensure that they reflect the relevant legislation

·         opportunities, if any, for independent scrutiny of the systems

·         the features of an optimal system

·         the implications of the use of expert systems from the perspective of the administrative law system and its decision-making procedures

·         aspects of access and equity that arise in relation to expert systems.

1.3           Consultation

The Council released its Automated Assistance in Administrative Decision Making issues paper in 2003. The paper considered the implications of the use of automated systems for primary decision making—such as the opportunities for greater accuracy, consistency and efficiency and the potential effects on the skills of decision makers. It also discussed how automated systems might best be designed and maintained and how their use affects administrative review of decisions.

In preparing the issues paper, the Council conducted a stocktake of current and proposed use of expert systems in administrative decision making by Commonwealth agencies. Appendix B presents the stocktake results. Following the stocktake, the Council decided to engage in further discussions with Comcare, the Department of Veterans’ Affairs, the Department of Family and Community Services, Centrelink and the Australian Taxation Office, all of which make extensive use of expert systems in high-volume decision-making areas. On the basis of this information, the Council used the issues paper to put forward its preliminary views about the use of expert systems and to seek feedback.

In addition to obtaining information through submissions (see Appendix C), on 12 November 2003 the Council held a forum in Melbourne to further consider the uses of expert systems. The forum was attended by about 60 people from a broad range of interest groups, including government, business and community organisations.

1.4           This report

Although the primary focus of the Council’s issues paper was rule-based systems, in this report the Council has broadened its focus to encompass other forms of expert systems used in administrative decision making. As the Department of Veterans’ Affairs observed in its response to the issues paper, regardless of the type of expert or decision-support tool used in administrative decision making, the primary consideration of ensuring that decisions ‘accord with the law, are fair and consistent and have appropriate privacy and audit controls’ remains relevant.[2] Further, continuing advances in technology mean it is possible that other expert systems that do not come within the definition of a rule-based system will be developed.

Having regard to the responses to its issues paper and to the information gathered through further consultations with respondents to the issues paper and others, in this report the Council focuses on the following:

·         the sorts of administrative decisions best suited to the use of expert systems

·         the advantages and disadvantages of using expert systems in administrative decision making

·         best-practice principles for developing and operating expert systems in administrative decision making

·         the need for expert systems in administrative decision making to comply with the five values the Council has previously identified as crucial elements of the administrative law system

–         lawfulness

–         fairness

–         rationality

–         openness or transparency

–         efficiency.

These values are coincident with concepts of administrative justice, which are seen to include four basic requirements for just decision making in a society governed by the rule of law—lawfulness, fairness, rationality and intelligibility.[3] Accessibility, affordability and timeliness are also said to be relevant from the perspective of the applicant and the general community.[4] Among the mechanisms for achieving administrative justice are education of decision makers, internal review by superior officers, and external administrative review.[5]

2                 Expert systems

Since the invention of the first digital computer in 1945 there have been astounding developments in computer technology. Artificial intelligence is an area in which there has been much progress, and one element of it—expert systems—has been used in a variety of fields, among them public administration. The Commonwealth has invested large amounts of money in developing expert systems to assist in the performance of its executive tasks, among them administrative decision making.

2.1           Types of expert systems

The Macquarie Dictionary defines ‘expert systems’ as computing systems that, when provided with basic information and a general set of rules for reasoning and drawing conclusions, can mimic the thought processes of a human expert. Another definition is computer programs constructed with the assistance of human experts and capable of functioning at the standard of human experts in a given field.[6]

A legal expert system is a type of expert system. It can be defined as a computer program that performs tasks for which the intelligence of a legal expert is usually thought to be required—whether the legal expertise be that of a lawyer or that of a non-lawyer with expertise in a particular area of the law.[7]

Expert systems are distinct from decision-support systems, which provide information to enable a human to make a decision without actually indicating what the outcome should be. It is important, however, to be aware that expert systems can be used as decision-support systems, as well as actually making a decision. They can provide decision support by guiding a decision maker through the relevant legislation and providing explanatory and supporting materials such as legislation, policy information and case law.

One way of classifying legal expert systems is according to the way in which such a system represents the law that it contains. There are two broad ways in which the system can represent the law—through constructed knowledge and through learned knowledge.

2.1.1           Constructed knowledge

A legal expert system represents the law through constructed knowledge when the law is interpreted and constructed by legal experts and then represented as explicit rules in the expert system.

Rule-based systems are the main type of legal expert system that use constructed knowledge. They involve the modelling of rules accompanied by an ‘engine’ that automates the process of investigating those rules by interacting with users to establish client details. Rule-based systems that model legislation perform two functions:

·         They interrogate the user, identifying the next relevant legislative matter and closing off irrelevant paths as they go.

·         They draw conclusions, applying the structural logic of the legislation on the basis of information collected from the user.[8]

The systems can operate online, with an officer asking an applicant questions upon prompting by the rule-based system. The questions the system asks give effect to the legislative requirements for the decision at issue—for example, eligibility for receipt of a particular benefit—and the relevant policy. The applicant and the officer can view on the computer screen commentary about the questions, the source legislation, relevant cases and the agency’s policy. Explanatory material can be provided using devices such as scrolling, tabs and hypertext links.[9]

Among the rule-based systems that model legislation are STATUTE Expert, developed by SoftLaw, and jnana, developed by Jnana Technologies Corporation. Commonwealth and state government agencies use STATUTE Expert in administrative decision making; legal firm Blake Dawson Waldron uses jnana to evaluate whether advertisements comply with the Trade Practices Act 1975 and other relevant laws and regulations and to determine whether the permit requirements in various jurisdictions have been satisfied in advance of trade promotions.

Rule-based systems can also be used to model complex rules that do not have a legislative basis. In the insurance market, for example, they can assist with quoting and determining insurance premiums. Other uses of rule-based systems are diagnostic risk assessment and airline scheduling. XpertRule, developed by ATTAR and used by the Australian Taxation Office, is a rule-based system that does not have a legislative basis.

2.1.2           Learned knowledge

A legal expert system represents the law through learned knowledge when the law has been ‘learnt’ by the system. The computer itself generates the rules from a series of examples or precedents. [10] Case-based reasoning and neural networks are examples of systems using learned knowledge.

Case-based reasoning relies on inductive processes whereby the system discerns patterns from cases with different characteristics.[11] For example, FINDER is a case-based system developed to give advice on the law of trover, an area of case law involving the rights of people who find lost chattels. FINDER provides advice by using a database of leading trover cases and a set of attributes that were of legal significance in those cases.[12]

Neural networks try to replicate the processes of the human brain.[13] They scan a series of decisions and the weight attached to each aspect of a decision and then generalise a rule. NetRisk, which is used by the Australian Taxation Office, is an example of a neural network system. It is a debt risk–profiling application that uses ‘client circumstances and past behaviour to predict the level of intervention required to best resolve an outstanding [tax] obligation’.[14]

2.2           Expert systems identified in the Council’s stocktake

As noted, in preparation for its issues paper, the Council did a stocktake of Commonwealth agencies’ current and proposed use of expert systems. It asked agencies the following questions:

·         Are expert systems currently used within your agency to make administrative decisions? If so, what decisions are the systems used to make?

·         Does your agency propose to use other expert systems in the future or to expand the categories of decisions currently made using existing expert systems? If so, please provide details.

The stocktake results showed that the most commonly used expert system was the rule-based system. Appendix B presents the results.

In the course of its consultations the Council received from Centrelink information about one of its pilot projects, a rule-based system known as the Edge Project.[15] Centrelink subsequently advised the Council that the project was not proceeding in its current form. Despite this, the experiences of Centrelink and the software producer, SoftLaw, remain relevant; as a result, the views expressed during consultations and in submissions from both Centrelink and SoftLaw are referred to in this report.

2.3           Terminology

In a number of responses to the issues paper questions were raised about some of the terminology the Council used in relation to expert systems.

One agency said the issues paper did not go far enough in defining expert systems, as opposed to business systems. It suggested that many of the examples provided by other departments seemed to be business systems with embedded business rules and not true expert systems.[16]

Another comment was that the Council’s definition of expert system was incorrect because expert systems modify their own behaviour over time. On this basis, it could be argued that static rule-based systems are more generally business applications, rather than expert systems.

The Council acknowledged in the issues paper that there are varying views about the term ‘expert system’. Indeed, it has been argued that the term is ambiguous.[17] The Council appreciates that there are various ways of classifying the different types of expert systems; for example, legal expert systems can be distinguished from expert systems that use business rules rather than legislative rules or rules based on policy or case law. Nevertheless, this report maintains the definitions the Council adopted in the issues paper in relation to expert systems.

2.4           Government and expert systems

Government has executive and administrative functions. One of its executive functions involves formulating policy on various matters. Policy can be debated and implemented through legislation; in turn, policy emerges to assist with the implementation of legislation. Administrative decision making involves application of that legislation and policy to an individual’s circumstances.

Administrative decision-making processes play an important role in the relationship between government and the citizen. At the micro level, adherence to sound administrative processes ensures that decisions are properly made and that the rights of applicants are protected in their dealings with government agencies. At the macro level, adherence to sound administrative decision-making processes improves the quality of government decision making generally.[18]

With advances in technology and the increase in the quantity and complexity of administrative decisions, technology can be used to facilitate and possibly improve the administrative decision-making process. Use of technology in this way is part of a broader ‘knowledge-management’ framework.

‘Knowledge management’ has been defined in a number of ways. For example, it has been used as ‘creating, using, sharing and learning from knowledge in order to improve an organisation’s capacity to act’[19] and as ‘getting the right knowledge to the right people at the right time to serve the right objectives’.[20] It is an interdisciplinary framework intended to help agencies engage in the wider information, or knowledge, economy.[21] Expert systems are one element of that knowledge-management framework: they provide a means whereby agencies can set out a consistent interpretation of complex legal rules and policy.

2.5           The suitability of expert systems for administrative decision making

It has been suggested that the use of expert systems other than rule-based systems in the administration of legislation may not be consistent with the principles of administrative law.[22] For example, case-based reasoning poses problems for administrative decision making because it is fact-based, not conceptual, and involves the system discerning patterns from cases with different characteristics. It has been argued that there is a fundamental difference between statutes and case law and that lawyers apply statutes in a rule-like fashion, whereas they reason with cases by arguing about their similarities and differences.[23] Given that administrative decision making is predominantly based on legislative rules rather than case-law, case-based reasoning systems are not generally suitable for administrative decision making.

Neural networks are inductive and over time distinguish the relevant combination of facts. But these systems do not easily provide reasons for their decisions, which means they are not generally suitable for administrative decision making.

The Council’s research indicates that the use of rule-based systems in Commonwealth administrative decision making is more widespread than is the use of other expert systems.

2.6           The level of assistance expert systems can provide for administrative decision making

Expert systems are not new. For example, Centrelink has used rate calculators for many years to determine statutory rates of payments and benefits. Changes in technology are, however, offering greater opportunities for using expert systems throughout the administrative decision-making process.

The Council found that expert systems can be used in different ways in administrative decision making. For example:

·         They can be automated to make the decision.

·         They can recommend a decision to the decision maker.

·         They can guide a user through relevant facts, legislation and policy, closing off irrelevant paths as they go.

·         They can be used as a decision-support system, providing useful commentary—including about relevant legislation, case law and policy—for the decision maker.

The systems use a combination of these decision-making and decision-support functions. For example, an expert system can provide useful information for the user (a decision-support function) as well as direct the user through particular paths of the legislative provisions, depending on the answers the user provides (a decision-making function). Accordingly, at one end of the spectrum they can automate a decision and at the other end they can be used as a decision-support tool.

The level of assistance expert systems provide in relation to administrative decision making can raise questions about particular administrative law values. For example, it is important that an expert system used to identify the relevant law for a decision maker—that is, a basic decision-support function—correctly identifies that law. But an expert system that is also used to identify the relevant facts, guide the user through the relevant legislation and then make a determination—in relation to a person’s tax liability, for example—can raise additional questions about whether the system has identified the relevant considerations and whether it has fettered any discretionary power of the decision maker.

Care must therefore be taken to ensure that the use of expert systems supports administrative law values. Chapter 4 discusses further the potential advantages and disadvantages associated with using expert systems in administrative decision making.

3                 Are expert systems suitable for use in all administrative decision making?

There are differing views about the desirability of using expert systems for all types of administrative decisions.

It has been argued that legal reasoning necessarily involves resort to social context and purpose and that as a result application of the law is not suited to an expert system. Proponents of this view suggest that the development of legal expert systems should be abandoned and the focus should move to computerising more mechanical tasks such as retrieving legal information.[24]

A second view is that expert systems have a legitimate role in making simple judgments and otherwise guiding people who are making decisions that involve multiple factors or the use of discretion.[25] This view emphasises the benefits of consistency in decision making, which expert systems can bring.

Some advocates of the use of expert systems are more ambitious: they suggest that other programming techniques—such as case-based reasoning and neural networks—could be added to rule-based systems to improve the ability of computers to make sophisticated decisions.[26]

Having considered the expert systems identified during the stocktake, the Council concluded that expert systems can operate to assist in administrative decision making—provided administrative law values are taken into consideration. The main dangers associated with the introduction of expert systems for decision making will come from how the systems are used, rather than from the systems themselves.[27]

The level of assistance expert systems should provide depends on the type of decision involved.

3.1           Non-discretionary decisions

The law operates in a social context and is not easily reduced to neat rules. Even a prescriptive legislative provision, which may seem to be already reduced to simple rules, can involve reference to case law in order that a human expert can interpret the provision.

Given the difficulties that can be involved in constructing an expert system that is capable of making a decision based on interpretation and representation of the law, the Council considers that using an expert system to make a decision—as opposed to helping or guiding a decision maker in making the decision—would generally be suitable only for decisions involving non-discretionary criteria.[28]

3.2           Discretionary decisions

One thing that became apparent during the course of the Council’s project concerns the level of assistance expert systems should provide in relation to decisions involving the exercise of discretion by the decision maker. It is fundamental to administrative decision making that, if a decision involves the exercise of discretion, the decision maker must exercise that discretion personally and not be fettered in doing so.

3.2.1           Which decisions involve the exercise of discretion?

The Social Security Appeals Tribunal submitted that although there are few true discretions left in the social security law—for example, the discretion of the Secretary of the Department of Family and Community Services in relation to granting Special Benefit—there are many social security laws that involve the development of an opinion or the exercise of judgment, or both.[29] Similarly, it has been suggested that discretionary powers are not as common in modern legislation and that many agencies characterise broad areas of judgment as discretion.[30]

Where identical findings of fact can lead to different legitimate outcomes, this indicates the presence of discretion. [31] Different outcomes can arise because different weights are attached to the relevant factors.

There are, however, different concepts of the meaning of discretion, ranging from narrow discretion to unfettered discretion.[32] For the purposes of this report, the Council adopted a broad view of discretionary decisions. References to the exercise of discretion and discretionary decisions thus include the following:

·         decisions where the decision maker has a wide discretion to choose between alternatives—for example, the power of the Minister for Immigration and Multicultural and Indigenous Affairs to substitute a more favourable decision for a decision of the Refugee Review Tribunal if the Minister ‘thinks it is in the public interest to do so’[33]

·         decisions where, although the decision maker’s power is more limited, the decision maker must still exercise their judgment in order to make a decision.

An example of the latter, narrower type of discretion is a decision that involves the question of whether to treat a person as a member of a couple for the purposes of determining eligibility for social security payments. Although a number of factors must be taken into account when determining whether a person is a member of a couple, the decision maker uses their own judgment to weigh up all the circumstances of the case and make the decision.[34]

3.2.2           The views of respondents

There was a general view among those who responded to the issues paper that expert systems should not be used in such a way as to fetter the decision-making power.

SoftLaw argued that a rule-based system that exhaustively and faithfully models the legislation will be capable of taking into account any circumstances contemplated by the legislation, allowing for residual discretion where the criteria are not met.[35] It further argued that rule-based systems can improve administrative decision making by properly modelling a subset of lawful circumstances, guiding staff through the various matters for consideration, and providing links to relevant legislation and other commentary.[36]

A number of respondents raised the question of using expert systems as decision-support tools in relation to discretionary decisions. Centrelink and the Department of Family and Community Services submitted that their expert system’s ability to highlight discretionary decisions for decision makers was desirable.[37] Similarly, the Department of Veterans’ Affairs advised the Council that issues to do with discretion are dealt with through the support commentary that accompanies each question in the expert system.[38] However both these agencies limit officers’ ability to override the ultimate result of the expert system, so it seems that the expert system is only being used as a decision-support tool in relation to the aspect of the decision that is discretionary.[39]

National Legal Aid advised the Council that legal aid commissions that are currently operating rule-based systems are using them as decision support tools[40]:

It is because many of the criteria in the guidelines for legal aid are not strict and the range of circumstances that individuals find themselves in are not easily reduced to business rules that Commissions consider any process to move to Rules Based Systems, even for use as decision support tools, will need to be iterative.[41]

The Social Security Appeals Tribunal noted that, although an expert system might lead a person through the relevant provisions and refer to policy and case law, ultimately it is a matter of assessing the evidence and making a judgment.[42]

The National Welfare Rights Network Inc. expressed apprehension about the use of rule-based systems in the administration and delivery of social security payments relying on discretionary criteria. Although acknowledging the usefulness of decision-support tools, it submitted that it is not possible to have a logic tree that covers every possibility in discretionary decision making and that the infinite variety of the human condition means that rule-based systems are not appropriate in administrative decision making. It argued that it is beyond the scope of these systems to consider and weigh up all the relevant matters—legal, administrative and social—that affect a person.[43]

3.2.3           Discretion and uses of expert systems identified by the Council

Although it would be possible for an expert system to automate the exercise of a discretion based on the data provided to it, the Council’s research suggests that this is not occurring. Instead, the expert systems it identified provide information or guidance to the decision maker in relation to discretionary decisions. The Department of Family and Community Services and Centrelink investigated the possibility of an expert system making discretionary decisions but rejected this approach on the basis that it was inappropriate.[44]

SoftLaw and the Axe Group described four approaches that have been adopted for dealing with the exercise of discretion by expert systems[45]:

·         Approach 1: Direct. The discretion to be exercised is asked as a base-level fact (to which the response is ‘yes’ or ‘no’) and the officer exercises the discretion whilst being guided by the commentary. The officer can record the reasons for the exercise of discretion. This approach is appropriate when questions of fact and value are inseparable in the exercise of discretion, such that a decision maker attaches value to a matter of fact even in choosing to have regard to that matter of fact.

The approach was adopted in the Department of Veterans’ Affairs Incapacity Calculator in relation to a decision about whether interim payments ought to be paid under s. 19 of the Safety, Rehabilitation and Compensation Act 1988. The rule-based system asks the assessor whether interim payments should be made. Commentary material is provided for the assessor. If the assessor’s answer is ‘yes’, the system applies the calculation rules in s. 19 to the employee’s case.

·         Approach 2: Recommendation. The system collects data related to the discretion and then recommends a decision to the officer. The officer is asked to confirm or override the decision and to fill in the reason box for audit purposes. This approach is used in limited situations when questions of fact and value are separate but must be reconciled by a decision maker in the exercise of discretion. When it is used, the decision maker is required to attach a value to each matter of fact before exercising the discretion. The commentary might suggest the value to be attached to each matter of fact.

An example of this approach is Comcare’s Initial Liability Module system in relation to an officer’s decision whether initial liability should be accepted under s. 14 of the Safety, Rehabilitation and Compensation Act. The rule-based system recommends a decision and it is open to the officer to override that decision. There is a facility that allows the officer to provide reasons or supporting explanation.

·         Approach 3: Guided. This approach is appropriate when questions of fact and value are entirely separate, such that the only question of value in the exercise of discretion is whether the discretion is exercised at all. For example, the discretionary provision sets out the matters of fact that must be considered if the discretion is to be exercised. The approach offers alternatives: the officer can exercise discretion as a base fact or can choose to be guided through the various factors that must be considered in exercising the discretion—a process that ensures the officer has considered all the factors relevant to exercising the discretion and the factors can be reviewed for audit purposes. The officer is then presented with a question (as in Approach 1) that requires them to enter the discretion as a base-level fact. There is also a reason box.

An example of this approach is a prototype being developed for the Australian Taxation Office but not yet in use. The prototype relates to s. 87-65 of the Income Tax Assessment Act 1997. When a business fails a test for a ‘personal services business’ the Commissioner has a residual discretion. Section 87-65(3) limits the exercise of that discretion but refers to the consideration of ‘unusual circumstances’, which are defined in s. 87-65(4). If the unusual circumstances are immediately obvious to the officer—because of their nature or the officer’s expertise—the discretion can be exercised immediately and the reasons for doing so entered. If, however, the officer wants to consider further the statutory limitations on the exercise of the discretion, the rule base directs the decision maker through the relevant screens. SoftLaw noted, ‘The availability of an “immediate” discretion may be limited to users above a particular level of experience. In this manner … rule-bases may be developed for less experienced users’.

·         Approach 4: Workflow. The Axe Group uses the Workflow approach, in which relevant facts are automatically routed to an expert who can make the decision(s) and then the rules decide on the ‘next step’ in the light of the expert’s decision. This seems to be similar to SoftLaw’s Direct approach.

3.2.4           Conclusion

It is the Council’s view that the automation of discretion is not in accordance with the administrative law values of lawfulness and fairness because it could fetter the decision maker in the exercise of their discretionary power. Nevertheless, the four approaches just described demonstrate that expert systems can be used as an administrative tool to help an officer exercise their discretion.

To ensure the lawfulness and fairness of discretionary decisions, expert systems must be designed in a way that reflects government policy and does not fetter the decision maker in exercising any discretion he or she has been given. The system should expressly advise the decision maker that the question being asked is a matter for the decision maker’s judgment. Just as hard-copy resources should accurately reflect government law and policy, so too should any information provided to a decision maker by an expert system.

In the course of its work on expert systems, the Council identified 27 principles of best practice, which it suggests should be adopted for the use of expert systems in the administrative decision-making process.

4                 The potential advantages and disadvantages of using expert systems in administrative decision making

The Council identified a range of considerations that highlight the advantages and disadvantages of using expert systems to make or help to make administrative decisions. The main considerations are as follows:

·         primary administrative law considerations

–         authority for using expert systems

–         overriding an expert system

–         grounds for review of decisions

–         privacy

–         disclosure requirements

–         possible narrowing of discretion

·         system development and operational considerations

–         accuracy and consistency

–         design and maintenance

–         time and cost

–         ‘skilling’ or ‘de-skilling’ of decision makers

–         human manipulation

–         the audit facility

–         data quality

–         review procedures

–         independent scrutiny

·         new service delivery options

–         diverse service delivery mechanisms

–         involvement of community organisations

–         self-assessment.

4.1           Authority for using expert systems

When an expert system is used to make a decision (or even part of one) there is a question whether the computer is the decision maker and, if so, whether there is a need for legislation permitting a decision to be made by a computer. This is relevant to the administrative law value of lawfulness.

Some respondents to the issues paper argued that it was not necessary for statutory recognition of the use of computer programs in administrative decision making because the programs are simply tools; others argued that some form of statutory recognition would be appropriate or useful.

One respondent considered that automated assistance in administrative decision making should be legislatively sanctioned, facilitated and governed—particularly where it affects personal rights—and that some people erroneously assume electronic transactions legislation authorises the use of computer systems in the decision-making process.[46]

4.1.1           The Electronic Transactions Act

The Electronic Transactions Act 1999 provides that, for the purposes of the Commonwealth, a transaction is not generally invalid because it took place wholly or partly by means of one or more electronic communications. It prescribes the circumstances in which the following legal requirements can be met by means of an electronic communication:

·         a requirement to give information in writing

·         a requirement to provide a signature

·         a requirement to produce a document

·         a requirement to record or retain information.[47]

The Act does not require that a person receive electronic communications from the Commonwealth; instead, a person must consent to receiving such communications.[48] In addition, the Electronic Transactions Regulations 2000 list a number of laws to which the Act does not apply. Accordingly, although the Act legitimises the use of electronic communications in dealings between Commonwealth agencies and the public, it does not cover the question of whether a decision can actually be made by a computer system such as an expert system.

4.1.2           Current legislation authorising the use of expert systems

There is little specific legislative authority for decisions made by computers. The Council is aware of provisions of social security law and family assistance law that provide for the use of computer programs in decision making and that the resultant decisions are deemed to have been made by the Secretary of the relevant department.[49] The Acts provide as follows[50]:

(1)           The Secretary may arrange for the use, under the Secretary’s control, of computer programs for any purposes for which the Secretary may make decisions under the social security [or family assistance] law.

(2)           A decision made by the operation of a computer program under an arrangement made under (1) above is taken to be a decision made by the Secretary.

The social security and family assistance provisions are similar to the provisions in the Migration Act 1958 and the Australian Citizenship Act 1948, as a result of amendments made in 2001.[51] The amendments provide that the Minister for Immigration and Multicultural and Indigenous Affairs is taken to have made a decision that was made by the operation of a computer program for the purposes of a designated migration law[52] or for the purposes for which the Minister may or must make a decision under the Australian Citizenship Act.[53] Accordingly, as noted in Chapter 3, references in this report to expert systems that make a decision or to decisions made by expert systems include decisions that are legislatively deemed to have been made by an agency officer.

The amendments to the Migration Act have been used in relation to electronic travel authorities, or ETAs, granted by the Department of Immigration and Multicultural and Indigenous Affairs. An ETA is an electronically stored authorisation for travel to Australia for either a short-term visit or business purposes.[54] Once information is entered into the system—for example, a travel agent enters the applicant’s passport details—the system automatically performs an online check of DIMIA warning records and if no adverse record is detected it advises that the ETA has been granted.[55] Although ETAs were introduced in 1996, the 2001 amendments mean that there is no need for a delegated officer to use the computer printouts of ETAs in their decision-making processes.

4.1.3           Delegating decision-making power to the expert system

None of the legislative schemes the Council identified seeks to authorise the computer as the decision maker; instead, the schemes deem the decision to have been made by a human.

Under the current law it may not be legally valid to delegate the decision-making power to a computer system. Both the Administrative Decisions (Judicial Review) Act 1977 and the Administrative Appeals Tribunal Act 1975 seem to assume that the decision maker is a person.[56] The Acts Interpretation Act 1901 also provides that, unless a contrary intention appears in the legislation conferring the power to delegate, the power of delegation shall be construed as including a power to delegate a power not only to a specified person but to any person from time to time holding, occupying or performing the duties of a specified office or position.[57]

4.1.4           Current case law

There is little case law providing guidance in relation to the validity and operation of computer-generated decisions—possibly because there is often nothing on the face of the decision to alert the affected person or a tribunal or court to the fact that the decision was computer generated or assisted.[58]

The Administrative Appeals Tribunal has noted the use of a computer system as a decision-making tool by delegates of the Department of Veterans’ Affairs. But that was because a major argument in the case in question concerned the delegate’s failure to consider a newly promulgated statement of principles that had not been programmed into the decision-making tool.[59]

One case that did consider the legitimacy of a decision made by an expert system was an unreported decision of the Social Security Appeals Tribunal. The Tribunal found that the computer program that had cancelled a person’s family payment had never been approved by the then Secretary of the Department of Social Security, as required by the Social Security Act 1991.[60] The Tribunal decided that the cancellation was therefore invalid.

4.1.5           Conclusion

When an expert system makes a decision, legislative sanction for the computer’s making of that decision should be obtained to ensure that the decision is compatible with the principles of authorised decision making. If an expert system is simply used as a tool for an officer who makes the actual decision, it would appear that legislative authority is not necessary.

4.2           Overriding an expert system

An officer’s capacity to override an expert system is an important consideration that is relevant to the administrative law values of lawfulness, fairness and efficiency. If the officer can override the expert system, the expert system is being used only as a tool to support the officer in making a decision, even if the officer chooses not to override the system. If the officer cannot override it, the system is responsible for the decision[61], which is based on the data entered by the officer.

4.2.1           Current agency practices

Practice appears to differ between agencies. For example, in Comcare an officer can override a recommendation of the expert system by discussing the matter with his or her team leader and documenting the reasons for overriding the decision. The action in overriding the decision must be justified to the quality assurance officer. Failure to accept the expert system’s decision means that Comcare’s quality assurance process is automatically activated.

In the Department of Veterans’ Affairs an officer cannot override a decision made by the expert system. An officer who believes a decision to be incorrect is required to check the accuracy of the information that has been entered into the system. If they are still concerned about the result, they must discuss the matter with their supervisor, and a course of action is then decided. The Department also encourages officers to ring its national office about matters of concern.

One submission argued that a rule-based system that faithfully models a particular piece of legislation will be capable of taking into account any circumstances contemplated by the legislation, including any legislative provision for a decision maker to have residual discretion.[62] It did, however, note the importance of having a formal process for dealing with situations where the rule-based system is thought to be producing incorrect results.[63]

4.2.2           Potential disadvantages and advantages of allowing overriding

When an expert system is used in administrative decision making the main disadvantage associated with allowing an officer to override the decision is that it might compromise the accuracy and consistency that can be provided by the use of expert systems. It is, however, possible that the situation could arise where an expert system does not take account of all the circumstances of a case and it would be preferable for an agency officer to be able to override a decision made by the system. Such a situation could occur, for example, if the expert system is malfunctioning, if particular circumstances were not contemplated and incorporated in the rules but should be considered, or if there has been a circumstance (such as a court judgment) that alters the interpretation of a rule.

If the problem could not be resolved within a reasonable period, the ability to override a decision would minimise disruptions to the agency and the affected person would not have to wait for changes to the expert system. It would be preferable, however, if malfunctions could be rectified, or updates made, promptly so as to obviate the need to override decisions.

It is also prudent to require the decision maker to contact a senior officer before overriding a decision—as is the practice of the Department of Veterans’ Affairs.

It might be advantageous to have a legislative provision, similar to those in the Migration Act and the Australian Citizenship Act[64], for situations in which the power to override may be exercised only by a senior officer or other person.

4.2.3           Case law relating to reconsideration of administrative decisions

Concerns about when a decision is made and whether it can be reconsidered or subject to review also arise in relation to manual decision making.

The question of whether decisions can be overridden is complex and has been the subject of much judicial comment. In the High Court decision in Minister for Immigration and Multicultural Affairs v Bhardwaj, Gleeson CJ stated:

There is nothing in the nature of an administrative decision which requires a conclusion that a power to make a decision, once purportedly exercised, is necessarily spent. In Ridge v Baldwin (27), Lord Reid said:

I do not doubt that if an officer or body realises that it has acted hastily and reconsiders the whole matter afresh, after affording to the person affected a proper opportunity to present his case, then its later decision will be valid.

That general proposition must yield to the legislation under which a decision-maker is acting. And much may depend upon the nature of the power that is being exercised and of the error that has been made.[65]

In a subsequent Full Federal Court decision, the majority of the Court considered that the decision in Bhardwaj showed that any legal and factual consequences of the decision depend on the particular statute involved.[66]

4.2.4           Conclusion

Situations could arise where overriding an expert system would be appropriate. Before the expert system is overridden, however, the primary decision maker should first discuss the decision with a senior officer.

4.3           Grounds for review

As with administrative decisions made manually, decisions made by or with the assistance of an expert system can be subject to review. Use of expert systems in administrative decision making might raise particular considerations in relation to the grounds for review. These considerations relate to administrative law values such as lawfulness, fairness and rationality.

There are two types of review of administrative decisions—merits review by an administrative tribunal and judicial review.

Merits review involves determining whether a decision was the ‘correct or preferable’ decision.[67] In contrast, judicial review is not the re-hearing of the merits of a particular case: rather, it is where a court reviews a decision to make sure that the decision maker used the correct legal reasoning or followed the correct legal procedures.[68] The right to have a decision judicially reviewed can arise from a number of sources, among them the common law, the Administrative Decisions (Judicial Review) Act 1977, the Judiciary Act 1903 and the Constitution.[69]

The grounds for judicial review, as set out in s. 5 of the Administrative Decisions (Judicial Review) Act are a useful starting point for considering the requirements for valid decisions made using expert systems. Section 5 of the Act provides as follows:

(1)           A person who is aggrieved by a decision to which this Act applies that is made after the commencement of this Act may apply to the Federal Court or the Federal Magistrates Court for an order of review in respect of the decision on any one or more of the following grounds:

(a)           that a breach of the rules of natural justice occurred in connection with the making of the decision;

(b)           that procedures that were required by law to be observed in connection with the making of the decision were not observed;

(c)           that the person who purported to make the decision did not have jurisdiction to make the decision;

(d)           that the decision was not authorised by the enactment in pursuance of which it was purported to be made;

(e)           that the making of the decision was an improper exercise of the power conferred by the enactment in pursuance of which it was purported to be made;

(f)            that the decision involved an error of law, whether or not the error appears on the record of the decision;

(g)           that the decision was induced or affected by fraud;

(h)           that there was no evidence or other material to justify making the decision;

(i)            that the decision was otherwise contrary to law.

4.3.1           Natural justice

Natural justice—sometimes referred to as procedural fairness—imposes on decision makers a flexible obligation to adopt fair procedures that are appropriate and adapted to the circumstances of the particular case.[70]

The right to be heard

An important principle of natural justice is that decision makers afford a person whose interests might be adversely affected by a decision an opportunity to be heard.

Automation of administrative decision making could give rise to the potential for decisions to be made without reference to the person affected by the decision. For example, if data entered via an expert system were used with other information obtained through either data matching or the use of another expert system and the information was adverse to the applicant’s interests, the applicant might not be aware of all the information being used to make a decision and therefore not have the opportunity to respond to it.[71] This would not be in accordance with the principles of natural justice.

The Department of Veterans’ Affairs advised the Council that its expert system obtains only some facts from the mainframe data—for example, service details, age and address—and that, in the case of service details, the system still requires the assessor to confirm the correctness of the details from source documents.[72] Centrelink advised the Council that its expert system uses information stored in its mainframe database. If other relevant information is obtained as a result of data matching, then, regardless of whether an expert system or mainframe was being used, action would be taken only after contact with the customer.[73]

No bias

Natural justice requires that a decision maker be disinterested, or unbiased, in relation to the matter to be decided. Expert systems might appear to reduce the possibility of bias on the part of the decision maker, but there could be inherent bias in such systems—for example, in the way questions are phrased. It has been suggested that some officers might enter particular data in order to produce an outcome adverse to the applicant’s interests: use of an expert system in this way could allow officers to shroud their decisions in legitimacy.

Probative evidence

It is sometimes argued that another aspect of natural justice is that a decision should be based on some rationally probative evidence.[74] An expert system can be designed in such a way that the veracity of any evidence provided in support of a decision can be considered by a human but the expert system guides the consideration of that evidence. When an expert system treats all evidence provided as probative—without discriminating according to levels of probativeness—this could skew an outcome unfairly.

Auditing the quality of the factual information on which a decision is based is critical to the accuracy of the outcome, and in the absence of any auditing function there could be a breach of the probative evidence rule.

4.3.2           Compliance with procedures required by law

A statute can provide that a power is to be exercised in a particular way or only after particular conditions have been satisfied.[75] When compliance with a procedure is a legal requirement, the question that arises is when does non-compliance with the procedure render administrative action unlawful?[76] Where an expert system is involved, such a question may arise in relation to whether the decision has been made by a nominated decision maker or wholly by an expert system.

The Administrative Decisions (Judicial Review) Act and the Administrative Appeals Tribunal Act assume that the decision maker is a person.[77] There may therefore be a requirement in some circumstances for legislative provisions deeming the computerised result to be that of a decision maker.

4.3.3           Improper exercise of power

Relevant and irrelevant considerations

Improper exercise of power is also a ground for review.[78] Taking an irrelevant consideration into account and failing to take a relevant consideration into account are examples of an improper exercise of power.[79] Whether these considerations arise with the use of an expert system will depend on the comprehensiveness of the rules and the extent to which the outcome can be influenced by an agency officer in, for example, choosing not to enter a relevant fact into the database. The decision maker could use the expert system to work through complex legislation and then, when discretion needs to be exercised, take an irrelevant consideration into account.

Use of expert systems could involve an improper exercise of power if the decision maker fails to enter, or selectively enters, into the system facts provided by an applicant. This could skew the outcome towards the decision maker’s predetermined view of a desirable outcome. The difficulties that arise in this context are the same as those that arise in relation to manual administrative decision making.

Dictation

Exercise of a personal discretionary power at the behest of another person is another example of the improper exercise of power.[80]

Expert systems could result in decision makers acting under dictation if they fail to query a decision of the system they think is incorrect. Decision makers could possibly be acting at the behest of another if they refuse to exercise judgment even when the expert system informs them that this is required.

The non-fettering rule

The exercise of a discretionary power in accordance with a rule or policy but without regard to the merits of the case is another instance of an improper exercise of power.[81] A decision maker’s exercise of discretion could be fettered if the construction of the expert system database has inappropriately narrowed the policy in question.

Unreasonableness

The exercise of a power that is so unreasonable that no reasonable person could have exercised it in that way is also an example of the improper exercise of power.[82] If the expert system is accurate, those components of the decision that are based on fact will not result in an exercise of power that is unreasonable. But the exercise of a power that is unreasonable could result if a decision maker uses discretion in an unreasonable manner—by, for example, ignoring particular options offered by the expert system, by selectively choosing facts to enter into the system, or by giving undue weight to particular facts.

4.3.4           Administrative review safeguards

In response to the issues paper, Centrelink and the Department of Family and Community Services advised the Council that they take several steps to ensure the preservation of traditional administrative law safeguards in the use of Centrelink’s expert systems, as follows[83]:

·         All commentary to be included in the expert system is cleared by legal and policy sections to ensure that it accurately reflects the legislation, that relevant considerations are taken into account, and that the duty of care owed to the customer is maintained.

·         Safeguards built into the system are only asking relevant questions, telling customers why questions are being asked (which makes the decision-making process more transparent) and recording and explaining to a customer the reason for a decision.

·         The appeal process is not altered. The expert system was considered to improve the basis for reviewing decisions by recording the decision-making process and giving reasons for discretionary decisions, which results in a better decision trail.

SoftLaw considers that a properly designed rule-based system can assist in preserving administrative law safeguards in a number of ways, among them the following[84]:

·         reducing the likelihood of bias by requiring decision makers to consider the applicant’s claim against all the relevant legislative and policy requirements

·         improving procedural fairness by collecting information in a logically probative way

·         ensuring that decision makers take into account relevant considerations and do not take into account irrelevant ones. This is achieved by the system taking the decision maker directly to the relevant legislation and policy and closing off provisions that are irrelevant.

4.3.5           Conclusion

If an expert system complies with the requirements of authorised decision making[85] and appropriately reflects the legislation and policy, and any discretion is exercised appropriately by the decision maker, the decision should generally be upheld on review—be it merits review or judicial review.

4.4           Privacy

Although privacy considerations are relevant to administrative decision making generally, the limitations of hard-copy data systems provide a degree of privacy protection.[86] The privacy concerns raised by the use of expert systems in administrative decision making are relevant to the administrative law values of lawfulness, fairness and openness.

4.4.1           Privacy obligations

The Information Privacy Principles set out in the Privacy Act 1988 place obligations on Commonwealth agencies. Among them are obligations relating to the collection of, the quality and security of, rights of access to, and use and disclosure of personal information. The Federal Privacy Commissioner has issued guidelines to the principles, providing the Commissioner’s view of how the principles affect federal government agencies.[87]

One important principle is that Commonwealth agencies may collect personal information only if it is for a lawful purpose and it is necessary for or directly related to that purpose.[88] Although this is relevant to administrative decision making generally, the use of computer technology in decision making offers greater capacity for data collection, processing and transmission. Agencies should therefore ensure that the data collected by expert systems used in administrative decision making are necessary to the decision in question.

In general, a person must consent to the use of their personal information for a purpose other than that for which it was collected.[89] There could thus be a breach of privacy if data collected for another purpose are used by an expert system in the automation of all or part of the administrative decision-making process.

When an agency seeks personal information from individuals, it has specific notification obligations. It must tell the person why the information is being collected and inform them of the legal authority for collecting the information and who the agency usually gives that sort of information to.[90] Such notification means that there are fewer surprises for people in relation to use of their personal information, which would be especially pertinent if expert systems technology is used to collect information from different sources and automate decision-making processes.

Additionally, before using the information an agency must be assured of its accuracy.[91] Rather than minimising the risk of inaccuracy, the use of data already held by an agency for another purpose could in fact increase the risk of inaccurate data being used; for example, the data might be old or the individual’s circumstances might have changed, making the data inaccurate. Processes need to be implemented to ensure the currency and accuracy of information used in administrative decision making.

4.4.2           Data matching

Government agencies are increasingly using information obtained by data matching, which involves the comparison of data collected from different agencies. Developments in computer technology have improved data-matching processes. [92]

Although data matching benefits administration by allowing for the comparison of large amounts of data from difference sources—which can help to identify and reduce fraud—it can also raise privacy concerns. Examples are information being used for purposes other than those for which it was collected and examination of the personal information of large numbers of people about whom there are no known grounds for suspicion.[93]

Compliance with agency notification requirements is also important in relation to data matching. This is particularly the case when the data matching occurs without the individuals concerned being informed. As a result, in addition to specific legislation regulating data matching by the use of tax file numbers[94], the Federal Privacy Commissioner has issued advisory data-matching guidelines.[95] The guidelines were developed to help ensure that data-matching programs used by Commonwealth agencies are designed and operated in accordance with sound privacy practices.[96]

The privacy concerns raised by data matching would be relevant if an expert system used information obtained through data matching in order to make or assist in making a decision.

4.4.3           Potential information-gathering benefits

The use of computer technology (including expert systems) offers significant capacity for data collection and retrieval.[97] For example, databases (including expert system databases) can be searched to find the number of applicants who are receiving a particular benefit and have a child under a specified age. An expert system can also allow for interactive data collection that can be highly personalised and can remove the need to ‘genericise’ the process.[98] For example, SoftLaw has argued that, depending on the goal of the interview, its systems can determine what questions to ask, which means that only the information that is needed and lawfully permitted is collected during the interview.[99] This limits the potential for privacy breaches caused by the collection of unnecessary personal information.

Expert systems can also help break down information ‘silos’ that could adversely affect applicants. For example, an applicant in receipt of a payment might communicate a change in circumstances to the area that administers that payment. That change in circumstances might also be relevant to the payment of another entitlement by a different area. If payment systems are not linked the change in circumstances might not be recorded in the other area. Expert systems can help by identifying rules that are common to different types of payments. They can also explore the flow-on effects of a change in circumstances—for example, whether a change in circumstances notified in respect of one benefit also affects another benefit being paid to a recipient. In this context, it has been noted that:

Because every data collection exercise can be tailored to the situation of the specific client, it is possible to collect a far broader range of information. The implications of this fact are profound. This enables far more precise targeting of policy and service delivery as well as far more detailed collection of management information than has traditionally been dreamt of, let alone possible.[100]

Although breaking down information silos in that way can benefit policy formulation and service delivery, it can also raise privacy concerns. For example, some individuals might think their privacy has been breached if they applied for a specific entitlement and received a response about a range of entitlements. In this circumstance, notification of the way in which information could be used would be important in order to give individuals their preferred service.

4.4.4           Privacy developments

The former Federal Privacy Commissioner, Malcolm Crompton, has stated, ‘New technologies are not necessarily destructive of privacy. They can be privacy enhancing technologies or privacy intrusive technologies, depending on how they are designed and the uses to which they are put’.[101] Privacy-enhancing technologies have been described as information and communications technology measures that protect privacy by eliminating or reducing personal data or by preventing unnecessary or undesired processing of personal data.[102]

A recent project between the Information and Privacy Commissioner Ontario (Canada) and IBM is an example of the development of privacy-enhancing technology. As part of the project, which aims to make compliance with privacy law easier and more cost effective, a digital template of Ontario’s privacy legislation is being created.[103] With an IT-readable version of the legislation, the privacy rules can be built into computer systems.[104] Doing this means that the systems can prevent privacy breaches. This notion of ensuring that privacy is built into computer technology has been encouraged by the Office of the Federal Privacy Commissioner in Australia.[105]

It should also be noted that some jurisdictions have specifically legislated in relation to privacy and automated decision making. For example, in 1995 the European Union issued its Directive on the Protection of Individuals with Regard to the Processing of Personal Data and on the Free Movement of Such Data. The Directive provides that member states guarantee data subjects the right to know the logic involved in automatic processing of data about them in the case of automated decisions.[106] Member states also guarantee that people have a right not to have decisions that affect them made solely on the basis of automated processing of data intended to evaluate personal characteristics such as performance at work, creditworthiness and reliability.[107] There are, however, exceptions to this guarantee—for example, where total reliance on automatic processing is authorised by a law that also sets out measures for safeguarding the data subject’s legitimate interests.[108]

A number of members of the European Union have enacted data protection legislation on the basis of the Directive. The United Kingdom’s Data Protection Act 1998 provides rights for people affected by decisions based solely on the automatic processing of their personal information. However, like the provisions in the EU Directive, these rights are not available where the decision in question is authorised or required by or under any enactment and the effect of the decision is to grant a request of the individual or steps have been taken to safeguard the legitimate interests of the individual (for example, by allowing him or her to make representations).[109] In the Australian context, this might limit the legislation’s applicability to administrative decision making.

4.4.5           Conclusion

As with the use of any technology in administrative decision making, it is important that expert systems be designed and operate in such a way that they are compatible with the privacy obligations of government agencies.

4.5           Disclosure requirements

It is important for transparency and fairness that the rules in an expert system be available to any person wishing to know about them.

Section 9(1)(a) of the Freedom of Information Act 1982 requires Commonwealth agencies to publish and make available for purchase by the public the internal law of the agency, including manuals and other documents containing interpretation, rules, guidelines, practices or precedents. The definition of ‘document’ contained in the Act would include information stored on computer software, such as the rules constituting an expert system.[110]

Generally, if a decision is subject to review by the Administrative Appeals Tribunal or if a provision of the Administrative Decisions (Judicial Review) Act applies to a decision, a person applying for review of the decision can seek reasons for the decision.