Evidence Tests, Types, and Sources in Government Auditing

Auditors collect and develop data to address the audit objectives. The data collected and developed that support the auditor’s findings and conclusions are considered evidence. GAGAS sets forth the requirements for evidence.

Evidence Tests (Standard)

GAGAS states that: “Auditors must obtain sufficient, appropriate evidence to provide a reasonable basis for addressing the audit objectives and supporting their findings and conclusions.” (par 8.90)

Sufficient

Sufficiency is a measure of the quantity of evidence used for addressing the audit objectives and supporting findings and conclusions. (par 8.99)

Criterion to judge whether evidence is sufficient.

In determining the sufficiency of evidence, auditors should ensure that enough evidence exists to persuade the intended recipient of the validity of findings, reasonableness of the conclusions, and the benefit of implementing the recommendations. When appropriate, statistical methods may be used to establish sufficiency.

Evidence sufficiency is judged relative to the audit universe, i.e., the scope.

Auditor's choices:

• Obtain data on the entire universe.
• Sample the universe.
• Limit findings to that portion or segment of the universe examined.

Sufficiency establishes that findings and positions taken on the basis of the audit scope have not inappropriately generalized or overstated the evidence the auditor obtained and recorded in the audit documentation.

Appropriate

Appropriateness is the measure of the quality of evidence and encompasses its relevance, validity, and reliability in providing support for findings and conclusions.

Relevant

Relevance refers to the extent to which the evidence has a logical relationship with, and importance to, the issue being addressed.

How to judge whether evidence is relevant?

Evidence is relevant if it has a relationship:

First, it is useful in answering the overall audit objectives.

Second, to

• The audit subject,
• The time period of the issue(s) being audited,
• The aspect or aspects of performance being examined, and
• The finding element to which the evidence pertains.

Criteria element: Auditors are responsible under the auditing standards for using criteria that are relevant. Because business practices and other gauges used as “criteria” to assess the adequacy of performance tend to become outdated with time, auditors must affirm the continued relevance of such gauges before using them as criteria. Provisions of laws, regulations, directives, policies, and controls, too, are subject to obsolescence; thus, auditors will want to affirm their continued relevance before using them as criteria.

Condition element: Evidence on “condition” must pertain to the

• audit subject,
• aspects of performance being examined, and
• time period addressed by the issue(s) being audited.

Effect element: Evidence on effect must be an extension of, and consistent with, the condition.

Cause element: Evidence on “cause” must have a causal relationship to the “condition” element. Auditors need to demonstrate and explain with evidence and reasoning the link between good and poor performance and the factor or factors they identified as the cause. Auditors cannot simply cite what they believe to be the cause or causes without supporting evidence. They must also show that the problem warrants corrective action; that the benefit will outweigh the cost.

Valid and Reliable

Validity refers to the extent to which evidence is a meaningful or reasonable basis for measuring what is being evaluated. In other words, validity refers to the extent to which evidence represents what it is purported to represent.

Reliability refers to the consistency of results when information is measured or tested and includes the concepts of being verifiable or supported. Audit literature describes reliable evidence as being accurate, complete, and authentic.

Criterion for judging evidence validity and reliability

Our criterion question should be, “Are we reasonably confident that our evidence presents a picture that is not significantly different from reality and that our analysis techniques are appropriate?”

Evidence Types

Three types of evidence cited in the auditing standards are:

• Physical
• Documentary
• Testimonial

Why Classification of Evidence is Important

• Method of collection differs for each type.
• Quality of evidence depends in part on the type.
• Methods of assuring quality differ with each type.

Physical Evidence

Physical evidence is anything that is directly apparent to the auditor’s senses. It can be heard, felt, smelled, tasted, and seen, and can be described. Natural phenomena exist all around us. The earth is its own evidence for existence, as is a bridge, the smell of popped popcorn, and the sound of thunder. An auditor sees a bridge and knows it exists.

Common forms:

Documentary Evidence

Documentary evidence consists of created data in written and graphic form gathered and prepared by someone other than the auditor, such as the auditee or third parties. It exists in:

Testimonial Evidence

Data obtained directly from people in response to the auditor's or other persons’ inquiries.

• It may be oral or written
• It may represent personal knowledge and fact, or opinion and belief
• It may be true, false, biased, and incomplete

Potential Weaknesses in Each Type of Evidence

This list of possible weaknesses for each type of evidence is not meant to be a complete list of every possible thing to look out for.

Look Out for Physical Evidence that:

• Is collected without a specified methodology; is not representative (isolated instances)
• Is a photograph without a record of observation (or memo to the record)
• Lacks a clear chain of custody (e.g., police confiscated drugs)
• Lacks a test of inventory (i.e. may look at an oil tank and assume it is full of oil when it actually contains mostly water with a layer of oil on top)
• Is unwitnessed (i.e. only one observer)
• Is collected by an unqualified observer
• Is actually a “set-up” (like a youth offenders program which tried to make it appear that a required educational program was taking place when auditors arrived; in reality, there was no program in place)

Look out for Documentary Evidence that is:

• Taken out of context (pulled out from other documents)
• From a newspaper or magazine article
• Other auditors’ work (i.e. rely on their work without checking)
• From a secondary, not a primary source
• Outdated
• Not official
• Refutable
• Controversial
• Incomplete
• Unclear (unidentified acronyms, etc.)
• Inconsistent

Look out for Testimonial Evidence that is:

• From a source whose knowledge about the topic is in doubt.
• From a source whose credibility is questionable.
• From a source that is not impartial, has a vested interest.
• The only evidence used to support a point (for example, if we rely only on statements by one person, one worker in a group of workers).
• Not corroborated.
• Used out of context.
• Inconsistent.
• Incomplete.
• A “snow job.”
• Not attested to or certified (if only one person).
• Disguised as documentary evidence (examples: (1) written comments or (2) numbers written or typed on a blank piece of paper by an official during an interview).
• Improperly substituted for documentary evidence (for example, relying on what an official told us the law said instead of obtaining a copy of the law itself or using testimonial evidence to support the date a contract was signed instead of the contract itself).
• From a source that needs to do homework on the topic.
• Possible miscommunication.

Other Useful Evidence Classifications

Analytical

Audit literature has historically recognized “analytical evidence” as a type of evidence. Analytical evidence comes from the auditor’s analysis and logical reasoning using physical, documentary, and testimonial evidence previously obtained.

Fact and Views

Probably the most fundamental distinction that can be made between kinds of data.

Fact: That which is reality or actual truth. Something presented as objectively real and having demonstrable existence. Something that has been objectively verified.

Views: A belief, conclusion, or opinion held with confidence, but not substantiated by positive knowledge or proof.

Quantitative and Qualitative

Another fundamental distinction that can be made between kinds of data.

Quantitative: Data in a form that can be counted or measured with a device.

Qualitative: Physical attributes and data in the form of words. Requires special methods to quantify.

Evidence in Court Proceedings

When auditors’ findings include activities that may suggest fraudulent activity that could result in legal action, evidence must meet the standards and requirements of the judicial system. In the judicial system, evidence is viewed as (1) direct, (2) circumstantial, or (3) oral, documentary, and real.

Direct evidence is that which, if believed, proves the existence of the principle or fact without any inference or presumption.

Circumstantial evidence is that which proves a fact by inference. In criminal cases, evidence is often circumstantial.

Oral evidence consists of statements made by witnesses under oath or affirmation.

Documentary evidence consists of official records, contracts, deeds, letters, and books of organizations or individuals.

Real evidence consists of tangible objects or property.

Data Sources and Collection Methods

Data sources include:

• original data gathered by auditors, and
• existing data gathered by either the auditee or a third party.

Collection Methods: Some methods for collecting data are listed below.

• Observing
• Listening
• Taking samples and having them analyzed
• Using measuring devices
• Reviewing paper records
• Reviewing computerized records on a monitor
• Extracting data from computerized records
• Interviewing face-to-face
• Interviewing via phone
• Using questionnaires
• Obtaining written statements
• Obtaining views or data from experts

Data Collection and Recording Methods

Collection Method: Physical: Anything apparent to the senses.

Recording Method

• Observation
• Feeling, smelling, hearing, tasting
• Written description, listing, tally, spreadsheet, flowchart, drawing, picture, video, or audio recording
• Laboratory analysis of samples provides its own measurement
• Devices provide their own, read and enter to tally

Collection Method: Documentary: Data in written and graphic form prepared by someone other than the auditor.

Recording Method

• Paper or film records: review; reproduce, photocopy, extract in pencil, extract to computer file/spreadsheet
• Electronic records:
  • Access with a computer and review on monitor; print out or extract in pencil (to a spreadsheet, tally, etc.).
  • Access with a computer and download or extract (CAATs) to flash drive or other storage in file, spreadsheet, database format.
  • Entity provides in electronic media already recorded.

Collection Method: Testimonial: Data obtained directly from people in response to the auditor's or other persons’ inquiries.

Recording Method

• Face-to-face interview: handwritten (as notes, tally, spreadsheet, questionnaire, etc.), audio or video recording.
• Telephone interview: handwritten (as notes, tally, spreadsheet, questionnaire, etc.), audio recording.
• Questionnaire: provides its own (will require analysis).
• Written statement/representation: provides its own.

Collection Method: Analytical: Comes from the auditor’s analysis and logical reasoning using data previously obtained.

Recording Method

• Uses data previously collected.
• Handwritten, computer-generated.

Ensuring Data Quality

When collecting existing data or new data, the auditor needs to ensure the validity and reliability (quality) of that data. The quality of data must be assured before it is used in an audit report and for subsequent decision-making.

Data validity and reliability involve consideration of the following concepts:

• Data Accuracy
• Data Completeness
• Data Authenticity

Importance of Data Quality

When uncertainties exist with data quality, they cannot be used as the primary evidence. The auditor will need to plan alternative approaches. Alternatives include, but are not limited to, the following:

1. Obtaining independent, corroborating evidence from other sources;
2. Redefining the audit objectives or limiting the audit scope to eliminate the need to use the data;
3. Present the findings and conclusions so that the supporting evidence is sufficient and appropriate, and describe in the report the limitations or uncertainties with the validity or reliability of the evidence, if such disclosure is necessary to avoid misleading the report users about the findings and conclusions.

Appropriateness of Computerized Data

Depending on the significance of computerized information systems controls to the audit objectives, auditors should determine which audit procedures related to information systems controls are needed to obtain appropriate evidence to support the audit findings and conclusions.

A common-sense, logic method for testing the reliability of auditee data, including data from the auditee’s computer systems, is to ask the following questions of those receiving the data (e.g., of a judgmental sample):

• Does anyone who receives the data use it?
• Do those who use it believe the data are accurate and sufficiently reliable for their needs?
• Do those who use the data review it to identify any errors? Are they able to identify errors?
• If they find errors, do they have a channel for pointing out or requesting that the errors be corrected? Do they do so?
• Do they know if the errors they point out are corrected? Do they have an opportunity to re-review the data to determine if the errors they pointed out are corrected, and do they make such a review?

Keep in mind that our reliability test is not whether the data are absolutely accurate, but that the data present a picture that is not significantly different from reality.