Measuring Information

Technology Performance
Introduction
 To evaluate an investment it must be
measured in some way for comparison.
 This chapter is focused on introducing a
variety of theories and methods for the
evaluation of IT investments.
The Economics of Information
 Economics is the study of the allocation of
scarce resources.
 Economics provides tools to analyze, evaluate,
and select among competing alternatives.
 IT investment decision-making is an economic
decision.
 Scarce resources must be allocated in an
optimal way, so as to select the best IT
investment alternative.
The Economics of Information
The Economics of Information refers to a
systematic series of concepts and theories that
explain the role which information and IT play to
assist an organization with product and service
design, development, manufacture, and delivery.

The economics of information presents rules

with which to assess the expected and actual
impact of investing in a particular IT
The Economics of Information
Transaction cost B
theory is based b Transaction
Transaction
on the logic that costs
cost reduction
occurs due to
the firm seeks to IT investment
economize on
transaction costs
A

C
As we can see in the
figure a reduction in
the transaction costs
from B to b, reduces
the size of the firm c a Size of
from a to c the firm
The Economics of Information
Agency theory
Agency
suggests that as a Agency costs cost reduction B
firm grows the firm occurs due to
has to increase its IT investment
employees who act b
as agents for the
owners. By investing
in IT it improves A
owner control,
saving time and
reduces costs,
mainly employee C
costs.

c a Size of
As we can see in the figure a reduction in the firm
the agency costs from B to b, reduces
the size of the firm from a to c
Why Measure IT Performance?
 Simply, to evaluate the functioning of an IT
investment. Measurement is vital to fully
understand the impact of IT investment on
overall organizational performance.
 Performance measurement allows decision-
makers to assess the business value, the
efficiency, and effectiveness of an IT
Why Measure IT Performance?

 A performance measurement system

evaluates the effects of an IT and may be used
to justify an initial IT investment, later to
assess its impact after implementation and
use.
Measures of IT Business Value
Business value of IT maybe defined as the overall value
of IT for a particular organization.

Assessing business value of IT attempts to provide

insights into the effects IT investments have on the
bottom line performance of an organization

IT investments may contribute to overall organizational

performance by improving
-Financial Performance
-Business Performance
-Strategic Performance
Measures of IT Business Value
The first step to determine business value of IT is to
identify the objectives of IT investments.

The next step is to select at least one measure to assess

each objective. There are many measures to choose
from and more than one is better.
Financial Performance Measures of IT
Ratios Calculation Explanation

Profitability
measures
Return on Income available to common Measures profitability of the
equity shareholders from continuing investment to the owners
operations divided by common
shareholder’s equity

Return on Income available to common Measures profitability and

assets shareholders from continuing how efficiently assets were
operations divided by average total utilized
assets

Return on Income available to common Measure profitability based

investment shareholders from continuing on total investment, both
operations divided by total invested debt and equity
capital
Financial Performance Measures of IT
Ratios Calculation Explanation

Profitability
measures
Return on Income available to common Measures profitability based
sales shareholders from continuing on sales
operations divided by net sales

Earnings per Total earning divided by total Measures earnings per share
share shares outstanding value

Revenue Revenue for the current period Measures growth in revenue

growth minus revenue from the prior over the prior period
period divided by revenue from
the prior period
Financial Performance Measures of IT
Ratios Calculation Explanation

Efficiency
measures
Sales by total Net sales divided by Measures how efficiently
assets average total assets assets were used to generate
sales

Sales by Net sales divided by Measures sales ability

employee number of employees effectiveness

Inventory Cost of goods sold divided Measures the liquidity of

turnover by average inventory inventory and how fast it is
sold
Business Performance Measures
of IT
 Business performance is the result of the
execution of business processes and the
allocation of resources to business
processes.
 Business performance depends on how
well a business allocates its resources to
business processes and how well it
performs its business processes.
Business Performance Measures
of IT
 The “balanced scorecard” may be a very
practical way to measure the business
performance of IT and incorporate both
financial and non-financial measures.
 The balanced scorecard method is a
performance measurement tool.
Business Performance Measures of IT
 The balanced scorecard provides a balanced
representation of financial and nonfinancial
performance through four perspectives:
 (1) customer perspective
 (2) internal business perspective
 (3) innovation and learning perspective
 (4) financial perspective
 Objectives (goals) are identified for each
perspective and then performance measures
are selected to measure each objective.
Business performance measures from the customers
perspective
Objective Examples of measures

Cost effectiveness Processing cost per some unit of

measure
Cost effectiveness Data center costs per workstation

Delivery speed Number of time periods to deliver

product
Support availability Mean-time-between-failures

On-time delivery Number of times actually met delivery

schedule

Defective products/services Number of defects

Training support Percent of user groups training is held

regularly

Responsive support Number of time periods to fix problem

Business performance measures from the internal
perspective

Objective Examples of measures

Efficient development Number of staff days per project by

size
Quality development Number of defects in test unit

Low levels of rework Number of times software was

reworked
Competent employees Number of years by job class

Highly skilled employees Number of hours of training per

employee
Business performance measures from the innovation
and learning perspective
Objective Examples of measures
Highly skilled employees Number of training days per time
period
Protect knowledge based on Number of employee resignations
employees
Increased market offerings Number of new product introductions
per time period
Encourage innovative research Percent of budget spent on IT
research
Improve innovative thinking Number of employee ideas
implemented
Foster innovative research Number of state of the art projects per
time periods
Business performance measures from the financial
perspective
Objectives Examples of measures

Profitable business Revenue growth

Profitable business Return on sales or assets

Profitable use of IT Revenue per user group

Profitable IT development Profit per employee

Efficient maintenance Cost of IT maintenance as a percent

of some unit of measure

Reliable planners Adherence to budget

Strategic Performance Measures
of IT
 IT investments may contribute to overall
organizational performance is through the
improvement of the strategic performance of
an organization.
 Strategic performance depends largely on a
few key areas that the organization must
excel at to survive and thrive and these areas
are called critical success factors (CSF).
Costs of IT
 To assess the business value of IT,
information must be collected as to the
actual amount spent on IT.
 All IT costs must be collected to determine
the true costs of IT to be used in
determining the business value of IT.
Costs of IT

Costs by Activity Costs by Resource

1. Development costs 1. Technology costs

2. Maintenance costs 2. Personnel costs

3. Operating costs 3. Outside services

costs

4. User support costs 4. Other costs

5. Administration and other

costs
Measures of IT Effectiveness Value
 IT effectiveness value may be defined as the
extent to which IT supports business processes
and employees.
 To determine the effectiveness of IT one may
evaluate the following three aspects of a business:
 (1) the extent to which IT supports business processes;
 (2) the extent to which IT supports employees;
 (3) the extent to which the IT sourcing function meets
business requirements.
Measures of IT Effectiveness
Supporting Business Processes
 To evaluate the extent to which IT supports
business processes, one should first identify
the business processes performed by the
organization and the different types of IT.
 A team of employees and management may
identify and list the different business
processes the organization uses to create
and deliver their products/services.
Continue…

 After identifying the business processes, a

team of employees and management
should identify the different categories of
IT. The IT categorization may be unique
to each individual organization.
Continue…
Major Types of IT
 Automational: eliminating human labor from a process.

 Informational: delivering information to customers as a

service or a product, and capturing process information

for purposes of management.
 Sequential: changing process sequence, or enabling

parallelism.
 Tracking: closely monitoring process status and
objects.
 •
Continue…
Major Types of IT
 Analytical: improving analysis of information and

decision making.
 Geographical: coordinating processes across
distances.
 Integrative: coordinating between tasks and
processes.
 Intellectual: capturing and distributing intellectual

assets.
 Disintermediating: eliminating intermediaries from

processes passing information.

Continue…
 These main types of IT can be mapped against
the different types of business processes,
resulting in a process/IT matrix and identify
the current IT application(s) in each cell of the
matrix.
 This allows the team of employees and
managers to determine which areas of the
business are supported by IT and which are not.
Continue…
 The following steps should be undertaken to assess
the extent to which IT supports business processes:
 1. Identify business processes;
 2. Identify types of IT;
 3. Create a matrix of business processes and types of IT
and identify the current IT application(s) for each cell in
the matrix
 4. Calculate the ratio of actual applications to potential
applications for each cell.
Measures of IT Effectiveness Supporting Business
Processes

IT IT Effectiveness IT Effectiveness
Effectiveness Criteria Measure
Factor
Actual application
Coverage of
of IT as a percent
Coverage management business
of potential per IT
processes
type
Actual application
Coverage of
of IT as a percent
Coverage operational business
of potential per IT
processes
type
Continue…
 The IT coverage of business processes is
just one way to assess the effectiveness of
IT.
 Organizations may select differing
appropriate IT effectiveness factors and
their corresponding IT effectiveness
criteria, then use them to devise
effectiveness measures.
Effectiveness Measures for IT to
Support Employees
 The first step here is to identify “IT
capabilities”. IT capabilities are everything
and anything an IT can accomplish.
 Again, a team of employees and managers
will identify IT capabilities.
 The second step is to define effectiveness
criteria and effectiveness measures for
each IT capability.
IT effectiveness criteria from
Employees Support perspective
 Reliability of IT applications: degree to
which IT applications are available when
needed, output is received according to
schedule, and availability problems are
quickly corrected.
 Reliability of information: degree of
correctness and integrity of the data
provided by IT applications.
IT effectiveness criteria from
Employees Support perspective
 Accessibility of information: promptness with
which information requested from IT applications
is received.
 Security of information: degree, to which data in
applications is protected from unauthorized
access.
 Ease of use: simplicity in using IT applications,
and the adequacy of outputs in any form, e.g.,
screen layouts, report formats, etc.
Effectiveness Measures for IT to Support Employees

IT Effectiveness IT Effectiveness Criteria IT Effectiveness Measure

Factor

Mean times between failures

Reliability of IT
Reliability and to repair; failures per
applications
100 hours of operation

Correct data as a percent of

Reliability Reliability of information
total data available

Accessibility of Mean response time, batch

Accessibility
information turnaround time
Effectiveness Measures for IT to Support Employees

IT Effectiveness IT Effectiveness Criteria IT Effectiveness Measure

Factor

Number of secured data sets

Security Security of information
as a percent of total

Time required to make

Flexibility Flexibility of IT
requested changes

User- User-friendliness rate on a

Ease of use
friendliness ratio scale