Category Archives: Data

Key Performance (mis) Indicators

Posted on by

2

kpi

Key Performance Indicators are meant to keep an organisation on track. By measuring the performance over time, you are able to look at deviations and to take measures. As Wikipedia defines it: A  key performance indicator (KPI) is a type of performance measurement. An organization may use KPIs to evaluate its success, or to evaluate the success of a particular activity in which it is engaged. Sometimes success is defined in terms of making progress toward strategic goals, but often success is simply the repeated, periodic achievement of some level of operational goal (e.g. zero defects, 10/10 customer satisfaction, etc.).

The concept behind Key Performance Indicators is to build a feedback loop between input and output. Its working principle does not differ from a thermostat, which senses the temperature of a system so that the system’s temperature is maintained near a desired set-point.

500px-Feedback_loop_with_descriptions_svg

In order to get not market driven organisations more efficient the adage “The numbers tell the tale”has become fashion among governments, institutions and not-for-profit companies. There are several metrics or key performance indicators.

However, Key Performance Indicators can also lead to perverse incentives and unintended consequences as a result of employees working to the specific measurements at the expense of the actual quality or value of their work. In the social sciences, unintended consequences (sometimes unanticipated consequences or unforeseen consequences) are outcomes that are not the ones intended by a purposeful action. Perverse incentives are a type of unintended consequence. A perverse incentive is an incentive that has an unintended and undesirable result which is contrary to the interests of the incentive makers.

There are a lot of examples of bad designed Key Performance Indicators. We came across, but not exhaustive:

  • Police officers get a predetermined quota of fines to give out. The unintended effect of this KPI that the police organisation will be focused on easy to obtain files, f.i. traffic fines instead of fighting serious crime;
  • An organisation involved in handling objections has a KPI for the amount of rejected complaints. Imagine how employees will approach complaints. . .
  • It is generally accepted that the progress of students is evaluated by tests. However, student tests assess only a small part of needed knowledge, skills and attitude of students. Also, often the purpose of the test, timely warning of learning difficulties and study delays, dilutes to “a (missed) ticket to the next hurdle”;
  • An agency of child protection is responsible for placing abused or emotional neglected children in foster parents and child care institutions. It is very logic to design a KPI: like the number of placed children. If this performance is coupled to the financing of the agency, it can easily lead  to placing children out of their home, against sound indications that there is no need for or against parents objections;
  • It is complete reasonable to expect higher efficiency and experience of surgeons as a hospital performs at least 30 knee surgery or angioplasty a year. However, such a KPI can lead to more instead of less knee surgery and angioplasty, an example of a perverse effect contrary to what was originally intended (an intended solution makes a problem worse);
  •  The selling of mortgages as an end in itself, even to people who could no pay the interest, led to the bank crisis in 2008. Another example of a negative, unexpected detriment occurring in addition to the desired effect of the policy to motivate sellers to do better their best.
  • To increase the efficiency of university studies universities are judged on the number of successful students per year. It is now tempting to reduce the requirements for passing exams.
  • In order to increase the efficiency of General Practitioners many assurance companies allow for not more than ten minutes consults by patients. This KPI leads to far more referrals to medical specialists because GP’s have not much time to carefully investigate the medical complaints. This is an example of a counterproductive KPI: it is more of an “obstacle” than a help in the achieving of a productive project or an objective;
  • Crews of warships run annual series of nautical and operational exercises. Through a complex multi-factor analysis, a KPI is derived: Operational Employ-ability. Members of Parliaments asked questions when the KPI decreased to 10%, as a warship was actually deployed in a crisis;
  • Notorious are budgets: the setting of expenditure levels for each of an organization’s functions. It expresses strategic plans of business units, organizations, activities or events in measurable terms. However, such budget tends to be exhausted at the end of the year, because organizational units realise that they will be shortened in budget for next year, because last year they needed not the full budget. So, as an example, in many towns you can observe that every five to ten years the same streets and squares are completely overhauled without any need but in order to use the full budget.

Many Key Performance Indicators have unintended effects. They function as rules for behavior. Key performance Indicators are designed to notice need for adjustments of the course of an organisation. However, more often than not, they are invitations to cheat, by employees but equally by companies and institutions,  especially when financial consequences are attached to the KPI.

Whenever designing or encountering a Key Performance Indication, be warned!

 

unintended consequences

For more examples of perverse incentives, see here. For examples of unintended consequences see here.

To built up your Thinkibility skills, imagine your are the director of a hospice. You have set a thinking task: how to improve the occupancy (KPI) of the hospice. Then check your answers with How Dying Became A Multibillion-Dollar Industry.

 

 

Creative Data Collection

Posted on by

2

Data collection is the process of gathering and  measuring variables of interest, in an established systematic fashion that enables you to answer stated research questions, test hypotheses, and evaluate outcomes. The goal for all data collection is to capture quality evidence that then translates to rich data analysis and allows the building of a convincing and credible answers to questions that have been posed.

flats_1742028c

However, many research and policy are based on already available information that do  not necessarily fit to the issue or question. For instance, the average income of residents in a district is estimated at the average market value of homes. Suppose there are some big villas in a district. However,  the houses are rented to migrant workers who sleep with twelve persons in a room. Actually, the district has poor inhabitants and impoverishment is lurking. A better measure of prosperity in a district could be the amount of call shops, Travelex foreign exchange bureaus or percentage windows with bed sheets as curtains or without curtains at all. 

Perhaps we could use data of the local supermarkets to assess the risk of impoverish of a district, not only by sales volume, but also the kind of products consumers buy. Income groups differ in the food they buy. Supermarkets may have more or less variety in products they offer, regarding the population.

supermarket

This could lead to a Reverse Open Data movement, where businesses make their data available for town halls to design social and economic policies. Open Data is a movement that  open data should enable third parties to leverage the potential of government data through the development of applications and services that address public and private demands. This information exchange could be made two-sided.

Perhaps we could measure the level of civilization of a nation by registering the percentage of cars that don’t stop for a pedestrian at a pedestrian crossing (a zebra crossing).

big mac

Another example is the Big Mac Index, a creative alternative to determine exchange rates, the rate at which one currency will be exchanged for another. It is based on the concept that a Big Mac is highly standardized all over the world. The difference in selling price in country A compared with the selling price of a Big Mac in country B gives a better idea of the value of the two currencies relative to each other.

brands

Maybe we can say something about the tendency to conformity by counting people who wear clothes of a specific brand. Or determine the percentage women that is involved in street sweeping.

Cleaner-Oct06-D3249sAR

It’s hard to get solid data on drug usage, because it’s traditionally gathered via questionnaires. Respondents can fudge the answers or forget details. Drug users also sometimes don’t know what they are really taking or whether other drugs are mixed in. However, the laboratory analysis of waste water has the potential to get more accurate results more quickly, as a recent study showed that cities’ sewer water exposes use of cocaine, cannabis, meth and ecstasy.

sewer

It is interesting to roam in a town and create some mini theories that explain what is observed, compared with another town. For instance:

  • There are substantially more gulls and open torn garbage bags ;
  • Advertising is everywhere and very blatant;
  • There is a lot of green space and many squares;
  • All doors and window frames are painted in the same color;
  • There are many pizzerias.

What does it mean? Or what does it explain? Can you quantify the phenomena and turn them into meaningful data?

 

Making Plans More Engaging – Thinkibility Nibble

Posted on by

0

Many organisations and many people struggle with implementing plans, strategies or intentions. As a Thinkibility nibble we will hypothesize here that it is caused by boring presentations, but also by neglecting the planning process with all stakeholders.

process

Planning is often an interactive process

We will put forward the idea that, although most planning tools use visuals, they are still boring. Look at these examples of a Gantt-chart

planning

or a PERT-diagram:

pert planning

Or something like this:

marketing-plan-strategy

But what would happen if you use a map of the to set up base camps at the Mount Everest?

everest base camp

Or reenacting Shackleton’s dramatic journey:

shackleton

Or to make your own map of dependencies like a Metro map?

metro map

Or Flight Plans:

flight chart

What if you use for maintenance planning the map of the Versailles?

versaille

For planning acquisitions and mergers you could use colonization maps:

colonization

Another idea is to go in the Third Interactivity Dimension, by building a planning town with paper, wood or Lego:

3D planning

See also the earlier post Life Redefined, about possible designs for a monthly planner by using known board games.

Colorful and creative planning!

Creative Ways of Using and Storing Data

Posted on by

0

In the future, more or less unlimited data storage capacity may be a possibility. The capacity to store huge amount of data is also increasingly becoming cheaper. At the same time, new exciting ways of storing and using data are emerging. Some of these methods and techniques involve moving away from electronic solutions to using cells and crystals to store data. Creative ways of displaying and using data are also appearing such as 3D and info graphs.

When discussing information several different perspectives can be explored. Who is going to use the information? Who has access to the information? Who owns the information? Some of the changes in the way data are stored, displayed, retrieved may lead to changes in the way we look upon ownership. Below is a list of some ideas that are currently being explored.

Data Storing Devices – New ways to increase the capacity and the reduce the storage space

Data and Information can be collected from

  • Our daily activities – human interactions and communications
  • Communications from electronic devices that we use such as computers and mobile phones can be collected
  • Inside our body
  • From our mind or brain
  • From other species
  • From devices we touch like doors, tables, seats
  • From the devices we use . like clothing, shoes, furniture, kitchen equipment, cars, bicycles, mobiles, laptops, sport equipment.

Ways of displaying Information and Data

In the video below, you can see how different world like Paris can be explored using Google Street map. Details in design of buildings and streets can be compared from different cities such as Prague and Barcelona.  Images can provide a cultural historical value. A possible way to explore cultural interaction between the different countries.

In the next blog post, we will explore some consequences of these ways of using and storing data.  How does our perception of data change when live cells are used to store date? What does our ideas about data change when we see a 3D picture of it? And what does it mean for your identity that more or less your whole lives interaction can be recorded and stored? What are the consequences of the idea that in future you will pay with data?

Photo: “Ideas Button” by Stuart Miles